Multi-target scene instance separation processing method based on visual identification

Abstract

The invention discloses a multi-target scene instance separation processing method based on visual identification, and relates to the technical field of image processing, and the method comprises the steps: carrying out the feature extraction of an input image, generating a plurality of rectangular candidate bounding boxes through a target detection algorithm based on a deep neural network, and extracting a local region image from each rectangular bounding box, according to boundary fitting of the local region image, obtaining a real boundary curve of the corresponding target; fitting result features of the real boundary curve of the target are analyzed, and a boundary extraction qualified value is calculated; judging a fitting result state of a real boundary curve of the target according to the boundary extraction qualified value, and outputting a real boundary of the target; thus, the target scene instances can be separated, computing resources are reduced, whether the boundary result of the sorted target is qualified or not can be judged, boundary refining and optimization are automatically carried out, the target separation effect is good, the recognition precision is high, and the computing resource cost is saved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method for separating and processing multiple target scene instances based on visual recognition. Background Technology

[0002] With the rapid development of computer vision and deep learning technologies, object detection and image segmentation have become two core tasks in the field of visual recognition. Existing object detection methods, such as Faster R-CNN, SSD, YOLO, and EAST, typically use convolutional neural networks to extract features from the input image and predict rectangular or rotated rectangular bounding boxes on the feature map to represent the approximate location of the target. These methods have good detection efficiency in conventional scenarios and can achieve rapid localization of multiple targets.

[0003] However, in complex natural scenes, such as street view photos, billboards, packaging printing, medical images, and aerial maps, target shapes often exhibit curved, tilted, irregular, or densely arranged characteristics, making it impossible for traditional rectangular detection boxes to accurately describe the true boundaries of the targets. For example, text information may be distributed along arcs, waves, or free curves; the boundaries of industrial parts may have unevenness; and structures in medical images often present complex contours. In such cases, rectangular or fixed-shape detection boxes can only provide a rough location and cannot accurately depict boundary details. While traditional methods that directly perform pixel-level segmentation can provide high-precision boundaries, they are computationally burdensome, have slow inference speeds, and are difficult to meet real-time requirements, especially on resource-constrained devices. Furthermore, traditional methods lack effective mechanisms for quality assessment and correction of the boundaries of each target scene instance, making it impossible to determine whether the boundary results are acceptable or to automatically refine and optimize the boundaries. This results in poor target separation performance, low recognition accuracy, and a significant waste of computational resources. Summary of the Invention

[0004] The purpose of this invention is to solve the problems mentioned above and provide a method for separating and processing multi-target scene instances based on visual recognition.

[0005] In a first aspect of this invention, a method for separating multi-target scene instances based on visual recognition is first proposed, the method comprising:

[0006] Next steps:

[0007] Feature extraction is performed on the input image, and multiple rectangular candidate bounding boxes are generated using a target detection algorithm based on a deep neural network. The rectangular candidate bounding boxes are used to represent the location of the target in the image, and multiple sets of rectangular bounding boxes are generated.

[0008] Extract local region images from each rectangular bounding box, and fit the boundaries of the local region images to obtain the true boundary curve of the corresponding target;

[0009] The fitting results of the target's true boundary curve are analyzed, and the boundary extraction qualification value is calculated.

[0010] Based on the qualified value extracted from the boundary, the fitting result of the true boundary curve of the target is determined, and the true boundary of the target is output.

[0011] Optionally, the step of generating multiple rectangular candidate bounding boxes using a deep neural network-based object detection algorithm includes:

[0012] The input image is processed by a convolutional neural network to extract features and generate multi-scale feature maps.

[0013] Anchor points are generated by sliding a window on the feature map. The probability of the target presence at each anchor point and the boundary regression offset relative to the anchor point are predicted by target classification and boundary regression, respectively.

[0014] Using the target's existing classification probability and boundary regression offset, calculate the rectangular bounding box corresponding to each anchor point, perform non-maximum suppression on the rectangular bounding boxes to remove duplicate boxes and retain boxes with high confidence.

[0015] Output the final set of candidate rectangular bounding boxes.

[0016] Optionally, the step of extracting local region images from each rectangular bounding box and fitting the true boundary curve of the potential target based on the boundaries of the local region images includes:

[0017] Local region images are extracted from each rectangular bounding box, and features are extracted from the local region images using a convolutional neural network to generate local feature maps;

[0018] On the extracted feature map, pixel aggregation and separation operations with instance-aware constraints are performed. Instance cohesion constraints ensure that the pixel features of the same target instance are clustered together in the feature space, making the pixel features within the same target instance consistent. Instance separation constraints ensure that the pixel features of different target instances are separated from each other, avoiding feature mixing or boundary overlap between different targets.

[0019] After constraint optimization, a mask for the target instance is generated. Based on the generated instance mask, the boundary points in the mask are extracted. The extracted boundary point set is fitted by a curve fitting method to generate the true boundary curve of the target.

[0020] The fitted boundary curves are smoothed to obtain the true boundary curves; and the true boundary curves corresponding to each rectangular bounding box are output.

[0021] Optionally, the steps for analyzing the fitting results of the target's true boundary curve and calculating the boundary extraction qualification value are as follows:

[0022] The fitting result features include boundary topological stability value and information propagation consistency value. The boundary topological stability value and information propagation consistency value are added together to obtain the boundary extraction qualification value.

[0023] Optionally, the calculation steps for the boundary topological stability value are as follows:

[0024] The true boundary curve of the target is divided into multiple boundary points, and the angle of each boundary point in the boundary point set is calculated.

[0025] Calculate the absolute difference in angles between adjacent boundary points, and use this as the angle difference between two adjacent boundary points;

[0026] Calculate the Euclidean distance between two adjacent boundary points; divide the angular difference between two adjacent boundary points by the Euclidean distance, and use the result of the division as the first ratio value. The mean of all first ratio values ​​is used as the boundary adaptive rate of change.

[0027] Calculate the curvature at each boundary point, calculate the mean of all curvatures, calculate the absolute difference between each curvature and the mean curvature, and record the mean of all absolute differences as the boundary morphology distortion.

[0028] The boundary topological stability value is calculated based on the boundary adaptive change rate and the boundary morphological distortion.

[0029] Optionally, the steps for calculating the boundary topological stability value based on the boundary adaptive rate of change and the boundary morphological distortion are as follows:

[0030] Calculate the sum of the boundary adaptive change rate, the boundary morphological distortion, and the value of 1, and use the reciprocal of the sum as the boundary topological stability value.

[0031] Optionally, the steps for calculating the information propagation consistency value are as follows:

[0032] The true boundary curve of the target is divided into multiple boundary points, forming a boundary point set. ;in Boundary point index, boundary point Represents the coordinates of each pixel on the target boundary;

[0033] For each boundary point Calculate the local orientation angle of the corresponding image information. , In the formula, Indicates the first A boundary point, This represents the grayscale value of the image at that location; , These represent the horizontal orientation of the image. and longitudinal Partial derivative of direction; Point The local orientation angle corresponding to the image information.

[0034] Optionally, the calculation step of the information propagation consistency value further includes:

[0035] For each pair of adjacent boundary points , Calculate its local directional jump value , And calculate the jump intensity ratio based on the jump values ​​in all directions. The information direction consistency factor is calculated by adding the reciprocal of the value of 1 to the jump intensity ratio; the formula is as follows: In the formula, Indicates the first Local directional jump value Indicates the jump intensity ratio;

[0036] For each point Constructing the local image information diffusion tensor And calculate the Frobenius norm of the diffusion tensor between adjacent points as the perturbation amplitude. ;

[0037] Based on all disturbance amplitudes Calculate the diffusion tensor perturbation ratio The diffusion uniformity factor is calculated by adding the perturbation ratio of the diffusion tensor to the reciprocal of the value of 1; the formula is as follows: , Indicates the first The amplitude of the disturbance;

[0038] Calculate the local directional jump value between the starting point and the ending point of the boundary as the first and last information directional difference, and add the reciprocal of the value of 1 to the first and last information directional difference as the return consistency factor.

[0039] Information direction consistency factor, diffusion consistency factor, and backhaul consistency factor Multiply them to obtain the consistency value of information dissemination.

[0040] Optionally, the steps of determining the fitting result of the target's true boundary curve based on the qualified value extracted from the boundary, and outputting the target's true boundary, include:

[0041] The qualified value of the boundary extracted from the true boundary curve of each target is compared with the preset threshold. If the qualified value of the boundary extracted is not less than the preset threshold, it means that the true boundary curve of the target is qualified, and the true boundary of the corresponding target is directly output.

[0042] Optionally, the step of determining the fitting result state of the target's true boundary curve based on the qualified value extracted from the boundary, and outputting the target's true boundary, further includes:

[0043] If the qualified value of the boundary extraction is less than the preset threshold, it means that the true boundary curve of the target is unqualified. Then, the boundary curve of the corresponding target is extracted again until the true boundary curve of the target is qualified, and the true boundary of the corresponding target is output.

[0044] The beneficial effects of this invention are:

[0045] This invention proposes a multi-target scene instance separation processing method based on visual recognition. It extracts features from the input image and generates multiple rectangular candidate bounding boxes using a deep neural network-based target detection algorithm. These rectangular candidate bounding boxes represent the positions of targets in the image, generating a set of multiple rectangular bounding boxes. Local region images are extracted from each rectangular bounding box, and the true boundary curve of the corresponding target is obtained by fitting the boundary of the local region image. The features of the fitting result of the true boundary curve of the target are analyzed, and a boundary extraction qualification value is calculated. Based on the boundary extraction qualification value, the fitting result state of the true boundary curve of the target is judged, and the true boundary of the target is output. In this way, each target scene instance can be separated, reducing computational resources while also judging whether the boundary result of the processed target is qualified, and automatically refining and optimizing the boundary, resulting in better target separation effect, higher recognition accuracy, and reduced computational resource costs. Attached Figure Description

[0046] The invention will now be further described with reference to the accompanying drawings.

[0047] Figure 1 This is a flowchart of a multi-target scene instance separation processing method based on visual recognition. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] This invention provides a method for separating multiple target scene instances based on visual recognition. See also... Figure 1 , Figure 1 A flowchart illustrating a multi-target scene instance separation processing method based on visual recognition provided in an embodiment of the present invention. The method includes the following steps:

[0050] S1: Extract features from the input image and generate multiple rectangular candidate bounding boxes using a target detection algorithm based on a deep neural network. The rectangular candidate bounding boxes are used to represent the location of the target in the image, and multiple sets of rectangular bounding boxes are generated.

[0051] S2: Extract local region images from each rectangular bounding box, and fit the boundaries of the local region images to obtain the true boundary curve of the corresponding target;

[0052] S3: Analyze the fitting results of the target's true boundary curve and calculate the boundary extraction qualification value;

[0053] S4: Based on the qualified value extracted from the boundary, determine the fitting result of the target's true boundary curve and output the target's true boundary.

[0054] Based on the visual recognition-based multi-target scene instance separation processing method provided in this embodiment of the invention, each target scene instance can be separated in the above manner, reducing computing resources while also judging whether the boundary results of the separated targets are qualified, and automatically refining and optimizing the boundaries, resulting in better target separation effect, higher recognition accuracy, and saving computing resource costs.

[0055] In one embodiment, S1: Feature extraction is performed on the input image, and multiple rectangular candidate bounding boxes are generated using a target detection algorithm based on a deep neural network. The rectangular candidate bounding boxes are used to represent the positions of potential targets in the image, and multiple sets of rectangular bounding boxes are generated.

[0056] In one implementation, the step of generating multiple rectangular candidate bounding boxes using a deep neural network-based object detection algorithm includes:

[0057] The input image is processed by a convolutional neural network (CNN) to extract features and generate multi-scale feature maps.

[0058] Anchor points are generated by sliding a window on the feature map. The probability of the target presence at each anchor point and the boundary regression offset relative to the anchor point are predicted by target classification and boundary regression, respectively. Each anchor point represents a possible target region.

[0059] Using the target classification probability and boundary regression offset, the rectangular bounding box corresponding to each anchor point is calculated. Non-maximum suppression is then performed on the rectangular bounding boxes to remove duplicate boxes and retain boxes with high confidence. Non-maximum suppression can effectively remove boxes with large overlap, reduce redundant information, and ensure the accuracy of the detection boxes.

[0060] The final set of rectangular candidate bounding boxes is output, which contains all the filtered target regions in the image, providing input for subsequent target instance separation and boundary fitting.

[0061] It should be noted that the steps for generating multiple rectangular candidate bounding boxes using a deep neural network-based object detection algorithm include: First, the input image undergoes feature extraction using a convolutional neural network (CNN) to generate multi-scale feature maps. The feature extraction process utilizes operations such as convolutional layers and pooling layers in the deep neural network to extract various visual features from the image, such as edges, textures, and color distribution. These feature maps contain image information at different levels, enabling the network to identify targets from coarse to fine detail.

[0062] On the generated feature map, the system employs a sliding window mechanism to generate a series of candidate anchor points. These anchor points represent the possible locations and sizes of targets in the image. Each anchor point undergoes target existence prediction through a target classification network, yielding a confidence score (probability of target presence) for that anchor point. Simultaneously, a boundary regression network predicts the offset of each anchor point relative to the true boundary of the target through regression operations. Specifically, the regression offset includes the offset of the anchor point's center position. Adjustment of anchor point size These predictions were used to adjust the anchor points to more accurate target bounding boxes.

[0063] Using target classification probability and boundary regression offset value To calculate the rectangular bounding box corresponding to each anchor point The calculation process typically uses the following formula:

[0064]

[0065] in, These are the center coordinates of the anchor point. , It refers to the width and height of the anchor point. These are the offsets from the regression predictions; through these predictions, the precise bounding boxes of each candidate region can be obtained.

[0066] To remove redundant candidate boxes and improve detection accuracy, the Non-Maximum Suppression (NMS) algorithm is used. The NMS operation calculates the degree of overlap between each candidate box and other boxes (usually using the Intersection over Union (IoU) as a metric), removes candidate boxes with high overlap, and retains the boxes with the highest confidence. The Non-Maximum Suppression algorithm can effectively eliminate redundant candidate boxes located in the same target region, retaining only boxes with high confidence and low overlap, thus reducing detection errors in overlapping regions.

[0067] Set of rectangular candidate bounding boxes after NMS operation This is the final output result; this set contains all filtered target regions in the image with high confidence, providing input for subsequent instance separation and boundary fitting. These candidate bounding boxes can not only accurately locate the target's position, but also provide clear regions for subsequent refined boundary fitting and target instance separation.

[0068] For example, in a street view image, multiple candidate bounding boxes may be detected, such as regions for "pedestrians," "vehicles," and "road signs." Each bounding box represents a potential target. The Non-Maximum Suppression (NMS) operation removes overlapping boxes, leaving only high-confidence target bounding boxes. This provides candidate target regions for subsequent instance-aware constraint-guided curve fitting steps. Through this series of steps, this invention can quickly and efficiently generate an accurate set of target bounding boxes, providing reliable preliminary results for subsequent high-precision target instance separation and boundary fitting.

[0069] In one embodiment, S2: The step of extracting local region images from each rectangular bounding box and fitting the true boundary curve of the potential target based on the boundaries of the local region images includes:

[0070] Local region images are extracted from each rectangular bounding box, and features are extracted from these local region images using a convolutional neural network to generate local feature maps. This process uses multi-layer convolution operations, pooling operations, and activation functions to obtain texture, edge, and semantic information from the images, providing rich visual features for subsequent instance perception and target separation.

[0071] On the extracted feature map, pixel aggregation and separation operations with instance-aware constraints are performed. Instance cohesion constraints ensure that the pixel features of the same target instance are clustered together in the feature space, making the pixel features within the same target instance consistent. Instance separation constraints ensure that the pixel features of different target instances are separated from each other, avoiding feature mixing or boundary overlap between different targets.

[0072] After constraint optimization, a mask for the target instance is generated. The mask contains pixel information within the target region, representing the probability that each pixel belongs to the target instance. This instance mask clearly marks the region where the target instance is located in the image, providing accurate target region data for subsequent boundary fitting.

[0073] Based on the generated instance mask, boundary pixels are extracted from the mask to represent the contour of the target instance. Boundary points are extracted using edge detection operators (such as the Canny operator) or other gradient thresholding methods. The extracted boundary point set is then fitted using curve fitting methods (such as Bézier curves or B-spline curves) to generate the true boundary curve of the target. This fitting process is optimized to ensure that the target boundary is smooth and continuous, enabling accurate reconstruction of the shape of the target instance.

[0074] The fitted boundary curves are smoothed to eliminate jagged edges or discontinuities caused by noise or pixel errors. The continuity and accuracy of the boundary curves are further optimized using Bézier curve smoothing or higher-order curve fitting methods, ensuring that the generated boundary curves conform to the true shape of the target.

[0075] Output the true boundary curves corresponding to each rectangular bounding box. The curves accurately describe the shape of each target instance and ensure the smoothness and continuity of the boundaries. The generated boundary curves not only describe the shape of the target but also provide accurate boundary representations for subsequent target instance separation, merging, or other follow-up tasks.

[0076] Specifically as follows:

[0077] For each rectangular bounding box Local region image in Perform feature extraction and generate local feature mapping tensors. ;

[0078] Local region image From the rectangular bounding box The image region cropped from the image, the center coordinates of the region are Width is The height is Feature extraction is achieved through convolutional neural networks (CNNs) or other deep neural network models, typically including operations such as convolutional layers, activation functions, and pooling layers.

[0079] The final local feature mapping tensor The size is ,in: and These are the height and width of the output feature map; It is the number of channels in the feature map, usually a high-dimensional feature space such as 256 or 512;

[0080] In the feature map tensor Above, perform instance-aware constraint pixel aggregation and instance separation operations; instance cohesion constraint To ensure that pixel features of the same instance are similar, the definition is: ; Indicates the first A set of pixels for each target instance; Represents pixels The feature vectors are taken from the feature mapping tensor. ; Indicates the first The feature center of each target instance, i.e., the feature mean of all pixels in that instance: This constraint ensures that pixel features within the same target remain consistent by minimizing the distance between the pixel and the instance center. Represents pixels Feature vectors and instance centers The Euclidean distance between them;

[0081] Inter-instance separation constraints Ensuring that the feature differences between different instances are maximized is defined as: ; and They are the first and the Feature centers of each target instance; Indicates the first and the Euclidean distance between the feature centers of each target instance; The minimum separation distance is used to avoid feature overlap between different target instances; this constraint ensures that the features of target instances do not overlap or become confused by maximizing the distance between different target instances.

[0082] Through these constraints, the network can effectively aggregate the pixel features of target instances. Pixel features within the same target instance tend to be consistent, while pixel features between different target instances maintain sufficient separation, thereby generating an optimized instance mask. ;

[0083] Based on the generated instance mask The target boundary is fitted to obtain the true boundary curve of the target. ;

[0084] From instance mask Extract the boundary pixel set These boundary points are located at the edges of the mask and represent the contour of the target; boundary points can be extracted using edge detection operators (such as the Canny operator) or gradient thresholding methods.

[0085] Use curve fitting methods (such as Bézier curves, B-splines, etc.) to fit the boundary point set. By fitting the data, a continuous target boundary curve is obtained. ;in For curve parameters;

[0086] When using Bézier curve fitting, the curve can be represented as: In the formula, Indicates the first One boundary point; No. Bessel basis functions It is the order of the basis functions; The parameter represents a point on the curve. This fitting process can accurately reconstruct the boundary curve of the target, making the target boundary smooth and continuous.

[0087] For the fitted boundary curve Smoothing is performed to eliminate jagged edges or discontinuities at the boundaries caused by noise or pixel errors; by using Bézier curve smoothing optimization or higher-order curve fitting methods, the boundary curves can be further smoothed to ensure the continuity and accuracy of the boundaries.

[0088] Finally, output the bounding box of each rectangle. Corresponding true boundary curve These boundary curves precisely describe the shape of each target instance.

[0089] The generated boundary curves not only describe the shape of the target, but also ensure the smoothness and continuity of the boundary, providing an accurate boundary representation for subsequent target instance separation, merging, or other follow-up tasks.

[0090] In one embodiment, S3: The step of analyzing the fitting results of the target's true boundary curve and calculating the boundary extraction qualification value is as follows:

[0091] The fitting results feature includes boundary topological stability value and information propagation consistency value. The boundary topological stability value and information propagation consistency value are added together to obtain the qualified value for boundary extraction.

[0092] In one implementation, the steps for calculating the boundary topological stability value are as follows:

[0093] The true boundary curve of the target is divided into multiple boundary points, forming a boundary point set. ;in Boundary point index, boundary point Represents the coordinates of each pixel on the target boundary;

[0094] Calculate the angle of each boundary point in the boundary point set. , It also calculates the absolute difference in angles between adjacent boundary points, which is used as the angle difference value between two adjacent boundary points. ; In the formula, Indicates the first The coordinates of the nth boundary point represent the coordinates of the nth boundary point on the target boundary. The horizontal and vertical positions of each pixel; Indicates the first The angle of each boundary point Indicates the first The and the first The angular difference between boundary points measures the difference in the tangent direction between two adjacent points;

[0095] Calculate the Euclidean distance between two adjacent boundary points. , Divide the angular difference between any two adjacent boundary points by the Euclidean distance, and use the result of the division as the first ratio value. The average of all first ratio values ​​is used as the boundary adaptive rate of change. , It is used to measure the sensitivity of the boundary to changes between adjacent points. The smaller the value, the more stable the boundary change; the larger the value, the greater the boundary change and the poorer the stability.

[0096] Calculate the curvature at each boundary point The calculation formula is: And calculate the mean of all curvatures. And calculate each curvature and the mean curvature. The absolute difference is calculated, and the mean of all absolute differences is recorded as the boundary morphology distortion. ; High distortion means that the boundary has changed significantly and may not look natural, while low distortion indicates that the boundary shape is more stable and continuous.

[0097] The boundary adaptive change rate and the boundary morphological distortion are added together, and the reciprocal of the sum is taken as the boundary topological stability value. The calculation formula is: .

[0098] It should be noted that in the calculation of the boundary topological stability value, all boundary point data used for calculation (such as boundary coordinates, tangent direction angle, curvature, Euclidean distance, etc.) are derived from the pixel-by-pixel analytical results of the target boundary in the target image. This boundary can be extracted using image segmentation algorithms (such as Mask R-CNN, U-Net, ContourNet, etc.). After extracting the complete boundary, each pixel on the boundary contour is traversed, and its two-dimensional coordinate information is recorded to form a boundary point set. All subsequent angle calculations are based on continuous point pairs in this boundary point set: for example, the angle is calculated by solving the arctangent function of the coordinates of two adjacent points, the Euclidean distance is calculated from the coordinate difference between the two points, and the curvature is calculated based on the positional relationship of the point before and after to construct the local curvature. Since these data can be directly extracted from the image without relying on external annotation or manual intervention, they have good automated acquisition capabilities and reproducibility, and can be widely applied to various boundary structure analysis tasks.

[0099] It should be noted that the boundary topological stability value is a comprehensive indicator used to measure whether the target boundary has good fit and natural continuity in terms of geometry and spatial structure. It is mainly used to assess whether the current boundary fitting result maintains the continuity, smoothness, and topological rationality of the real target boundary. This value combines two dimensions: the angle change between adjacent points on the boundary (i.e., the severity of the boundary transition) and the spatial curvature fluctuation of the boundary (i.e., the stability of the boundary morphology). By calculating the sensitivity of the angle change between adjacent points on the boundary and the overall consistency of curvature, it reflects whether there are abrupt, unnatural oscillations, reversals, or discontinuities in the overall boundary. A larger value indicates that the boundary direction changes little between adjacent points, meaning the boundary is continuous and natural, without sharp corners or abnormal jumps. The curvature is also relatively stable, with similar curvature trends at each point. This suggests that the entire boundary curve is smooth, coherent, and regular at both the local and global levels, conforming to the natural contour of a real object. Conversely, a smaller value indicates irregular broken lines, fluctuations, sharp corners, or distortions during the boundary fitting process. These may be due to occlusion, noise, boundary detection errors, or overfitting, failing to accurately recreate the original shape of the target. For example, in actual image segmentation, for detecting the edge of a leaf, a smooth curve indicates high boundary topological stability and a good fit. Conversely, a smaller value indicates severe boundary distortion if the fitted boundary exhibits numerous jagged edges, broken lines, or abrupt corners. Therefore, a larger boundary topological stability value indicates that the current fitting result is closer to the natural boundary of the real target, with a more continuous and stable boundary curve structure. This is suitable for judging whether the fitting has reached the expected quality level in target detection, instance segmentation, or boundary restoration tasks.

[0100] It should be noted that calculating the boundary topological stability value using the above method has advantages such as high geometric intuitiveness, topological sensitivity, and strong adaptability. It can more comprehensively and accurately reflect the naturalness and stability of the target boundary fitting. Its design process integrates multiple dimensions of variation characteristics such as angle change, distance scale, and curvature fluctuation. First, at the boundary point level, the ratio between the angle change of adjacent points and the Euclidean distance is used to sensitively capture the deformation intensity of the boundary at local locations. The smaller this ratio, the smaller the actual profile disturbance amplitude even if there is an angle offset, resulting in a smoother fitting result that is closer to the real boundary structure. Second, the curvature change is introduced as an overall distortion index of the boundary structure. If the boundary curvature does not fluctuate much between points, it indicates that the fitted boundary maintains good continuity and natural transition, avoiding anomalous shapes such as sharp breaks and reversals, further enhancing the stability of the boundary. The stability assessment is based on a combined modeling of the sensitivity of boundary changes and the degree of curvature fluctuation, with the reciprocal of their sum used as the final stability value. This approach not only ensures the intuitive consistency of "the more stable, the larger the value" in numerical design but also possesses excellent numerical compression characteristics. It can significantly distinguish between boundaries with local perturbations but overall continuity and those with local mutations or overall instability, avoiding misjudgments caused by relying solely on curvature or error indicators. For example, in cases where some target boundaries have a few cusps but are overall continuous, traditional error methods might consider the fit unacceptable, while the above value, after considering both angle difference and curvature changes, still provides a high stability, demonstrating its robustness and fault tolerance. This makes the value highly practical and widely applicable in scenarios such as complex boundary morphology analysis, image segmentation fitting quality evaluation, and boundary correction feedback control.

[0101] In one implementation method, the formula for calculating the consistency value of information propagation is:

[0102] The true boundary curve of the target is divided into multiple boundary points, forming a boundary point set. ;in Boundary point index, boundary point Represents the coordinates of each pixel on the target boundary;

[0103] For each boundary point Calculate the local orientation angle of the corresponding image information. , In the formula, Indicates the first A boundary point, This represents the grayscale value of the image at that location; , These represent the horizontal orientation of the image. and longitudinal Partial derivative of direction; Point Local orientation angle corresponding to image information;

[0104] For each pair of adjacent boundary points , Calculate its local directional jump value , And calculate the jump intensity ratio based on the jump values ​​in all directions. The reciprocal of the value of 1 plus the jump intensity ratio is used as the information direction consistency factor. The calculation formula is: ; In the formula, Indicates the first Local directional jump value Indicates the jump intensity ratio;

[0105] For each point Constructing the local image information diffusion tensor , In the formula, Indicates at the boundary point At this point, the gray-level variance of the image in the horizontal direction; At the boundary point At this point, the covariance of the image in the horizontal and vertical directions represents the degree of coupling of information in these two directions; Indicates at the boundary point At this point, calculate the gray-level variance of the image in the vertical direction; and calculate the perturbation amplitude of the diffusion tensor between adjacent points. , : Indicates the first The point and the first The Frobenius norm of the diffusion tensor between points (i.e., the square root of the sum of squares of matrix elements) represents the magnitude of change in the local information structure.

[0106] Based on all disturbance amplitudes Calculate the diffusion tensor perturbation ratio The diffusion uniformity factor is obtained by adding the diffusion tensor perturbation ratio to the reciprocal of the value of 1. The calculation formula is: ; , Indicates the first The amplitude of the disturbance;

[0107] Calculate the starting point of the boundary and the end point The local directional jump value between them is used as the directional difference between the beginning and end information. The difference in direction between the first and last information, plus the reciprocal of the value of 1, is used as the return consistency factor. The calculation formula is: ;

[0108] Information direction consistency factor Diffusion Consistency Factor and return consistency factor Multiply them to obtain the consistency value of information dissemination.

[0109] It should be noted that in the calculation of the above information propagation consistency value, all required data comes from the pixel information of the true boundary region in the target image. By performing edge extraction on the image, the true boundary curve of the target can be obtained and divided into several continuous boundary points. The position of each boundary point is composed of its horizontal and vertical coordinates in the image. For each boundary point, the image grayscale value or depth feature value of the region is extracted within its local neighborhood window. Based on the grayscale change trend of the pixels in the horizontal and vertical directions in the local region, the partial derivative values ​​of the image in the horizontal and vertical directions are calculated to obtain the main direction angle of local information propagation. Furthermore, by calculating the change amplitude of the direction angle between two adjacent boundary points, the direction jump value sequence can be obtained, and the following can be derived. The system generates a directional consistency index. For each boundary point, the grayscale distribution is statistically analyzed within a fixed-size sliding window. The variance in the horizontal and vertical directions and the covariance between these two directions are calculated to construct the image diffusion tensor for that point. Subsequently, the perturbation amplitude is calculated using the difference matrix of the diffusion tensors between two adjacent points, forming a perturbation amplitude sequence and deriving the diffusion consistency index. Simultaneously, the difference in the local principal direction angle between the boundary start point and the end point is directly used as the head-tail propagation direction difference to characterize whether the boundary information has good overall transmission coherence. All data acquisition is obtained by calculating the pixel distribution of the real boundary in the image and the grayscale or feature response of its neighborhood window. This data can be directly calculated based on the image itself and does not depend on external sensors or training data.

[0110] It should be noted that the information propagation consistency value is an evaluation index used to measure whether the information direction, structural diffusion characteristics, and beginning-end coherence of the target's true boundary curve remain consistent within a local area of ​​an image. Its core function is to reflect whether the information transmission between points on the boundary is continuous, stable, and coordinated. Its purpose is to determine whether the boundary, during the image boundary fitting process, maintains the continuous flow and consistency of the diffusion structure of image information such as grayscale or texture. A larger value indicates smaller changes in the main direction between adjacent points on the boundary, meaning the information flow is stable; simultaneously, the information diffusion structure in the local area of ​​the boundary point also changes less between two points, indicating strong structural continuity of the image content; furthermore, the directional difference between the starting and ending points of the boundary is also small, indicating that the overall information propagation path is a closed loop and continuous, without abrupt jumps or structural breaks. In a typical example, if the target boundary is a regular circle, its boundary direction is continuous, the local structure is stable, and the start and end points are connected. In this case, the value will be close to 1, indicating a very natural and realistic fit. Conversely, if the boundary is jagged, has frequent local jumps, or the start and end directions of the boundary are significantly inconsistent, the value will be close to 0, indicating serious inconsistencies and distortions in the fit. Therefore, this value can serve as an important standard for evaluating the quality of the fit. The larger the value, the more consistent the fitted curve is with the true content of the image in terms of information propagation structure, and thus the more reliable the fitting result.

[0111] It should be noted that the greatest advantage of calculating the information propagation consistency value using the above method lies in its simultaneous integration of three highly complementary and structurally diverse feature factors: local directional changes at image boundary points, the stability of the information diffusion structure, and the overall coherence of the boundary. This allows for a comprehensive reflection of the true information transmission during the boundary fitting process from multiple non-redundant perspectives, thus avoiding the one-sided misjudgments that may arise from single-dimensional indicators. Compared to conventional methods that rely solely on gradient direction or edge intensity to determine the similarity of adjacent points, the above method, through the synergistic evaluation of three factors—directional jump, diffusion tensor perturbation, and the difference between the beginning and end directions—enables the algorithm to accurately perceive the local continuity of boundary lines, capture structural diffusion differences caused by texture changes, and determine whether there are unreasonable structural breaks or missing loops in the overall boundary. This makes it particularly suitable for boundary fitting evaluation under complex targets, multi-scale textures, or weak boundary conditions. Furthermore, the above method constructs multiple dimensionless ratio factors and aggregates them unweightedly through multiplication, which can effectively suppress the amplification interference of a certain local anomaly on the overall result, improve the robustness and physical interpretability of the evaluation, and the final value is not only numerically stable and easy to understand, but also has strong engineering applicability and cross-scenario adaptability.

[0112] In one embodiment, S4: The step of determining the fitting result state of the target's true boundary curve based on the qualified value extracted from the boundary, and outputting the target's true boundary, includes:

[0113] The boundary extraction pass value corresponding to the true boundary curve of each target is compared with the preset threshold. If the boundary extraction pass value is not less than the preset threshold, it means that the true boundary curve of the target is qualified, and the true boundary of the corresponding target is directly output.

[0114] If the qualified value of the boundary extraction is less than the preset threshold, it means that the true boundary curve of the target is unqualified. Then, repeat the curve fitting and extraction of step S2 for the boundary curve of the corresponding target until the true boundary curve of the target is qualified, and output the true boundary of the corresponding target.

[0115] It should be noted that by calculating the boundary extraction qualification value corresponding to the target boundary and comparing this value with a pre-set threshold, if the boundary extraction qualification value is not less than the threshold, it means that the boundary has met the requirements in multiple dimensions such as topological structure, local orientation consistency, and information propagation stability. At this time, the boundary can be directly used as the final true boundary output of the target for subsequent analysis, identification, or control operations. However, if the boundary extraction qualification value is lower than the preset threshold, it means that the current fitted boundary has problems such as large local jumps, abnormal structural diffusion, or overall loop breakage, which will interfere with the final identification or segmentation results. Therefore, the system will automatically re-trigger the boundary extraction module and reconstruct the boundary of the target based on updated parameters or strategies until the re-extracted boundary meets the evaluation requirements, thereby avoiding misjudgment of the boundary results due to a one-time error or local anomaly. For example, in an industrial inspection task, if a part has slight reflection or stains on its edge, the initial fitting may result in a low pass value due to edge jumps. In this case, the system will not directly output the boundary, but will extract it again. It may optimize the boundary curve by enhancing local texture, adjusting the extraction step size, or ignoring low-confidence segments until the boundary extraction pass value exceeds the set threshold before outputting it. This ensures that the final boundary data has high reliability and engineering application value in terms of geometric structure and information continuity.

[0116] The foregoing has described one embodiment of the present invention in detail, but this content is merely a preferred embodiment and should not be considered as limiting the scope of the present invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the scope of the claims of this invention.

Claims

1. A method for separating multiple target scene instances based on visual recognition, characterized in that, Includes the following steps: Feature extraction is performed on the input image, and multiple rectangular candidate bounding boxes are generated using a target detection algorithm based on a deep neural network. The rectangular candidate bounding boxes are used to represent the location of the target in the image, and multiple sets of rectangular bounding boxes are generated. Extract local region images from each rectangular bounding box, and fit the boundaries of the local region images to obtain the true boundary curve of the corresponding target; The fitting results of the target's true boundary curve are analyzed, and the boundary extraction qualification value is calculated. Based on the qualified value extracted from the boundary, the fitting result of the true boundary curve of the target is determined, and the true boundary of the target is output.

2. The multi-target scene instance separation processing method based on visual recognition according to claim 1, characterized in that, The steps for generating multiple rectangular candidate bounding boxes using a deep neural network-based object detection algorithm include: The input image is processed by a convolutional neural network to extract features and generate multi-scale feature maps. Anchor points are generated by sliding a window on the feature map. The probability of the target presence at each anchor point and the boundary regression offset relative to the anchor point are predicted by target classification and boundary regression, respectively. Using the target's existing classification probability and boundary regression offset, calculate the rectangular bounding box corresponding to each anchor point, perform non-maximum suppression on the rectangular bounding boxes to remove duplicate boxes and retain boxes with high confidence. Output the final set of candidate rectangular bounding boxes.

3. The multi-target scene instance separation processing method based on visual recognition according to claim 1, characterized in that, The steps of extracting local region images from each rectangular bounding box and fitting the true boundary curve of the potential target based on the boundaries of the local region images include: Local region images are extracted from each rectangular bounding box, and features are extracted from the local region images using a convolutional neural network to generate local feature maps; On the extracted feature map, pixel aggregation and separation operations with instance-aware constraints are performed. Instance cohesion constraints ensure that the pixel features of the same target instance are clustered together in the feature space, making the pixel features within the same target instance consistent. Instance separation constraints ensure that the pixel features of different target instances are separated from each other, avoiding feature mixing or boundary overlap between different targets. After constraint optimization, a mask for the target instance is generated. Based on the generated instance mask, the boundary points in the mask are extracted. The extracted boundary point set is fitted by a curve fitting method to generate the true boundary curve of the target. The fitted boundary curves are smoothed to obtain the true boundary curves; and the true boundary curves corresponding to each rectangular bounding box are output.

4. The multi-target scene instance separation processing method based on visual recognition according to claim 1, characterized in that, The steps for analyzing the fitting results of the target's true boundary curve and calculating the qualified boundary value are as follows: The fitting result features include boundary topological stability value and information propagation consistency value. The boundary topological stability value and information propagation consistency value are added together to obtain the boundary extraction qualification value.

5. The multi-target scene instance separation processing method based on visual recognition according to claim 4, characterized in that, The steps for calculating the boundary topological stability value are as follows: The true boundary curve of the target is divided into multiple boundary points, and the angle of each boundary point in the boundary point set is calculated. Calculate the absolute difference in angles between adjacent boundary points, and use this as the angle difference between two adjacent boundary points; Calculate the Euclidean distance between two adjacent boundary points; divide the angular difference between two adjacent boundary points by the Euclidean distance, and use the result of the division as the first ratio value. The mean of all first ratio values ​​is used as the boundary adaptive rate of change. Calculate the curvature at each boundary point, calculate the mean of all curvatures, calculate the absolute difference between each curvature and the mean curvature, and record the mean of all absolute differences as the boundary morphology distortion. The boundary topological stability value is calculated based on the boundary adaptive change rate and the boundary morphological distortion.

6. The multi-target scene instance separation processing method based on visual recognition according to claim 5, characterized in that, The steps for calculating the boundary topological stability value based on the boundary adaptive rate of change and the boundary morphological distortion are as follows: Calculate the sum of the boundary adaptive change rate, the boundary morphological distortion, and the value of 1, and use the reciprocal of the sum as the boundary topological stability value.

7. The multi-target scene instance separation processing method based on visual recognition according to claim 4, characterized in that, The steps for calculating the consistency value of information propagation are as follows: The true boundary curve of the target is divided into multiple boundary points, forming a boundary point set. ;in Boundary point index, boundary point Represents the coordinates of each pixel on the target boundary; For each boundary point Calculate the local orientation angle of the corresponding image information. , In the formula, Indicates the first A boundary point, This represents the grayscale value of the image at that location; , These represent the horizontal orientation of the image. and longitudinal Partial derivative of direction; Point The local orientation angle corresponding to the image information.

8. The multi-target scene instance separation processing method based on visual recognition according to claim 7, characterized in that, The steps for calculating the information propagation consistency value also include: For each pair of adjacent boundary points , Calculate its local directional jump value , And calculate the jump intensity ratio based on the jump values ​​in all directions. The information direction consistency factor is calculated by adding the reciprocal of the value of 1 to the jump intensity ratio; the formula is as follows: In the formula, Indicates the first Local directional jump value Indicates the jump intensity ratio; For each point Constructing the local image information diffusion tensor And calculate the Frobenius norm of the diffusion tensor between adjacent points as the perturbation amplitude. ; Based on all disturbance amplitudes Calculate the diffusion tensor perturbation ratio The diffusion uniformity factor is calculated by adding the perturbation ratio of the diffusion tensor to the reciprocal of the value of 1; the formula is as follows: , Indicates the first The amplitude of the disturbance; Calculate the local directional jump value between the starting point and the ending point of the boundary as the first and last information directional difference, and add the reciprocal of the value of 1 to the first and last information directional difference as the return consistency factor. Information direction consistency factor, diffusion consistency factor, and backhaul consistency factor Multiply them to obtain the consistency value of information dissemination.

9. The multi-target scene instance separation processing method based on visual recognition according to claim 1, characterized in that, The steps for determining the fitting result of the target's true boundary curve based on the qualified value extracted from the boundary, and outputting the target's true boundary, include: The qualified value of the boundary extracted from the true boundary curve of each target is compared with the preset threshold. If the qualified value of the boundary extracted is not less than the preset threshold, it means that the true boundary curve of the target is qualified, and the true boundary of the corresponding target is directly output.

10. The multi-target scene instance separation processing method based on visual recognition according to claim 1, characterized in that, The steps of determining the fitting result of the true boundary curve of the target based on the qualified value extracted from the boundary, and outputting the true boundary of the target, also include: If the qualified value of the boundary extraction is less than the preset threshold, it means that the true boundary curve of the target is unqualified. Then, the boundary curve of the corresponding target is extracted again until the true boundary curve of the target is qualified, and the true boundary of the corresponding target is output.

Images