A sub-pixel feature extraction method based on semantic-geometric dual perception

Abstract

The application relates to the technical field of industrial robot vision feature extraction, and provides a subpixel feature extraction method based on semantic-geometric dual perception. The subpixel feature extraction method based on semantic-geometric dual perception comprises the following steps: light-weight semantic target detection, dynamic ROI extraction and target locking, subpixel edge refinement and geometric solution, time domain feedback enhancement and time delay compensation. The subpixel feature extraction method based on semantic-geometric dual perception has the advantages of high positioning accuracy, strong environmental adaptability and low running time delay, can realize stable extraction of target features in an unstructured industrial scene, can inhibit environmental interference and signal jitter, can adapt to the deployment requirements of industrial edge computing devices, and can provide visual perception support for precise assembly work of an industrial robot.

Description

Technical Field

[0001] This application relates to the technical field of visual feature extraction for industrial robots, and in particular to a sub-pixel feature extraction method based on semantic-geometric dual perception. Background Technology

[0002] With the continuous development of industrial automation technology, the performance requirements of visual perception systems for industrial robots in precision assembly and other operational scenarios are constantly increasing. Visual servoing technology, as a key support for enabling adaptive operations of industrial robots, generates feedback signals for robot control by acquiring visual information from the work environment. It is a crucial link connecting the physical workspace and the motion control system in an industrial robot system. Currently, visual perception technologies used for target extraction and localization in industrial scenarios are mainly divided into two categories: geometric feature detection technology based on digital image processing, and semantic feature detection technology based on deep convolutional neural networks.

[0003] Geometric feature detection technology based on digital image processing relies on the rigid geometry of the target to first locate the target in the image, and then extract and locate the target features through edge detection and geometric parameter calculation. Semantic feature detection technology based on deep convolutional neural networks builds a model through training on a large-scale dataset, extracts high-dimensional semantic features of the target, and achieves target capture and position recognition in complex and ever-changing industrial operation scenarios, providing corresponding position references for the motion control of industrial robots.

[0004] In existing technologies, geometric feature detection techniques based on digital image processing are prone to false detections or feature extraction failures in unstructured industrial scenarios with varying lighting, surface reflections, partial occlusion, or background clutter, making it impossible to maintain stable and continuous feature output. Semantic feature detection techniques based on deep convolutional neural networks typically only achieve pixel-level resolution in their target localization results, failing to meet the positioning accuracy requirements of precision industrial assembly operations. Furthermore, the complex network structure makes real-time operation difficult on industrial edge devices with limited computing power. Existing technologies cannot simultaneously achieve stable capture and high-precision localization of target features in unstructured industrial operation scenarios, making it difficult to adapt to the comprehensive performance requirements of visual perception systems in precision assembly operations of industrial robots. Summary of the Invention

[0005] In view of this, this application aims to propose a sub-pixel feature extraction method based on semantic-geometric dual perception to solve the above-mentioned technical problems.

[0006] To achieve the above objectives, the technical solution of this application is implemented as follows: A sub-pixel feature extraction method based on semantic-geometric dual perception includes: S1, lightweight semantic object detection, performs global detection on the input industrial scene image, outputs pixel-level prediction boxes of the target, and establishes global semantic constraints; S2, Dynamic ROI extraction and target locking: Based on the pixel-level prediction box, a dynamic region of interest (ROI) is constructed. Through the multi-dimensional scoring mechanism of task topology constraints and temporal correlation logic, the current task target is uniquely locked. S3, Subpixel edge refinement and geometric solution: Reconstruct the target edge within the locked ROI, output a subpixel level edge point set, fit the subpixel level edge point set using the second-order robust least squares method, and output the target's subpixel level geometric feature vector. S4, Temporal feedback enhancement and time delay compensation: The sub-pixel level geometric feature vector is smoothed in the temporal domain and its motion state is estimated by a Kalman state observer to construct a perception-prediction adaptive closed loop. At the same time, the target feature state is fed forward to predict and output a synchronous alignment feature signal for visual servoing.

[0007] Furthermore, S1 includes: S11, Construct an improved YOLOv8s lightweight detection model as the perception front end; S12 introduces the GhostNet design concept into the model backbone network, generating phantom feature maps through GhostConv and C3Ghost modules to compress the model's computational load; S13 embeds the CBAM attention mechanism at the end of feature extraction, which enhances the perception of the target's physical structure through channel and spatial dimension recalibration. S14, the WIoUv3 loss function with dynamic non-monotonic focusing mechanism is used to optimize the bounding box regression; S15 uses a trained lightweight detection model to perform global detection on the input industrial scene image, outputs the pixel-level predicted bounding box coordinates of the target, and establishes global semantic constraints.

[0008] Furthermore, S2 includes: S21. Based on the pixel-level prediction box output in step S1, an expansion coefficient is introduced to construct a dynamic region of interest (ROI) to ensure complete coverage of the target edge information. S22. For array-type homogeneous targets, a task topology scoring function is constructed, which comprehensively considers the polar angle deviation, roundness index and confidence of candidate targets, and selects the optimal target for the current process by matching it with the preset task template sequence. S23, based on the principle of minimum spatial offset, calculates the Euclidean displacement deviation of targets in adjacent frames to establish a temporal correlation mechanism, and uniquely locks the current operation target.

[0009] Furthermore, S2 also includes: S24, Dynamic tracking window construction: A local tracking window is generated based on the locked target center, and image information outside the window is forcibly suppressed, retaining only the target area within the window for subsequent processing.

[0010] Furthermore, the task topology scoring function in S22 comprehensively considers three parameters of the candidate target: polar angle deviation, roundness index, and confidence level, and assigns corresponding weight coefficients to each of the three parameters. The optimal target for the current process is determined by minimizing the value of the task topology scoring function.

[0011] Furthermore, in step S3, before reconstructing the target edge, preprocessing of the image within the locked ROI is also included. The preprocessing steps at least include: Local image enhancement involves non-linear brightness stretching of the ROI image through Gamma correction to suppress high-gloss reflections from metal surfaces. Pixel-level coarse edge extraction utilizes the Canny operator to calculate the image gradient magnitude and gradient direction, and extracts the coarse edge point set of the target through non-maximum suppression and double thresholding algorithms; Contour selection involves filling edge breaks through morphological closing operations and filtering candidate contours based on target geometric priors to obtain the baseline edge for sub-pixel reconstruction.

[0012] Furthermore, the sub-pixel edge reconstruction and geometric solution method in S3 is as follows: The Zernike orthogonal moment operator is used to perform subpixel relocalization on the coarse edge point set, correcting the integer pixel coordinates to subpixel coordinates and constructing a subpixel level edge point set. The subpixel level edge point set is initially fitted and outlier is removed using the second-order robust least squares method. The filtered clean edge point set is then used for secondary fitting to output the target's subpixel level geometric feature vector.

[0013] Furthermore, S4 includes: S41, based on the pinhole camera model, the sub-pixel level geometric feature vector is normalized, the system's state vector and linear state transition equation are constructed, and a Kalman filter observation model is established; S42 uses a Kalman state observer to perform temporal smoothing and motion state estimation on target features, and outputs the feature evolution rate estimate and intermediate filtering parameters. S43 generates feedback adjustment instructions based on intermediate filtering parameters, and reversely adjusts the algorithm parameters of the front-end sub-pixel edge refinement and geometric solution steps to construct a perception-prediction adaptive closed loop. S44 combines the feature evolution rate estimate with the cumulative delay of the sensing link to perform kinematic feedforward prediction of the target feature state and outputs a synchronous alignment feature signal for visual servoing.

[0014] Furthermore, the feedback adjustment command in S43 includes at least: A window adjustment instruction based on the trace generation of the predicted covariance matrix is ​​used to dynamically intervene in the neighborhood window size of the Zernike orthogonal moment calculation; The threshold adjustment command based on the normalized observation residual is used to dynamically adjust the robust threshold of the second-order robust least squares method.

[0015] Furthermore, the calculation method for the feedforward prediction in S44 is as follows: By combining the estimated feature evolution rate with the cumulative delay τ of the sensing link, the predicted feature state of the target at the control execution time is extrapolated, thus offsetting the phase lag caused by image processing.

[0016] Compared with existing technologies, the sub-pixel feature extraction method based on semantic-geometric dual perception proposed in this application has the following advantages: (1) This application achieves stable target capture in complex unstructured industrial scenes by performing global lightweight semantic target detection on the input industrial scene image, outputting the pixel-level prediction box of the target and establishing global semantic constraints. At the same time, it reduces the computational overhead of the semantic detection model, enabling it to adapt to the operating requirements of industrial edge computing equipment with limited computing power.

[0017] (2) This application constructs a dynamic region of interest based on pixel-level prediction boxes, combines a multi-dimensional scoring mechanism of task topology constraints with temporal correlation logic to uniquely lock the current task target, completes image preprocessing, sub-pixel edge refinement and geometric calculation within the locked region, and outputs sub-pixel-level geometric feature vectors. This achieves high-precision feature localization that breaks through the pixel-level resolution limitation, while effectively avoiding the feature mismatch problem caused by homogeneous targets, and improving the stability and continuity of the feature extraction process under unstructured interference environment.

[0018] (3) This application uses a Kalman state observer to perform temporal smoothing and motion state estimation on sub-pixel level geometric feature vectors, constructs an adaptive closed loop for perception and prediction, and simultaneously performs feedforward prediction on target feature state, outputting a synchronous alignment feature signal for visual servoing. This achieves effective compensation for the processing delay of the visual perception link, suppresses temporal random jitter of feature signals, provides a stable and continuous high-precision feedback signal for visual servo control of industrial robots, and meets the comprehensive performance requirements of industrial precision assembly operations. Attached Figure Description

[0019] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1This is a flowchart illustrating the sub-pixel feature extraction method based on semantic-geometric dual perception as described in the embodiments of this application; Figure 2 This is a schematic diagram showing the arrangement of holes in different workpieces in the embodiments of this application; Figure 3 This diagram illustrates the image processing and filtering process of the sub-pixel feature extraction method based on semantic-geometric dual perception as described in the embodiments of this application. Detailed Implementation

[0020] To make the technical solution and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0022] Furthermore, it should be noted that in the description of this application, if terms such as "upper," "lower," "inner," or "outer" appear, indicating orientation or positional relationship, these are based on the orientation or positional relationship shown in the accompanying drawings and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. In addition, if terms such as "first" or "second" appear, they are also used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0023] Furthermore, in the description of this application, unless otherwise expressly defined, the terms "installation," "connection," "joining," and "connector" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application in light of the specific circumstances.

[0024] In this application, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0025] The present application will now be described in detail through exemplary embodiments. However, it should be understood that, without further description, elements, structures, and features in one embodiment may be advantageously incorporated into other embodiments.

[0026] This application provides a sub-pixel feature extraction method based on semantic-geometric dual perception, which is applied to the technical field of visual feature extraction for industrial robots.

[0027] To achieve stable extraction and high-precision localization of target features in unstructured industrial scenarios, the sub-pixel feature extraction method based on semantic-geometric dual perception in this embodiment is configured to include four sequentially executed steps, as described above. Figure 1 The overall process of the method is as follows: S1 Lightweight semantic target detection, S2 Dynamic ROI extraction and target locking, S3 Subpixel edge refinement and geometric calculation, S4 Temporal feedback enhancement and latency compensation.

[0028] S1 Lightweight semantic object detection performs global detection on the input industrial scene image, outputting pixel-level predicted bounding boxes for the target and establishing global semantic constraints. S2 Dynamic ROI extraction and target locking construct a dynamic Region of Interest (ROI) based on the pixel-level predicted bounding boxes. Through a multi-dimensional scoring mechanism and temporal correlation logic based on task topology constraints, the current task target is uniquely locked. S3 Subpixel edge refinement and geometric calculation reconstructs the target edges within the locked ROI, outputting a subpixel-level edge point set. This set is then fitted using a second-order robust least squares method to output the target's subpixel-level geometric feature vector. S4 Temporal feedback enhancement and latency compensation uses a Kalman state observer to perform temporal smoothing and motion state estimation on the subpixel-level geometric feature vector, constructing a perception-prediction adaptive closed loop. Simultaneously, feedforward prediction of the target feature state is performed, outputting a synchronous alignment feature signal for visual servoing.

[0029] Based on the invention's objective of improving the adaptability of semantic detection models for deployment on industrial edge devices, in this embodiment, the specific implementation steps of lightweight semantic target detection in step S1 include: S11 constructing an improved YOLOv8s lightweight detection model as the perception front end; S12 introducing the GhostNet design concept into the model backbone network, generating phantom feature maps through GhostConv and C3Ghost modules to compress the model's computational load; S13 embedding the CBAM attention mechanism at the end of feature extraction, enhancing the perception of the target's physical structure through channel and spatial dimension recalibration; S14 using the WIoUv3 loss function with a dynamic non-monotonic focusing mechanism to optimize bounding box regression; and S15 performing global detection on the input industrial scene image through the trained lightweight detection model, outputting the pixel-level predicted bounding box coordinates of the target, and establishing global semantic constraints.

[0030] To address the physical characteristics of metal hole edges in industrial environments, where gradients are extremely strong but easily diffused due to reflections, GhostConv and C3Ghost modules utilize their inherent feature redundancy extraction capabilities to construct a detail enhancement perception mechanism for metal hole edge details. To mitigate the interference of metal surface processing textures on feature extraction, this mechanism embeds spatial consistency constraints in a lightweight path, significantly reducing the model's floating-point computation while ensuring accurate capture of high-frequency edge features of the holes, effectively alleviating the problem of local gradient loss caused by uneven illumination.

[0031] The CBAM attention mechanism enhances the perception of the essential physical structure of metal apertures through adaptive recalibration of the channel and spatial dimensions, suppresses background clutter, automatically identifies and suppresses high-light noise channels caused by metal mirror reflection through the channel attention mechanism, and recalibrates the feature map weights through the spatial attention mechanism, so that the network focus is strongly locked on the circumferential physical configuration of the aperture.

[0032] The WIoUv3 loss function addresses outlier interference caused by tool movement or oil smudges in production environments. It establishes an outlier suppression mechanism by adjusting the non-monotonic focusing coefficient, dynamically reducing the gradient gain of high-quality samples and mitigating the negative impact of extreme noise samples, thus improving the model's regression robustness under blurred edge conditions. In multi-target interference scenarios, step S15 prioritizes recording the bounding box with the highest confidence and its center coordinates as a benchmark for subsequent logical judgments. The target bounding box is composed of its center coordinates, width, height, and class confidence. The target bounding box is determined by taking the maximum confidence value, thereby establishing global semantic constraints and the spatial search domain. Through these steps, the model achieves rapid target retrieval in complex full-frame images while significantly reducing the computational load and parameter count, enabling smooth deployment in industrial embedded devices with limited computing power.

[0033] To quantitatively determine the target bounding box, this embodiment uses the following formula to perform target filtering:

[0034] In the formula, The pixel coordinates of the target's geometric center. These are the width and height of the prediction box, respectively. `B_target` is the final target bounding box. `x_c` is the x-coordinate of the target bounding box center in pixels. `y_c` is the y-coordinate of the target bounding box center in pixels. `P'` is the set of all candidate bounding boxes output by the lightweight detection model. `arg max` indicates that the candidate bounding box that maximizes the confidence score is taken as the output.

[0035] In multi-target interference scenarios, step S15 prioritizes recording the bounding box with the highest confidence and its center coordinates as a benchmark reference for subsequent logical judgments, thereby establishing global semantic constraints and spatial search domain. Through the above steps, the effect of quickly retrieving targets in complex full-frame images is achieved, while significantly reducing the computational load and parameter count of the model, enabling it to be smoothly deployed in industrial embedded devices with limited computing power.

[0036] To achieve unique target locking based on semantic constraints and avoid mismatch problems caused by homogeneous targets, the specific implementation steps of dynamic ROI extraction and target locking in this embodiment include: S21 Based on the pixel-level prediction box output in step S1, an expansion coefficient is introduced to construct a dynamic region of interest (ROI) to ensure complete coverage of target edge information; S22 For array-type homogeneous targets, a task topology scoring function is constructed, which comprehensively considers the polar angle deviation, roundness index, and confidence of candidate targets, and selects the optimal target for the current process by matching with the preset task template sequence; S23 Based on the principle of minimum spatial offset, the Euclidean displacement deviation of targets in adjacent frames is calculated to establish a temporal correlation mechanism to uniquely lock the current task target. In step S21, the expansion coefficient is set to 0.2. The value of the expansion coefficient needs to balance the redundancy of the search range and the shielding effect of background interference. If the expansion coefficient is too small, the edge of the target may exceed the prediction box when the robot arm moves and shakes, resulting in the loss of edge information. If the expansion coefficient is too large, too much background clutter will be introduced, increasing the computational cost of subsequent processing. Therefore, the expansion coefficient is set to 0.2, which can ensure that the edge information of the local target is not lost due to the drift of the prediction box when the robot moves and shakes through spatial redundancy design, and can also effectively control the range of the search area and reduce background interference.

[0037] To quantify the coordinates and dimensions of the dynamic region of interest (ROI), this embodiment constructs the ROI using the following formula: the x-coordinate of the top-left corner of the ROI is the maximum value between 0 and the x-coordinate of the predicted box center minus half the predicted box width multiplied by an expansion factor; the y-coordinate of the top-left corner of the ROI is the maximum value between 0 and the y-coordinate of the predicted box center minus half the predicted box height multiplied by an expansion factor; the width of the ROI is the minimum value between the original image width and the predicted box width multiplied by an expansion factor; and the height of the ROI is the minimum value between the original image height and the predicted box height multiplied by an expansion factor. Through this step, the local regions containing all potential targets can be separated from the complex overall background, forming a sequence of candidate ROI images.

[0038]

[0039] The above formula is used to calculate the top-left corner coordinates and width and height parameters of the dynamic region of interest (ROI) based on the parameters of the target prediction bounding box, ensuring that the target edge information is completely covered within the ROI. In the formula, x_roi is the x-coordinate of the top-left corner of the dynamic ROI, in pixels. y_roi is the y-coordinate of the top-left corner of the dynamic ROI, in pixels. w_roi is the width of the dynamic ROI, in pixels. h_roi is the height of the dynamic ROI, in pixels. x_c is the x-coordinate of the center of the target prediction bounding box, in pixels. y_c is the y-coordinate of the center of the target prediction bounding box, in pixels. w is the width of the target prediction bounding box, in pixels. h is the height of the target prediction bounding box, in pixels. λ is the expansion coefficient. W_img is the width of the original input image, in pixels. H_img is the height of the original input image, in pixels. max indicates taking the larger of the two values ​​within parentheses. min indicates taking the smaller of the two values ​​within parentheses.

[0040] Reference Figure 2 The diagram below shows the arrangement of holes in different workpieces in this application embodiment. Step S22 performs multi-dimensional scoring and screening based on task topology guidance for the array arrangement characteristics of clutch holes. First, the bounding box of the clutch workpiece body is obtained and its center is used as the reference origin to establish a global polar coordinate system. The polar angles of all candidate targets in the field of view are calculated. For the hole distribution characteristics of workpieces of different specifications, the system uses the cross-tightening criterion in industrial assembly to preset the angle sequence of the task template. By minimizing the value of the task topology scoring function, the optimal target of the current process is determined.

[0041] To quantify and select the optimal objective for the current process, this embodiment constructs the following task topology scoring function:

[0042] This formula is used to comprehensively score each candidate target. By minimizing the value of the scoring function, the task target with the highest matching degree with the current process is selected, avoiding mismatch problems caused by homogeneous targets.

[0043] In the formula, S(C_i) is the comprehensive score of the i-th candidate target. C_i is the i-th candidate target. α is the weighting coefficient of the polar angle deviation term. φ_i is the polar angle of the i-th candidate target. θ_ref is the preset reference angle of the current process. β is the weighting coefficient of the roundness index term. C_i is the roundness index of the i-th candidate target. γ is the weighting coefficient of the confidence level term. P_i is the category confidence level of the i-th candidate target.

[0044] Based on the invention objective of improving the accuracy of homogeneous target screening, in this embodiment, the task topology scoring function in S22 comprehensively considers three parameters of candidate targets: polar angle deviation, roundness index, and confidence level, and configures corresponding weight coefficients for each of the three parameters. The optimal target for the current process is determined by minimizing the value of the task topology scoring function. The task topology scoring function consists of a weighted sum of three terms. The first term is the polar angle deviation term, multiplied by the corresponding weight coefficient. The polar angle deviation term is the absolute value of the difference between the polar angle of the candidate target and the reference angle of the current process, used to verify whether the target is within the preset angle sector of the current task. The second term is the roundness compensation term, multiplied by the corresponding weight coefficient. The roundness compensation term is the result of subtracting the roundness index of the candidate target, used to evaluate the edge quality of the target and eliminate irregular interference edges caused by reflection and oil stains. The third term is the confidence compensation term, multiplied by the corresponding weight coefficient. The confidence compensation term is the result of subtracting the confidence of the candidate target category, used to reduce the competitive priority of targets with low confidence. This screening principle tightly couples the visual inspection results with the assembly process logic, shielding the interference of background noise and false targets from the source.

[0045] Step S23 calculates the Euclidean displacement deviation of targets in adjacent frames based on the principle of minimum spatial offset, establishes a temporal locking mechanism, and ensures that visual servo commands always act on the same physical object. The system maintains the motion state of the current target in real time and automatically calculates the spatial correlation strength between the candidate set of subsequent frames and the motion state of the locked target in the previous frame. It prioritizes locking the closest instance. The Euclidean displacement deviation is the square of the difference between the horizontal coordinate of the candidate target and the horizontal coordinate of the locked target in the previous frame, plus the square of the difference between the vertical coordinate of the candidate target and the vertical coordinate of the locked target in the previous frame, and then takes the minimum square root to measure the spatial correlation strength between targets.

[0046] To quantify the spatial correlation strength between targets in adjacent frames, this embodiment calculates the Euclidean displacement deviation using the following formula:

[0047] This formula is used to calculate the minimum Euclidean displacement deviation between the candidate target in the current frame and the locked target in the previous frame, thereby measuring the spatial correlation strength between targets and ensuring the uniqueness of temporal locking.

[0048] In the formula, D_k is the minimum Euclidean displacement deviation in the k-th frame. x_i is the x-coordinate of the i-th candidate target in the current frame, in pixels. y_i is the y-coordinate of the i-th candidate target in the current frame, in pixels. x_{k-1} is the x-coordinate of the locked target in the (k-1)-th frame, in pixels. y_{k-1} is the y-coordinate of the locked target in the (k-1)-th frame, in pixels. min indicates taking the minimum displacement deviation among all candidate targets. sqrt indicates taking the square root of the value within the parentheses. This formula uniquely identifies the instance with the highest spatial correlation to the target in the previous frame, ensuring that servo commands always apply to the same physical object.

[0049] To further improve the computational efficiency of the algorithm and shield against background clutter interference at the source, the S2 dynamic ROI extraction and target locking in this embodiment also includes S24 dynamic tracking window construction. The dynamic tracking window construction generates a local tracking window based on the locked target center, forcibly suppressing image information outside the window and retaining only the target area within the window for subsequent processing. Once the target is accurately locked, the system immediately switches from global search to local tracking. At this point, instead of searching the entire image, a dynamic tracking window is constructed, generating a local window of the corresponding size based on the target center, and forcibly suppressing image information outside the window. The image within the window retains the grayscale values ​​of the original input image, while the grayscale values ​​of the image outside the window are set to 0. This step not only improves the computational efficiency of the algorithm but also shields against background clutter interference at the source.

[0050] To quantitatively suppress image information outside the window, this embodiment uses the following formula to perform region filtering of the image:

[0051] This formula is used to filter regions of the original input image, retaining only the image information within the tracking window and forcing the grayscale value of the image outside the window to be set to 0, thus blocking the interference of background clutter from the source.

[0052] In the formula, I_process represents the processed image. u is the x-coordinate of the image pixel, in pixels. v is the y-coordinate of the image pixel, in pixels. I_raw is the original input image. u_t is the x-coordinate of the target center, in pixels. v_t is the y-coordinate of the target center, in pixels. W is the width of the tracking window, in pixels. H is the height of the tracking window, in pixels. `if` indicates that the corresponding value is taken when the corresponding condition is met. `otherwise` indicates that the corresponding value is taken when the condition is not met. This step not only improves the computational efficiency of the algorithm but also filters out background clutter interference at the source.

[0053] To improve the signal-to-noise ratio of target edge features and establish a smooth brightness baseline for subsequent sub-pixel extraction, the S3 sub-pixel edge refinement and geometric calculation in this embodiment includes preprocessing the image within the locked ROI before reconstructing the target edge. The preprocessing steps include at least local image enhancement, pixel-level coarse edge extraction, and contour filtering. Local image enhancement uses Gamma correction to perform non-linear brightness stretching on the ROI image, suppressing high-gloss reflections from the metal surface.

[0054] The calculation method for gamma correction is to take the gray value of the smoothed image by a power of one of the gamma coefficients, and then perform normalization to obtain the enhanced image. The gamma coefficient is set to 2.0. The value of the gamma coefficient needs to be adapted to the image characteristics of the industrial assembly site. The overall gray value of the workpiece surface in the assembly site is low, while the gray value of the highlight area is high. A gamma coefficient of 2.0 can perform non-linear brightness stretching of the image, effectively suppressing the influence of uneven illumination and significantly improving the signal-to-noise ratio of the target edge. If the gamma coefficient is too small, it cannot effectively enhance the details in the dark area. If the gamma coefficient is too large, it will amplify the image noise and affect the accuracy of subsequent edge extraction.

[0055] To quantify and achieve non-linear brightness stretching of the image, this embodiment uses the following Gamma correction formula to perform image enhancement: Ienhance(u,v)=Normalize(Ismooth(u,v))1 / γ This formula is used to perform non-linear brightness stretching on the smoothed image, enhance details in dark areas, suppress specular reflections on metal surfaces, and improve the signal-to-noise ratio at target edges.

[0056] In the formula, I_enhance represents the enhanced image. u is the x-coordinate of the image pixels, in pixels. v is the y-coordinate of the image pixels, in pixels. Normalize indicates that the values ​​within the parentheses are normalized, mapping them to a grayscale range of 0 to 255. I_smooth represents the smoothed input image. γ is the gamma correction coefficient.

[0057] Pixel-level coarse edge extraction utilizes the Canny operator to calculate the gradient magnitude and direction of the image. Non-maximum suppression and a double thresholding algorithm are then used to extract the set of coarse edge points of the target. First, the Sobel operator is used to calculate the gradient components of the image in the horizontal and vertical directions. Based on these components, the gradient magnitude and direction are calculated. The gradient magnitude is the square root of the sum of the squares of the horizontal and vertical gradient components. The gradient direction is the arctangent of the vertical gradient component divided by the horizontal gradient component. Then, non-maximum suppression is used to refine the edges, and a double thresholding algorithm is combined to extract strong edge points. Weak edge points adjacent to these points and exceeding the lower threshold are recursively connected, resulting in a binarized edge image.

[0058] Contour selection fills edge breaks through morphological closing operations and, combined with prior geometric knowledge of the target, filters candidate contours to obtain baseline edges for sub-pixel reconstruction. First, morphological closing operations are performed on the binary edge map extracted by Canny to fill minor edge breaks and smooth the contours. Then, all independent closed contours are extracted. Finally, based on prior knowledge that the hole target within the ROI region is centered and its geometric shape is close to a circle, each candidate contour is comprehensively evaluated. The scoring function considers parameters such as the Euclidean distance between the contour centroid and the ROI center, shape roundness, and aspect ratio. The contour with the lowest score is automatically selected as the baseline for subsequent sub-pixel extraction. Figure 3 , Figure 3 This diagram illustrates the image processing and filtering process of the sub-pixel feature extraction method based on semantic-geometric dual perception described in this application embodiment. This step effectively eliminates complex background clutter while ensuring real-time computation, significantly improving the robustness of the system.

[0059] To quantify the gradient information of an image, this embodiment calculates the gradient magnitude and gradient direction using the following formula:

[0060] The above formula is used to calculate the gradient magnitude and gradient vector direction of each pixel in the image, providing basic data for subsequent edge extraction.

[0061] In the formula, M(u,v) represents the gradient magnitude of a pixel. u is the x-coordinate of the image pixel, in pixels. v is the y-coordinate of the image pixel, in pixels. G_x represents the gradient component of the pixel in the horizontal direction. G_y represents the gradient component of the pixel in the vertical direction. sqrt indicates taking the square root of the value within the parentheses. θ(u,v) represents the gradient vector direction of the pixel, in radians. arctan indicates taking the arctangent of the value within the parentheses.

[0062] To overcome the resolution bottleneck of physical pixels and achieve high-precision sub-pixel positioning of target features, the sub-pixel edge reconstruction and geometric solution method in the S3 sub-pixel edge refinement and geometric solution in this embodiment is as follows: the coarse edge point set is repositioned using the Zernike orthogonal moment operator to correct the integer pixel coordinates to sub-pixel coordinates, and a sub-pixel level edge point set is constructed. The sub-pixel level edge point set is initially fitted and outlier is removed using the second-order robust least squares method. The filtered clean edge point set is then used for secondary fitting to output the sub-pixel level geometric feature vector of the target.

[0063] The Zernike orthogonal moment operator utilizes the rotation invariance of Zernike orthogonal moments to establish an analytical relationship between image moments and edge geometric parameters. Within a dynamically generated neighborhood window, it performs a mapping from pixels to a unit circle and calculates the three lower-order moments of the Zernike orthogonal moments: the zeroth, first, and second moments. These three lower-order moments are used to solve for the sub-pixel geometric parameters of the edge line relative to the current pixel center, including the normal rotation angle of the ideal step edge and the perpendicular distance from the edge line to the current pixel center. The normal rotation angle is the arctangent of the imaginary part of the first moment divided by the real part of the first moment. The perpendicular distance is the square root of the sum of three times the second moment divided by the square of eight times the real part of the first moment plus the square of the imaginary part of the first moment. Based on the calculated vertical distance and normal angle, the original integer pixel coordinates are compensated along the edge normal direction, thereby correcting them into sub-pixel coordinates with spatial continuity. The horizontal coordinate correction value is the original horizontal coordinate plus the cosine value of the vertical distance multiplied by the normal angle, and the vertical coordinate correction value is the original vertical coordinate plus the sine value of the vertical distance multiplied by the normal angle. By traversing all coarse edge points and performing the above calculations, a high-precision fine edge point set is finally constructed.

[0064] To quantify the calculation of the geometric parameters of the edge, this embodiment completes the calculation using the following formula:

[0065] The above formula is used to calculate the normal rotation angle of the ideal step edge and the vertical distance from the edge line to the current pixel center, providing parameter support for subsequent sub-pixel coordinate correction.

[0066] In the formula, φ is the normal rotation angle of the ideal step edge, in radians, reflecting the instantaneous direction of the edge tangent. arctan represents taking the arctangent of the value within the parentheses. Im(A_11) is the imaginary part of the first-order complex moment A_11. Re(A_11) is the real part of the first-order complex moment A_11. l is the perpendicular distance from the edge line to the center of the current pixel, in pixels. A_20 is the second-order Zernike orthogonal moment. sqrt represents taking the square root of the value within the parentheses.

[0067] Based on the calculated vertical distance and normal angle, the original integer pixel coordinates are compensated along the edge normal direction, thereby correcting them into sub-pixel coordinates with spatial continuity. To quantify the coordinate correction, this embodiment calculates the sub-pixel coordinates using the following formula:

[0068] The above formula is used to compensate the original integer pixel coordinates along the edge normal direction, correct them to sub-pixel coordinates, and construct a high-precision sub-pixel edge point set.

[0069] In the formula, x'_i is the corrected sub-pixel x-coordinate of the i-th edge point, in pixels. y'_i is the corrected sub-pixel y-coordinate of the i-th edge point, in pixels. x_i is the original integer pixel x-coordinate of the i-th edge point, in pixels. y_i is the original integer pixel y-coordinate of the i-th edge point, in pixels. l is the vertical distance from the edge line to the center of the current pixel, in pixels. cos represents taking the cosine of the angle within the parentheses. sin represents taking the sine of the angle within the parentheses. φ is the normal rotation angle of the ideal step edge, in radians.

[0070] Compared to traditional interpolation methods, this method achieves a balance between accuracy and search domain tolerance at the algorithm level by dynamically adjusting the window size, which significantly improves the adaptability of the visual servoing system under non-steady-state conditions. Since it uses all pixel information within the local window for integration calculation, rather than relying solely on gradient extreme points, it can effectively resist grayscale fluctuations caused by minor scratches on the metal surface and uneven illumination.

[0071] The second-order robust least squares method is used for circular targets. The standard Euclidean geometric equation for a circular target is: the x-coordinate minus the square of the x-coordinate of the center plus the y-coordinate minus the square of the y-coordinate of the center equals the square of the radius. To achieve efficient solution, this equation is expanded and linearized, constructing an overdetermined system of equations. Each row of the coefficient matrix of the overdetermined system consists of twice the x-coordinate, twice the y-coordinate, and a '-'. The parameter vector to be solved consists of the x-coordinate of the center, the y-coordinate of the center, and a constant term. The mapping relationship between the constant term and the radius is: the radius equals the square root of the constant term plus the square of the square of the x-coordinate of the center plus the square of the square of the y-coordinate of the center. After solving the parameter vector using the least squares method, the local quality is evaluated by calculating the radial residual from each edge point to the fitted circle. The radial residual is the absolute value of the difference between the distance from the edge point to the fitted circle center and the fitted radius.

[0072] To achieve efficient solution, the equations are expanded and linearized, resulting in the following overdetermined system of equations:

[0073] In the formula, Subpixel edge coordinates; Let be the parameter vector to be solved, where As the initial center, constant term With radius The mapping relationship is Solve using the least squares method. Then, the radial residuals from each edge point to the fitted circle were calculated. To assess local quality:

[0074] Following outlier removal and secondary fine-fitting, an elastic robustness threshold is introduced. If the radial residual of an edge point exceeds the robustness threshold, it is identified as an abnormal interference point and removed. Finally, a second least-squares fitting is performed using the filtered clean point set to directly extract sub-pixel-level geometric radius and centroid coordinates, outputting the final geometric feature vector. The standard value for the robustness threshold is 1.5 pixels, which can be increased to 2.5 pixels under high dynamic conditions. The robustness threshold needs to balance positioning accuracy and tracking continuity. Under standard conditions, a value of 1.5 pixels can pass rigorous geometric consistency checks, remove small radial deviation points, and ensure sub-pixel accuracy of the solution. If the threshold is too small, effective edge points will be mistakenly removed, resulting in insufficient fitting samples. If the threshold is too large, interference points cannot be effectively removed, affecting positioning accuracy. Under high dynamic conditions, increasing the threshold to 2.5 pixels can prevent effective edge points from being mistakenly identified as outliers due to motion blur or sudden changes in lighting, thus ensuring the continuity of feature extraction under complex conditions.

[0075] To achieve temporal smoothing of feature signals and effective compensation for perception delay, and to provide stable and continuous feedback signals for visual servo control, the specific implementation steps of temporal feedback enhancement and delay compensation in this embodiment include: S41 Normalizing sub-pixel-level geometric feature vectors based on a pinhole camera model, constructing the system's state vector and linear state transition equations, and establishing a Kalman filter observation model; S42 Smoothing and motion state estimation of target features using a Kalman state observer, outputting feature evolution rate estimates and intermediate filtering parameters; S43 Generating feedback adjustment commands based on the intermediate filtering parameters, adjusting the algorithm parameters of the front-end sub-pixel edge refinement and geometric solution steps in reverse, and constructing a perception-prediction adaptive closed loop; S44 Combining the feature evolution rate estimates and the cumulative delay of the perception link, performing kinematic feedforward prediction of the target feature state, and outputting a synchronization alignment feature signal for visual servoing.

[0076] In step S41, to enhance the algorithm's adaptability to sensors with different resolutions, the extracted geometric feature vector is first normalized using camera intrinsic parameters. Based on the inverse transformation of the pinhole camera model, the state components in the normalized coordinate system are obtained. The normalized abscissa is the centroid abscissa minus the camera principal point abscissa, divided by the pixel's horizontal focal length. The normalized ordinate is the centroid ordinate minus the camera principal point ordinate, divided by the pixel's vertical focal length. The normalized radius is the feature radius divided by the square root of the product of the horizontal and vertical pixel equivalents. Combining the image feature change rate, the system's state vector is constructed. The state vector consists of the normalized abscissa, normalized ordinate, normalized radius, normalized abscissa change rate, normalized ordinate change rate, and normalized radius change rate.

[0077] To enhance the algorithm's adaptability to sensors with different resolutions, the extracted geometric feature vectors are first processed using camera intrinsic parameters. Normalization is performed. Based on the inverse transformation of the pinhole camera model, the state components in the normalized coordinate system are obtained:

[0078] In the formula, As the camera's main point, Focal length in pixels. Combined with the rate of change of image features. Construct the system's state vector .

[0079] Based on the kinematic consistency assumption, it is assumed that the target features move at approximately a constant speed in the image plane within an extremely short sampling period. Accordingly, a linear state transition equation is established. The state transition equation is that the current state vector is equal to the state transition matrix multiplied by the state vector of the previous time step plus the process excitation noise. The state transition matrix is ​​composed of a third-order identity matrix, a sampling period multiplied by a third-order identity matrix, a third-order zero matrix, and a third-order identity matrix in blocks. The process excitation noise is used to characterize the model deviation caused by minute vibrations and non-ideal motions during the operation of the work platform.

[0080] Based on the kinematic consistency assumption, it is believed that within an extremely short sampling period... Within the image plane, the target features move at approximately a constant velocity, and a linear state transition equation is established based on this:

[0081] In the formula, This is the state transition matrix, which reflects the kinematic derivation logic of the features over time; This is the process excitation noise, used to characterize the model deviations caused by minute vibrations and non-ideal motions during the operation of the work platform. The specific form is defined as follows:

[0082] Subsequently, an observation model was established. The observation value of each frame of the system directly comes from the sub-pixel feature quantity obtained by the hierarchical extraction strategy. The observation equation is that the observation value at the current time is equal to the observation matrix multiplied by the state vector at the current time plus the measurement noise. The observation matrix is ​​composed of a third-order identity matrix and a third-order zero matrix. The measurement noise is the random measurement noise of the image sensor, and its statistical characteristics are represented by the covariance matrix. The covariance matrix is ​​a diagonal matrix, and the diagonal elements are the variances of the horizontal axis component, the vertical axis component, and the radius component, respectively. The value of the covariance matrix is ​​determined based on actual measurement statistics. By performing mathematical statistical characterization on the jitter residual of the sub-pixel feature extraction algorithm under static conditions, the variance of the feature components is established. This noise modeling method based on actual measurement accuracy ensures that the filter can suppress pixel quantization interference to the greatest extent, providing a smooth and continuous motion gradient for the servo layer, and eliminating the risk of actuator jitter from the physical source.

[0083] Step S42 uses a Kalman state observer to perform temporal smoothing and motion state estimation on the target features, executing prediction and update steps. In the prediction step, based on the state estimate and state transition matrix from the previous time step, the prior state estimate for the current time step is calculated. Based on the estimated covariance matrix and state transition matrix from the previous time step, the prior covariance matrix for the current time step is calculated. In the update step, the prediction residual is calculated based on the prior state estimate and the observation matrix. The Kalman gain is calculated based on the prior covariance matrix, the observation matrix, and the measurement noise covariance matrix. The state estimate and covariance matrix are updated based on the Kalman gain and the prediction residual. Finally, the feature evolution rate estimate and intermediate filtering parameters are output. In this embodiment, the system calculates the prediction covariance matrix based on the state transition matrix and the process noise covariance (corresponding) using the following formula:

[0084] Extract its traces This metric quantifies the degree of dispersion in the system's prediction of the target's current location.

[0085] Based on the invention objective of improving the system's environmental adaptability, in this embodiment, the feedback adjustment instruction in S43 includes at least a window adjustment instruction based on the trace generation of the predicted covariance matrix and a threshold adjustment instruction based on the generation of normalized observation residuals. The window adjustment instruction is used to dynamically intervene in the neighborhood window size of the Zernike orthogonal moment calculation, and the threshold adjustment instruction is used to dynamically adjust the robustness threshold of the second robust least squares method.

[0086] During each frame's filter update process, the system extracts intermediate filtering parameters in real time and establishes a feedback mapping to achieve reverse control of the sensing front end. First, it generates window adjustment instructions based on the trace of the predicted covariance matrix. The trace of the predicted covariance matrix quantifies the degree of dispersion of the system's prediction of the target's current position. It is injected back into the sub-pixel edge refinement step to dynamically drive the size adjustment of the Zernike moment extraction window. In steady-state high-precision mode, when the trace of the predicted covariance matrix is ​​less than a preset threshold, it indicates that the target movement is stable and the prediction is accurate. The system sets the neighborhood window size for Zernike moment calculation to multiply by 1 to filter out small background noise to the greatest extent and ensure fine positioning. In dynamic fault-tolerant mode, when the trace of the predicted covariance matrix increases, it indicates that the robot arm's rapid movement has increased the prediction uncertainty. The system automatically expands the calculation window to 11 by 11. This dynamic expansion mechanism ensures that when the prediction model has a slight shift or the edge has gradient dispersion due to movement, the search domain can still completely cover the distribution range of the physical edge, thereby avoiding feature loss. During each frame's filter update process, the system extracts the following intermediate parameters in real time and establishes a feedback mapping to achieve reverse control of the sensing front end: Based on the adaptive feedback of the search window due to prediction uncertainty, this feedback is injected back into the dynamically driving Zernike moment extraction window. The size can be adjusted to enable the physical search domain to expand automatically as prediction uncertainty increases.

[0087]

[0088] This residual reflects in real time the deviation between the physical reality captured by the visual sensor and the expected motion of the system. Normalization of this residual yields... Post-injection drive robustness threshold Dynamic adjustment.

[0089] The choice of window size needs to balance computational efficiency, positioning accuracy, and search coverage. A multiplied window size can balance computational efficiency and positioning accuracy, cover enough pixel information to accurately solve edge parameters, and avoid introducing too much noise. An 11x11 window size can expand the search range and effectively capture edge information under high dynamic conditions. If the window size is too small, it cannot cover the complete edge distribution. If the window size is too large, it will introduce too much background interference and increase computational overhead.

[0090] Based on the threshold adjustment command generated by the normalized observation residual, the observation residual reflects in real time the deviation between the physical reality captured by the visual sensor and the expected system motion. After normalization, it is injected into the geometric solution step to drive the dynamic adjustment of the robust threshold. In quasi-static high-precision mode, when the normalized observation residual is in a low range, indicating stable target motion, the threshold is automatically tightened to 1.5 pixels. Through rigorous geometric consistency verification, tiny radial deviation points are eliminated to ensure sub-pixel accuracy of the solution. In high-dynamic fault-tolerant mode, when the normalized observation residual increases significantly, indicating rapid movement of the robotic arm or drastic changes in working conditions, the threshold is automatically and elastically increased to 2.5 pixels to prevent effective edge points from being misidentified as outliers due to motion blur or sudden changes in light and shadow, thus ensuring the continuity of feature extraction under complex working conditions. Through the above feedback adjustment mechanism, a logic of prediction-guided perception and perception-corrected prediction is established, realizing real-time online optimization of perception parameters. This enables the robot to autonomously adapt to extreme working conditions such as variable acceleration motion and drastic changes in light and shadow, achieving an optimal allocation between accuracy and robustness.

[0091] To compensate for the phase lag caused by image processing and achieve real-time synchronous alignment of dynamic targets, the feedforward prediction calculation method in S44 of this embodiment combines the feature evolution rate estimate with the cumulative delay of the perception link to extrapolate the predicted feature state of the target at the control execution time, thus compensating for the phase lag caused by image processing. Considering the cumulative delay of the perception link from image acquisition, semantic model inference to fine positioning, the observation values ​​acquired by the visual sensor essentially reflect the historical state of the target. If the current observation values ​​are directly used for servoing, it will cause a significant timing lag at the actuator end. To achieve synchronization between the perception flow and the execution end, the feature evolution rate estimate output by the Kalman filter is used to synchronously extrapolate the feature state at the control execution time. The predicted feature state is equal to the corrected feature position estimate at the current time plus the feature evolution rate estimate multiplied by the cumulative delay of the perception link. This mechanism not only predicts the expected image state of the target features at the control execution time but also calculates the feature velocity estimate at the corresponding time. Finally, the predicted feature state is used as a feedback reference input to the visual servo controller to compensate for the phase lag in the closed-loop control and achieve real-time high-fidelity synchronous alignment of dynamic targets.

[0092] Considering the cumulative delay in the perception link caused by image acquisition, YOLO inference, and fine localization The observations acquired by visual sensors essentially reflect the historical state of the target. If used directly... Servoying the observations at specific times will cause a significant timing lag at the actuator end. To achieve synchronization between the sensing flow and the actuator, this study utilizes the characteristic evolution rate estimate output by the Kalman filter. ,right Synchronous extrapolation of the characteristic states at each moment:

[0093] In the formula, This is a revised estimate of the feature position at the current moment. This mechanism not only predicts the expected image state of the target features at the control execution moment, but also calculates the feature velocity estimate at the corresponding moment. Finally, the predicted feature states will be... As a feedback reference input to the visual servo controller, it cancels out the phase lag in the closed-loop control, and achieves real-time high-fidelity synchronous alignment of dynamic targets.

[0094] Through the coordinated execution of all the above steps, lightweight semantic target detection first achieves stable target capture in complex unstructured industrial scenarios, while reducing the computational overhead of the model, making it adaptable to the operational needs of industrial edge computing devices with limited computing power. This solves the problem that deep convolutional neural network models in existing technologies are difficult to run in real time on industrial edge devices.

[0095] By combining dynamic ROI extraction and target locking with sub-pixel edge refinement and geometric calculation, high-precision feature localization that breaks through the pixel-level resolution limitation is achieved. At the same time, it effectively avoids the feature mismatch problem caused by homogeneous targets, improves the stability and continuity of the feature extraction process in unstructured interference environments, and solves the problems of geometric feature detection technology being susceptible to environmental interference failure and semantic detection technology having insufficient localization accuracy in existing technologies.

[0096] By enhancing the time-domain feedback and compensating for the time delay, effective compensation for the processing delay of the visual perception link is achieved, suppressing the random jitter of the feature signal in the time domain, providing a stable and continuous high-precision feedback signal for the visual servo control of industrial robots, adapting to the comprehensive performance requirements of industrial precision assembly operations, and solving the problem that existing technologies cannot simultaneously achieve stable capture and high-precision positioning of target features.

[0097] The above descriptions are merely some embodiments of this application and are not intended to limit this application. The technical features or structures in the foregoing different embodiments can be arbitrarily combined to form other specific technical solutions as needed. For those skilled in the art, this application can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of the claims of this application.

Claims

1. A sub-pixel feature extraction method based on semantic-geometric dual perception, characterized in that, include: S1, lightweight semantic object detection, performs global detection on the input industrial scene image, outputs pixel-level prediction boxes of the target, and establishes global semantic constraints; S2, Dynamic ROI extraction and target locking: Based on the pixel-level prediction box, a dynamic region of interest (ROI) is constructed. Through the multi-dimensional scoring mechanism of task topology constraints and temporal correlation logic, the current task target is uniquely locked. S3, Subpixel edge refinement and geometric solution: Reconstruct the target edge within the locked ROI, output a subpixel level edge point set, fit the subpixel level edge point set using the second-order robust least squares method, and output the target's subpixel level geometric feature vector. S4, Temporal feedback enhancement and time delay compensation: The sub-pixel level geometric feature vector is smoothed in the temporal domain and its motion state is estimated by a Kalman state observer to construct a perception-prediction adaptive closed loop. At the same time, the target feature state is fed forward to predict and output a synchronous alignment feature signal for visual servoing.

2. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 1, characterized in that, S1 includes: S11, Construct an improved YOLOv8s lightweight detection model as the perception front end; S12 introduces the GhostNet design concept into the model backbone network, generating phantom feature maps through GhostConv and C3Ghost modules to compress the model's computational load; S13 embeds the CBAM attention mechanism at the end of feature extraction, which enhances the perception of the target's physical structure through channel and spatial dimension recalibration. S14, the WIoUv3 loss function with dynamic non-monotonic focusing mechanism is used to optimize the bounding box regression; S15 uses a trained lightweight detection model to perform global detection on the input industrial scene image, outputs the pixel-level predicted bounding box coordinates of the target, and establishes global semantic constraints.

3. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 1, characterized in that, S2 includes: S21. Based on the pixel-level prediction box output in step S1, an expansion coefficient is introduced to construct a dynamic region of interest (ROI) to ensure complete coverage of the target edge information. S22. For array-type homogeneous targets, a task topology scoring function is constructed, which comprehensively considers the polar angle deviation, roundness index and confidence of candidate targets, and selects the optimal target for the current process by matching it with the preset task template sequence. S23, based on the principle of minimum spatial offset, calculates the Euclidean displacement deviation of targets in adjacent frames to establish a temporal correlation mechanism, and uniquely locks the current operation target.

4. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 3, characterized in that, S2 further includes: S24, Dynamic tracking window construction: A local tracking window is generated based on the locked target center, and image information outside the window is forcibly suppressed, retaining only the target area within the window for subsequent processing.

5. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 3, characterized in that, The task topology scoring function in S22 comprehensively considers three parameters of the candidate target: polar angle deviation, roundness index, and confidence level. It also assigns corresponding weight coefficients to the three parameters and determines the optimal target for the current process by minimizing the value of the task topology scoring function.

6. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 1, characterized in that, In step S3, before reconstructing the target edge, the image within the locked ROI is preprocessed. The preprocessing steps include at least: Local image enhancement involves non-linear brightness stretching of the ROI image through Gamma correction to suppress high-gloss reflections from metal surfaces. Pixel-level coarse edge extraction utilizes the Canny operator to calculate the image gradient magnitude and gradient direction, and extracts the coarse edge point set of the target through non-maximum suppression and double thresholding algorithms; Contour selection involves filling edge breaks through morphological closing operations and filtering candidate contours based on target geometric priors to obtain the baseline edge for sub-pixel reconstruction.

7. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 1, characterized in that, The sub-pixel edge reconstruction and geometric solution method in S3 is as follows: The Zernike orthogonal moment operator is used to perform subpixel relocalization on the coarse edge point set, correcting the integer pixel coordinates to subpixel coordinates and constructing a subpixel level edge point set. The subpixel level edge point set is initially fitted and outlier is removed using the second-order robust least squares method. The filtered clean edge point set is then used for secondary fitting to output the target's subpixel level geometric feature vector.

8. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 1, characterized in that, S4 includes: S41, based on the pinhole camera model, the sub-pixel level geometric feature vector is normalized, the system's state vector and linear state transition equation are constructed, and a Kalman filter observation model is established; S42 uses a Kalman state observer to perform temporal smoothing and motion state estimation on target features, and outputs the feature evolution rate estimate and intermediate filtering parameters. S43 generates feedback adjustment instructions based on intermediate filtering parameters, and reversely adjusts the algorithm parameters of the front-end sub-pixel edge refinement and geometric solution steps to construct a perception-prediction adaptive closed loop. S44 combines the feature evolution rate estimate with the cumulative delay of the sensing link to perform kinematic feedforward prediction of the target feature state and outputs a synchronous alignment feature signal for visual servoing.

9. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 8, characterized in that, The feedback adjustment command in S43 includes at least: A window adjustment instruction based on the trace generation of the predicted covariance matrix is ​​used to dynamically intervene in the neighborhood window size of the Zernike orthogonal moment calculation; The threshold adjustment command based on the normalized observation residual is used to dynamically adjust the robust threshold of the second-order robust least squares method.

10. The sub-pixel feature extraction method based on semantic-geometric dual perception according to claim 8, characterized in that, The calculation method for the feedforward prediction in S44 is as follows: By combining the estimated feature evolution rate with the cumulative delay τ of the sensing link, the predicted feature state of the target at the control execution time is extrapolated, thus offsetting the phase lag caused by image processing.

Images