A visual target deformation feature recognition method and system

Abstract

The application relates to the field of computer vision and image recognition technology, and discloses a visual target deformation feature recognition method and system. The method generates scale parameters, rotation parameters and local deformation parameters based on an input feature map, adjusts the distance and direction of reference sampling points to form direction-scale transformation coordinates, and superimposes the local deformation parameters to obtain initial adaptive sampling points; the difference between the initial adaptive sampling points and the direction-scale transformation coordinates is taken as local deformation displacement, and a graph Laplacian operator is used for consistency constraint according to the adjacency relationship of the sampling nodes; the constrained sampling points are taken as the center of an elliptical Gaussian kernel, the rotation parameters and the scale parameters are reused to determine the aggregation direction and range, the neighborhood features are weighted and aggregated, a deformation adaptive feature map and a target recognition result are generated. The application constrains local deformation while retaining the direction-scale structure, and makes the sampling position and the feature aggregation area keep geometric linkage.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image recognition technology, and in particular to a method and system for recognizing the deformable features of visual targets. Background Technology

[0002] In the field of computer vision and image recognition, visual target recognition typically involves first converting the image to be recognized into an input feature map using a feature extraction network, then extracting local features according to a fixed sampling grid or variable sampling positions, and finally inputting the extraction results into target classification, target localization, or target key point prediction modules. While this approach can perform feature extraction and recognition of conventional targets, it is prone to problems such as sampling positions deviating from the target region, unstable spatial relationships between adjacent sampling points, and inconsistencies between the local feature coverage and the target's structural orientation when the target undergoes rotation, scale changes, local bending, or partial occlusion.

[0003] Existing deformable feature extraction schemes mostly rely on adjusting the sampling position using independent offset parameters corresponding to each sampling point, or on processing target pose changes by rotating and scaling the candidate region, template, or input image as a whole. When different sampling points move according to local parameters, the original adjacency relationship between sampling points is usually not directly used for coordinate constraints, and adjacent sampling points may produce discontinuous displacement, local clustering, or excessive separation. When the target contains slender structures, continuously curved structures, or irregular boundaries, the interpolation features at a single sampling coordinate are also difficult to correspond to the continuous distribution of the target in a specific direction.

[0004] Furthermore, some schemes set separate parameters for sampling location adjustment and neighborhood feature aggregation. Even after the sampling points change with the target direction and scale, the subsequent feature reading area may still use a fixed direction or range, resulting in a lack of correspondence between the sampling center, sampling geometry, and the neighborhood feature coverage area. Therefore, it is necessary to address the problem of constraining local deformation displacement while preserving the direction-scale structure formed by scale and rotation, and ensuring that the sampling location and the region feature aggregation direction and range use the same source geometric parameters. Summary of the Invention

[0005] To address the above problems, this invention provides a visual target deformation feature recognition method and system, which, while preserving the orientation-scale structure formed by scale parameters and rotation parameters, implements spatial consistency constraints on the local deformation displacement of adjacent sampling points, and reuses the same source scale parameters and rotation parameters to determine the region aggregation kernel.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] S100: Obtain the input feature map of the image to be recognized, determine the set of reference sampling points and the adjacency relationship of sampling nodes according to the reference sampling grid, and generate scale parameters, rotation parameters and local deformation parameters associated with each reference sampling point based on the input feature map;

[0008] S200. Adjust the distance of the reference sampling point relative to the sampling center according to the scale parameter, and adjust its direction according to the rotation parameter to obtain the direction-scale transformation coordinates; superimpose the local deformation parameter onto the corresponding direction-scale transformation coordinates to generate an initial adaptive sampling point set.

[0009] S300. The difference between each initial adaptive sampling point in the initial adaptive sampling point set and the corresponding direction-scale transformation coordinate is determined as the local deformation displacement. The difference between adjacent local deformation displacements is calculated based on the adjacency relationship of the sampling nodes, and the local deformation displacement is constrained using the graph Laplacian operator. The constrained local deformation displacement is superimposed with the corresponding direction-scale transformation coordinate to generate a constrained sampling point set.

[0010] S400. Using each constraint sampling point in the constraint sampling point set as the center of the elliptical Gaussian kernel, the rotation parameters and scale parameters associated with each constraint sampling point are reused to determine the direction and range of the corresponding elliptical Gaussian kernel; the neighborhood features in the input feature map are weighted and aggregated according to the elliptical Gaussian kernel to generate a deformation adaptation feature map, and a target recognition result is generated based on the deformation adaptation feature map.

[0011] Furthermore, the generation of the scale parameters, rotation parameters, and local deformation parameters includes:

[0012] The input feature map is convolved to generate a parameter tensor that is divided into parameter channel groups according to the reference sampling point identifier;

[0013] The scale parameters, rotation parameters, and local deformation parameters of the corresponding reference sampling points are extracted from each parameter channel group. Positive bounded mapping is performed on the scale parameters, angular bounded mapping is performed on the rotation parameters, and displacement limiting is performed on the local deformation parameters.

[0014] Further, adjusting the distance of the reference sampling point relative to the sampling center according to the scale parameter, and adjusting the orientation of the reference sampling point relative to the sampling center according to the rotation parameter, includes:

[0015] Convert the coordinates of the reference sampling point relative to the sampling center into radius and polar angle;

[0016] The radius is corrected according to the scale parameter, and the polar angle is corrected according to the rotation parameter;

[0017] The corrected radius and polar angle are converted into input feature map coordinates.

[0018] Furthermore, the determination of the initial adaptive sampling point and the local deformation displacement includes:

[0019] The lateral and longitudinal components of the local deformation parameters are superimposed onto the orientation-scale transformation coordinates to obtain initial adaptive sampling points; the difference between the initial adaptive sampling points and the corresponding orientation-scale transformation coordinates is determined as the local deformation displacement, and they are arranged according to the reference sampling point identifiers.

[0020] Furthermore, determining the adjacency relationship of the sampling nodes includes:

[0021] An adjacency matrix is ​​established based on the horizontal, vertical, and diagonal adjacency relationships of the reference sampling points in the reference sampling grid;

[0022] The adjacent initial adaptive sampling points in the initial adaptive sampling point set are determined based on the adjacency matrix.

[0023] Furthermore, constraining the local deformation displacement using the graph Laplacian operator includes:

[0024] A graph Laplacian operator L is constructed based on the adjacency matrix, and each local deformation displacement is arranged into a local deformation displacement vector U according to the reference sampling point identifier;

[0025] The constrained local deformation displacement vector U' is obtained by using (I + λL)U' = U, where I is the identity matrix and λ is the constraint weight.

[0026] Add each constrained local deformation displacement in the constrained local deformation displacement vector U' to the corresponding direction-scale transformation coordinate to generate a set of constrained sampling points;

[0027] The distance between adjacent constraint sampling points in the constraint sampling point set is detected; when the distance exceeds a preset range, the adjacent constraint sampling point corresponding to the distance is identified as an abnormal sampling point, and each abnormal sampling point is backed up to its corresponding direction-scale transformation coordinate.

[0028] Furthermore, the determination of the elliptic Gaussian kernel includes:

[0029] The constrained sampling point is taken as the center of the elliptical Gaussian kernel;

[0030] The major axis direction is determined based on the same rotation parameter involved in generating the orientation-scale transformation coordinates, and the major and minor axes are determined based on the same scale parameter involved in generating the orientation-scale transformation coordinates.

[0031] Construct a covariance matrix based on the major axis direction, major axis, and minor axis.

[0032] Furthermore, the weighted aggregation includes:

[0033] The Mahalanobis distance between the candidate feature positions and the center of the elliptical Gaussian kernel is calculated based on the covariance matrix of the elliptical Gaussian kernel, and the feature positions whose Mahalanobis distance is not greater than a preset cutoff distance are retained.

[0034] The spatial weights of each preserved feature location are calculated based on the Mahalanobis distance, and the spatial weights are then normalized.

[0035] Multiply the features at each retained feature location with their corresponding spatial weights and sum them to generate local aggregated features;

[0036] The local aggregated features are arranged according to the reference sampling point identifier and the output spatial position to generate the deformation adaptation feature map.

[0037] Further, generating the target recognition result based on the deformation adaptation feature map includes:

[0038] The deformation adaptation feature map is input into the target recognition module to perform target category prediction, target location prediction, and target key point prediction, respectively.

[0039] The target category, target location, target key points, and recognition confidence are correlated to generate the target recognition result.

[0040] The present invention also provides a visual target deformation feature recognition system, comprising:

[0041] The feature map acquisition module is used to acquire the input feature map, the set of reference sampling points, and the adjacency relationship of the sampling nodes;

[0042] The geometric parameter generation module is used to generate scale parameters, rotation parameters, and local deformation parameters associated with each reference sampling point;

[0043] The sampling point transformation module is used to generate orientation-scale transformation coordinates based on the scale parameters and rotation parameters, and to superimpose the local deformation parameters onto the corresponding orientation-scale transformation coordinates to generate an initial adaptive sampling point set.

[0044] The spatial consistency constraint module is used to determine the difference between each initial adaptive sampling point in the initial adaptive sampling point set and the corresponding direction-scale transformation coordinate as the local deformation displacement, constrain the local deformation displacement using the graph Laplacian operator based on the adjacency relationship of the sampling nodes, and superimpose the constrained local deformation displacement with the corresponding direction-scale transformation coordinate to generate a constrained sampling point set.

[0045] The region aggregation kernel generation module is used to determine the direction and range of the corresponding elliptical Gaussian kernel by reusing the rotation parameters and scale parameters associated with each constraint sampling point in the set of constraint sampling points as the center of the elliptical Gaussian kernel.

[0046] The region feature aggregation module is used to perform weighted aggregation of neighborhood features in the input feature map based on the elliptical Gaussian kernel to generate a deformation adaptation feature map.

[0047] The target recognition module is used to generate target recognition results based on the deformation adaptation feature map.

[0048] The key innovations of this invention include:

[0049] (1) The orientation-scale transformation coordinates formed by the scale parameters and rotation parameters are separated from the local deformation displacements formed by the local deformation parameters. The initial adaptive sampling points are obtained by superimposing the two, so that the orientation-scale transformation and the local residuals have a clear data relationship.

[0050] (2) Based on the adjacency relationship of sampling nodes, only the local deformation displacement difference between adjacent sampling points is subject to graph Laplace consistency constraint, and the constrained local deformation displacement is superimposed on the orientation-scale transformation coordinates to avoid directly smoothing the absolute coordinates and weakening the existing orientation-scale structure.

[0051] (3) Using the constrained sampling point as the center of the elliptical Gaussian kernel, the same rotation parameter and scale parameter used in generating the direction-scale transformation coordinates are reused to determine the kernel direction and range, and the neighborhood features are weighted and aggregated according to the Mahalanobis distance to form a processing closed loop that links sampling position constraints with regional feature reading.

[0052] The following are its main beneficial effects:

[0053] (1) By distinguishing between direction-scale transformation coordinates and local deformation displacement, the sampling position instability caused by the mixing of different geometric changes in ordinary two-dimensional offset is reduced, and each parameter has a clear purpose in subsequent constraint and aggregation steps.

[0054] (2) By constraining only the local deformation displacement, the local displacement that deviates from the change trend of adjacent nodes is corrected, reducing the probability of local clustering, excessive separation or discontinuous movement of sampling points, while preserving the orientation-scale structure formed by the scale parameter and rotation parameter.

[0055] (3) By constraining the sampling points and the same source rotation parameters and scale parameters, the center, direction and range of the elliptical Gaussian kernel are defined, so that the neighborhood feature reading area changes synchronously with the sampling geometry. By weighting and aggregating multiple neighborhood features according to Mahalanobis distance, the sensitivity of single-point interpolation to coordinate deviation can be reduced and the local structural directionality can be preserved. Attached Figure Description

[0056] Figure 1 A flowchart illustrating a visual target deformation feature recognition method provided in an embodiment of this application;

[0057] Figure 2 This application provides a structural block diagram of a visual target deformation feature recognition system.

[0058] Figure 3 This is a schematic diagram of a feature deformation sampling and region aggregation structure provided in an embodiment of this application. Detailed Implementation

[0059] To enable those skilled in the art to better understand the technical solution, the present invention will be described in detail below with reference to embodiments. The description in this part is only exemplary and explanatory, and should not be used to limit the scope of protection of the present invention in any way.

[0060] Example 1: Refer to Figure 1 This is a flowchart illustrating a visual target deformation feature recognition method provided in an embodiment of the present invention. The flowchart may include at least steps S100-S400:

[0061] S100: Obtain the input feature map of the image to be recognized, determine the set of reference sampling points and the adjacency relationship of sampling nodes according to the reference sampling grid, and generate scale parameters, rotation parameters and local deformation parameters associated with each reference sampling point based on the input feature map;

[0062] S200. Adjust the distance of the reference sampling point relative to the sampling center according to the scale parameter, and adjust its direction according to the rotation parameter to obtain the direction-scale transformation coordinates; superimpose the local deformation parameter onto the corresponding direction-scale transformation coordinates to generate an initial adaptive sampling point set.

[0063] S300. The difference between each initial adaptive sampling point in the initial adaptive sampling point set and the corresponding direction-scale transformation coordinate is determined as the local deformation displacement. The difference between adjacent local deformation displacements is calculated based on the adjacency relationship of the sampling nodes, and the local deformation displacement is constrained using the graph Laplacian operator. The constrained local deformation displacement is superimposed with the corresponding direction-scale transformation coordinate to generate a constrained sampling point set.

[0064] S400. Using each constraint sampling point in the constraint sampling point set as the center of the elliptical Gaussian kernel, the rotation parameters and scale parameters associated with each constraint sampling point are reused to determine the direction and range of the corresponding elliptical Gaussian kernel; the neighborhood features in the input feature map are weighted and aggregated according to the elliptical Gaussian kernel to generate a deformation adaptation feature map, and a target recognition result is generated based on the deformation adaptation feature map.

[0065] Figure 3 This embodiment illustrates the processing relationships of generating geometric parameters from the input feature map, completing sampling point transformation and spatial consistency constraints, and performing region feature aggregation based on an elliptical Gaussian kernel. Figure 3 The geometric parameter generation branch corresponds to the generation process of scale parameters, rotation parameters, and local deformation parameters; the sampling point transformation and constraint module corresponds to the adjustment of the position of the reference sampling point and the consistency constraint process of the graph Laplacian space based on the adjacency relationship of the sampling nodes; the region aggregation kernel generation and feature aggregation part corresponds to the constraint sampling point set, elliptical Gaussian kernel set, and neighborhood feature weighted aggregation process.

[0066] S100: Obtain the input feature map of the image to be recognized, determine the set of reference sampling points and the adjacency relationship of sampling nodes according to the reference sampling grid, and generate scale parameters, rotation parameters and local deformation parameters associated with each reference sampling point based on the input feature map;

[0067] In one implementation, the visual target deformation feature recognition system is deployed in a robot vision controller, edge computing device, or image processing server, and an image acquisition device is connected to the visual target deformation feature recognition system. After the image acquisition device completes the exposure and transmission of one frame of the image to be recognized, the feature map acquisition module reads the image to be recognized and inputs it into a pre-stage feature extraction network. The pre-stage feature extraction network performs convolution operations according to a preset level and outputs an input feature map. The input feature map is represented as a feature tensor with height, width, and channel dimensions, where each spatial location corresponds to a receptive region in the image to be recognized, and each channel stores the image feature response of that receptive region.

[0068] The input feature map is not limited to the final output of the preceding feature extraction network. In one embodiment, the feature map acquisition module reads the input feature map from a preset feature layer of the backbone network, ensuring that the input feature map retains target edges, local textures, and spatial relationships. In another embodiment, the feature map acquisition module reads the feature map after multi-scale feature fusion. The feature map acquisition module establishes a correspondence between the coordinates of the image to be identified and the coordinates of the input feature map according to the step size of the selected feature layer. This correspondence is used when the target position and target key points are subsequently transformed from the input feature map coordinates to the coordinates of the image to be identified. If the preset feature layer does not output a valid feature map, or if non-numerical elements appear in the input feature map, the current frame does not enter the sampling point transformation process; the system retains the image identifier to be identified and re-executes feature extraction after the next frame arrives.

[0069] The reference sampling grid is determined by the sampling specifications of the feature extraction operator. For an output spatial location in the input feature map, the feature map acquisition module uses the coordinates corresponding to that output spatial location as the sampling center and arranges multiple reference sampling points around the sampling center. The reference sampling points are represented by coordinates relative to the sampling center, and their positions remain unchanged before geometric transformation. In one embodiment, the reference sampling grid is arranged in a 3×3 pattern, with 9 reference sampling points corresponding to the sampling center and 8 locations around it; in another embodiment, the reference sampling grid is arranged according to other rules based on the kernel size of the feature extraction operator. Each reference sampling point is configured with a reference sampling point identifier, which remains consistent throughout parameter extraction, sampling point transformation, spatial consistency constraints, and region feature aggregation.

[0070] The feature map acquisition module establishes adjacency relationships between sampling nodes based on the arrangement of reference sampling points in the reference sampling grid. Specifically, connections are established between horizontally adjacent, vertically adjacent, and diagonally adjacent reference sampling points, and an adjacency matrix is ​​formed based on the connection results. The rows and columns of the adjacency matrix correspond to the identifiers of the reference sampling points; when two reference sampling points have a preset adjacency relationship, the corresponding matrix element takes the connection weight; when two reference sampling points do not have a preset adjacency relationship, the corresponding matrix element is zero. In one implementation, all connections use the same weight; in another implementation, horizontal and vertical connections use a first weight, and diagonal connections use a second weight less than the first weight. This adjacency matrix is ​​pre-stored along with the reference sampling grid and reconstructed when the reference sampling grid changes, without being repeatedly changed with each frame of the image.

[0071] After receiving the input feature map, the geometric parameter generation module performs a convolution operation on the input feature map to obtain a parameter tensor with a corresponding spatial location to the input feature map. The parameter tensor is divided into multiple parameter channel groups according to the reference sampling point identifier. Each parameter channel group corresponds to a reference sampling point and includes scale parameters, rotation parameters, and local deformation parameters. In one embodiment, each reference sampling point corresponds to one scale parameter, one rotation parameter, and a local deformation parameter composed of horizontal and vertical components; when the number of reference sampling points is K, the parameter tensor contains 4K parameter channels at each spatial location. The geometric parameter generation module reads the corresponding parameter channel group according to the reference sampling point identifier, ensuring that the three types of parameters at the same reference sampling point remain correlated in subsequent steps.

[0072] The scale parameter represents the distance adjustment of the reference sampling point relative to the sampling center, the rotation parameter represents the orientation adjustment of the reference sampling point relative to the sampling center, and the local deformation parameter represents the local coordinate change applied to a single sampling point after the orientation and distance adjustments are completed. In one implementation, the geometric parameter generation module performs bounded mapping on the original scale parameter, ensuring that the mapped scale parameter is within a preset scale range; performs angular mapping on the original rotation parameter, ensuring that the mapped rotation parameter is within a preset angle range; and performs amplitude limiting on the lateral and longitudinal components of the local deformation parameter. If the parameter channel corresponding to a certain spatial location contains non-numerical elements or the absolute value of the parameter exceeds the configuration boundary, then the scale parameter of that spatial location is set to a unit scale, the rotation parameter is set to zero, and the local deformation parameter is set to zero; the features of this spatial location are read by subsequent steps according to the reference sampling grid.

[0073] During the training phase, the recognition task loss corresponding to the target category label, target location label, or target key point label is combined with the consistency regularization term corresponding to the difference in adjacent local deformation displacements to update the network parameters of the preceding feature extraction network and the geometric parameter generation module. The consistency regularization term is calculated based on the adjacency relationship of the sampling nodes. After training is completed, the recognition phase generates target recognition results according to S100 to S400.

[0074] The input feature map, reference sampling point set, sampling node adjacency relationship, scale parameter, rotation parameter, and local deformation parameter together constitute the geometric sampling configuration result. The geometric sampling configuration result also retains the correspondence between the spatial location of the input feature map and the reference sampling point identifier, enabling S200 to read the reference sampling point and the three types of geometric parameters corresponding to the same location for each output spatial location in the input feature map. The sampling node adjacency relationship is used in S300 to find adjacent initial adaptive sampling points; the scale parameter and rotation parameter, in addition to participating in the sampling point position transformation in S200, also participate in the calculation of the region aggregation kernel direction and range in S400.

[0075] S200. Adjust the distance of the reference sampling point relative to the sampling center according to the scale parameter, and adjust its direction according to the rotation parameter to obtain the direction-scale transformation coordinates; superimpose the local deformation parameter onto the corresponding direction-scale transformation coordinates to generate an initial adaptive sampling point set.

[0076] After receiving the geometric sampling configuration result formed by S100, the sampling point transformation module performs sampling point transformation position by position according to the output spatial position of the input feature map. For the current output spatial position, the sampling point transformation module reads the sampling center, reference sampling point set, scale parameter, rotation parameter, and local deformation parameter corresponding to that position. The parameters between different output spatial positions are read independently, but each output spatial position uses the same reference sampling point identifier and reference sampling grid arrangement, thereby ensuring that the subsequently formed local aggregated features can still be arranged according to the spatial order of the input feature map.

[0077] For a reference sampling point identified as k, the sampling point transformation module first calculates the lateral and longitudinal coordinate differences between the reference sampling point and the sampling center, and then converts these differences into a radius and a polar angle. The radius represents the distance between the reference sampling point and the sampling center, and the polar angle represents the direction from the sampling center to the reference sampling point. The radius of the reference sampling point located at the sampling center is zero, and its polar angle is set to a preset initial value. When the radius is zero, the rotation parameters do not change the position of the reference sampling point, and the local deformation parameters still participate in subsequent coordinate correction.

[0078] The sampling point transformation module applies a scale parameter to the radius and a rotation parameter to the polar angle. In one embodiment, the corrected radius is obtained by multiplying the reference radius by the scale parameter, and the corrected polar angle is obtained by summing the reference polar angle and the rotation parameter. When the scale parameter is a unit scale, the distance between the reference sampling point and the sampling center remains unchanged; when the scale parameter is greater than a unit scale, the reference sampling point moves away from the sampling center along its original direction; when the scale parameter is less than a unit scale, the reference sampling point moves closer to the sampling center along its original direction. When the rotation parameter is zero, the direction of the reference sampling point remains unchanged; when the rotation parameter is positive or negative, the reference sampling point changes angle around the sampling center along the corresponding direction.

[0079] The corrected radius and polar angle are converted back to the input feature map coordinates, forming orientation-scale transformation coordinates. These orientation-scale transformation coordinates describe the orientation and scale changes of the corresponding reference sampling point at the current input feature map position. The sampling point transformation module then reads the local deformation parameters corresponding to the reference sampling point identifier, superimposing the lateral components of the local deformation parameters onto the abscissa of the orientation-scale transformation coordinates, and superimposing the ordinate components onto the ordinate of the orientation-scale transformation coordinates. Because the local deformation parameters are superimposed after the orientation-scale transformation, their effect is limited to a single reference sampling point, without changing the orientation and distance already formed by other reference sampling points.

[0080] In one implementation, the sampling point transformation module denotes the coordinates of the reference sampling point relative to the sampling center as r_k, the scale parameter as s_k, the rotation parameter as θ_k, and the local deformation parameter as δ_k. After the reference sampling point coordinates r_k are converted into radius ρ_k and polar angle φ_k, the orientation-scale transformation coordinates are obtained by converting the corrected radius ρ'_k and the corrected polar angle φ'_k, where ρ'_k is calculated by ρ_k and s_k, and φ'_k is calculated by φ_k and θ_k; the initial adaptive sampling point coordinates are obtained by adding the orientation-scale transformation coordinates to δ_k. This calculation is performed within the input feature map coordinate system without performing an overall rotation on the image to be recognized or the candidate target region.

[0081] As an equivalent Cartesian coordinate representation, the direction-scale transformation coordinate g_k can be expressed as g_k=c+s_kR(θ_k)r_k, and the initial adaptive sampling point p_k^0 can be expressed as p_k^0=g_k+δ_k, where c is the coordinate of the sampling center, R(θ_k) is the two-dimensional rotation matrix constructed from the rotation parameters, and the local deformation displacement u_k=p_k^0−g_k. This representation is used to clarify that the constraint object of S300 is u_k, rather than the absolute coordinates of p_k^0.

[0082] After calculating the coordinates, the sampling point transformation module checks whether the initial adaptive sampling point falls within the accessible range of the input feature map. If the initial adaptive sampling point exceeds the boundary of the input feature map, in one implementation, the coordinates of the sampling point are truncated to the boundary of the input feature map; in another implementation, the original coordinates are retained and the boundary state is configured so that S400 ignores feature positions exceeding the input feature map when aggregating regional features. For cases where multiple sampling points overlap even after scaling or local deformation parameters are applied, this step does not directly delete the overlapping sampling points, but retains the identifiers and coordinates of each sampling point, and S300 performs coordinate correction based on the adjacency relationship of the sampling nodes.

[0083] The transformed coordinates corresponding to each output spatial location are arranged according to the reference sampling point identifier, forming an initial adaptive sampling point subset for that output spatial location; the initial adaptive sampling point subsets corresponding to all output spatial locations in the input feature map constitute the initial adaptive sampling point set. The initial adaptive sampling point set retains the orientation-scale transformation coordinates, local deformation parameters, scale parameters, and rotation parameters corresponding to each sampling point. Among them, the difference between the orientation-scale transformation coordinates and the initial adaptive sampling point coordinates represents the local deformation displacement, which is treated as the object of spatial consistency constraint processing in S300.

[0084] S300. The difference between each initial adaptive sampling point in the initial adaptive sampling point set and the corresponding direction-scale transformation coordinate is determined as the local deformation displacement. The difference between adjacent local deformation displacements is calculated based on the adjacency relationship of the sampling nodes, and the local deformation displacement is constrained using the graph Laplacian operator. The constrained local deformation displacement is superimposed with the corresponding direction-scale transformation coordinate to generate a constrained sampling point set.

[0085] After receiving the initial adaptive sampling point set formed by S200, the spatial consistency constraint module processes multiple initial adaptive sampling points corresponding to the same reference sampling grid according to their output spatial positions. This module reads the adjacency matrix established by S100 and finds the adjacent initial adaptive sampling points of the current sampling point based on the connection elements in the adjacency matrix. For two sampling points that are not connected, the relative displacement difference between them is not calculated; for sampling points that are adjacent laterally, longitudinally, or diagonally, they participate in coordinate constraints according to the connection weights in the adjacency matrix.

[0086] In one implementation, the spatial consistency constraint module does not directly smooth the absolute coordinates of the initial adaptive sampling points. Instead, it first calculates the local deformation displacement of each initial adaptive sampling point relative to the corresponding orientation-scale transformation coordinates. Directly smoothing the absolute coordinates would simultaneously change the orientation-scale structure already formed by the scale and rotation parameters. Therefore, this implementation applies graph constraints to the local deformation displacement. For adjacent sampling points i and j, the relative displacement difference between them is obtained by subtracting the local deformation displacement of sampling point j from the local deformation displacement of sampling point i; the relative displacement difference represents the difference in change between adjacent sampling points during the local deformation stage.

[0087] The spatial consistency constraint module constructs a degree matrix based on the adjacency matrix, and the difference between the degree matrix and the adjacency matrix forms the graph Laplacian operator. The diagonal elements of the degree matrix are the sum of the connection weights of the corresponding sampled nodes, while the off-diagonal elements are zero. After the graph Laplacian operator is applied to the local deformation displacement vectors arranged according to the reference sampled point identifiers, the local displacement change of each sampled node relative to its neighboring nodes is obtained. Sampled points with many connected nodes are constrained by multiple neighboring nodes, while sampled points located at the edge of the reference sampled grid are constrained by their actual connected neighboring nodes.

[0088] In one implementation, the spatial consistency constraint module uses the graphical Laplacian operator to obtain the constrained local deformation displacement vector. Let the local deformation displacement vector be denoted as U, the graphical Laplacian operator as L, and the constraint weight as λ. The constrained local deformation displacement vector U' is then solved by the equation (I + λL)U' = U, where I represents the identity matrix. The spatial consistency constraint module solves the equation according to the number of sampling points in the reference sampling grid, and adds each constrained local deformation displacement in U' to the corresponding direction-scale transformation coordinates to obtain the coordinates of the constrained sampling points.

[0089] In another implementation, the spatial consistency constraint module uses an iterative approach to obtain the constrained local deformation displacement vector. Each iteration calculates the correction direction based on the product of the current local deformation displacement vector and the graph Laplacian operator, and then updates the local deformation displacement according to a preset step size. The update stops when the displacement change between two adjacent iterations is less than a preset convergence threshold, or when the number of iterations reaches a preset number. When using the iterative approach, the preset step size is matched to the maximum connection weight of the adjacency matrix to prevent a single coordinate update from spanning multiple input feature map positions.

[0090] After coordinate correction, the spatial consistency constraint module checks the distance between constraint sampling points corresponding to each adjacent edge. If the length of an adjacent edge is less than a preset lower limit, it indicates that the corresponding sampling node has local clustering; if the length of an adjacent edge is greater than a preset upper limit, it indicates that the corresponding sampling node has excessive separation relative to the reference sampling grid. In one implementation, for the above situations, the constrained local deformation displacement of the abnormal sampling point is combined with the unconstrained local deformation displacement according to a preset ratio; if the combined adjacent edge length still does not meet the range configuration, the abnormal sampling point is reverted to the corresponding direction-scale transformation coordinates. The reversion only applies to the abnormal sampling point; the coordinates of the remaining sampling points continue to be calculated using the graph Laplacian operator.

[0091] For constraint sampling points located near the boundary of the input feature map, the spatial consistency constraint module retains their actual coordinates and boundary states. If a constraint sampling point is outside the input feature map and exceeds a preset boundary range, the constraint sampling point is moved back to the boundary position of the input feature map; if the distance exceeds the preset boundary range, the coordinates are retained, and positions outside the input feature map are excluded by S400 during region weight normalization. This process ensures that the relative position constraints of the sampling nodes use the same coordinate boundaries as subsequent region feature aggregation.

[0092] The constraint sampling points corresponding to each output spatial location are arranged according to the reference sampling point identifier, forming a constraint sampling point set. This constraint sampling point set also retains the scale and rotation parameters associated with each constraint sampling point. S400 reads the coordinates of the constraint sampling points as the center of the region aggregation kernel, reads the rotation parameters as the direction input of the region aggregation kernel, and reads the scale parameters as the range input of the region aggregation kernel. Therefore, the coordinate correction result generated by S300 directly changes the feature positions that subsequently participate in weighted aggregation in the input feature map.

[0093] S400. Using each constraint sampling point in the set of constraint sampling points as the center of the elliptical Gaussian kernel, the rotation parameters and scale parameters associated with each constraint sampling point are reused to determine the direction and range of the corresponding elliptical Gaussian kernel; the neighborhood features in the input feature map are weighted and aggregated according to the elliptical Gaussian kernel to generate a deformation adaptation feature map, and a target recognition result is generated based on the deformation adaptation feature map.

[0094] After receiving the set of constraint sampling points formed by S300, the region aggregation kernel generation module creates a region aggregation kernel for each constraint sampling point. The center of the region aggregation kernel is taken as the coordinates of the constraint sampling point in the input feature map coordinate system, the direction of the region aggregation kernel is taken as the rotation parameter associated with the constraint sampling point, and the range of the region aggregation kernel is calculated by the scale parameter associated with the constraint sampling point. Since S200 and S400 use the same rotation parameter and scale parameter, the directional change of the sampling point relative to the sampling center corresponds to the directional change of the region aggregation kernel. The region aggregation kernel will not remain fixed along the horizontal or vertical direction of the input feature map after the sampling point has been rotated.

[0095] In one embodiment, the region aggregation kernel is an elliptic Gaussian kernel. The region aggregation kernel generation module uses the constrained sampling points as the center of the elliptic Gaussian kernel, constructs a two-dimensional rotation matrix based on rotation parameters, and calculates the major and minor axes of the elliptic Gaussian kernel based on scale parameters. Specifically, a base major axis and a base minor axis are pre-configured, and the scale parameters are multiplied by the base major axis and the base minor axis respectively to obtain the major and minor axes of the current elliptic Gaussian kernel; when the base major axis and the base minor axis take different values, the elliptic Gaussian kernel has different coverage areas in two orthogonal directions. In another embodiment, the scale parameters include a radial scale component and an axial ratio scale component. The radial scale component controls the overall range of the elliptic Gaussian kernel, and the axial ratio scale component controls the ratio between the major and minor axes.

[0096] The region aggregation kernel generation module constructs a covariance matrix based on the major axis, minor axis, and rotation matrix. The covariance matrix is ​​first calculated in an unrotated coordinate system using the major and minor axis variances as diagonal elements, and then transformed to the input feature map coordinate system using the rotation matrix. If either the major or minor axis is less than a configured lower limit, the corresponding axis length is set to the configured lower limit. If the calculated covariance matrix is ​​non-positive definite, a preset stabilizing value is added to the diagonal elements of the covariance matrix before recalculation. The corrected covariance matrix, together with the coordinates of the constrained sampling points, constitutes an elliptical Gaussian kernel.

[0097] In one implementation, the major and minor axes are denoted as a_k and b_k, respectively, and the rotation matrix is ​​denoted as R(θ_k). Then, the covariance matrix Σ_k = R(θ_k)diag(a_k^2, b_k^2)R(θ_k)^T. For a candidate feature position q, its unnormalized spatial weight is exp[−0.5(q−p_k)^TΣ_k^−1(q−p_k)], where p_k is the center of the elliptical Gaussian kernel. The region feature aggregation module sums and normalizes the weights of the effective candidate feature positions.

[0098] The region feature aggregation module selects multiple feature locations in the input feature map based on the coverage of the elliptical Gaussian kernel. Specifically, it first constructs a bounding rectangle containing the elliptical Gaussian kernel based on the major and minor axes, and then reads the grid positions of the input feature map within the bounding rectangle. For each grid position, it calculates the coordinate difference between the grid position and the constrained sampling points, and calculates the Mahalanobis distance based on the coordinate difference and the covariance matrix. When the Mahalanobis distance is not greater than a preset cutoff distance, the grid position is retained as a feature position for aggregation; when the Mahalanobis distance is greater than the preset cutoff distance, the feature at that grid position is not read.

[0099] For each retained feature location, the region feature aggregation module calculates spatial weights based on the Mahalanobis distance. Feature locations closer to the center of the elliptical Gaussian kernel and located along the major axis have corresponding weights, while feature locations farther from the center of the elliptical Gaussian kernel have smaller weights. The region feature aggregation module calculates the sum of all effective spatial weights within the current elliptical Gaussian kernel and normalizes each spatial weight using this sum. If the elliptical Gaussian kernel crosses the boundary of the input feature map, locations outside the input feature map are not included in the weight summation, and the spatial weights within the boundary are re-normalized; if the sum of effective spatial weights is less than a preset value, the major and minor axes are adjusted to the basic axis lengths, and new feature locations are selected.

[0100] The regional feature aggregation module reads the feature values ​​of each feature location in the input feature map by channel, multiplies the feature values ​​by the corresponding normalized spatial weights, and sums the weighted feature values ​​within the coverage area of ​​the same elliptical Gaussian kernel to obtain the local aggregated feature corresponding to the constrained sampling point. This local aggregated feature maintains the channel dimension of the input feature map. For multiple constrained sampling points corresponding to the same output spatial location, the regional feature aggregation module restores the arrangement order of each local aggregated feature according to the reference sampling point identifier. In one implementation, each local aggregated feature is multiplied by the convolution kernel weights of the corresponding reference sampling point in the feature extraction operator, and summed along the reference sampling point dimension to obtain the output feature vector for that output spatial location.

[0101] Each output spatial location in the input feature map is used to generate an output feature vector in the manner described above. The region feature aggregation module then arranges these output feature vectors in row and column order according to their output spatial locations to form a deformation-adaptive feature map. The spatial dimensions of the deformation-adaptive feature map are determined by the size of the input feature map, the step size of the feature extraction operator, and the boundary configuration. The number of its channels is determined by the output channel configuration of the region feature aggregation module. Each output spatial location maintains its mapping relationship with the corresponding receptive region in the original input feature map, thus enabling the target recognition module to convert the predicted location to the coordinates of the image to be recognized.

[0102] After receiving the deformation adaptation feature map, the target recognition module performs target category prediction, target location prediction, and target keypoint prediction respectively. In one embodiment, the target category prediction branch outputs the category score for each target category based on the deformation adaptation feature map; the target location prediction branch outputs the target center coordinates, target width, and target height; and the target keypoint prediction branch outputs the coordinates of each target keypoint relative to the target location and the keypoint confidence score. All prediction branches use the same output spatial location identifier, enabling the category, location, and keypoint results generated at the same output spatial location to be correlated.

[0103] The target recognition module associates the target category, target location, target key points, and recognition confidence score with the output spatial location and target candidate identifier. When multiple target candidates correspond to the same image region, the module calculates the overlap between target locations and retains one candidate based on the recognition confidence score. Target candidates with a recognition confidence score below a preset output threshold are not included in the target recognition result. The target location and target key points, after being transformed from input feature map coordinates to the coordinates of the image to be recognized, are combined with the target category and recognition confidence score to form the target recognition result.

[0104] The target recognition result is associated with the image identifier of the current image to be recognized and output to the robot vision controller, image display device, or subsequent target processing module. When the target recognition result is called by the robot vision controller, the target position is used to determine the spatial region of the target in the image, the target key points are used to determine the position of the target's local structure, and the target category and recognition confidence are used to distinguish different target candidates. After the current frame is processed, the visual target deformation feature recognition system releases the initial adaptive sampling point set and constraint sampling point set corresponding to the current frame; when the next frame of the image to be recognized arrives, the geometric sampling configuration result corresponding to the input feature map of the next frame is regenerated from S100.

[0105] Example 2: Figure 2 A structural block diagram of a visual target deformation feature recognition system according to an embodiment of the present invention is shown. Figure 2 As shown, the structure may include:

[0106] The feature map acquisition module 01 is used to acquire the input feature map corresponding to the image to be recognized, and to determine the set of reference sampling points and the adjacency relationship of sampling nodes according to the reference sampling grid. The feature map acquisition module is connected to the image acquisition device or the upstream image processing unit. After receiving the image to be recognized, it inputs the image to the front-end feature extraction network and reads the input feature map with spatial and channel dimensions from the preset feature layer. The preset feature layer is specified by the system configuration item to ensure that the spatial position in the input feature map corresponds to the receptive region in the image to be recognized. When the front-end feature extraction network does not output a valid feature map, or when the input feature map contains non-numerical elements, the feature map acquisition module stops the subsequent sampling processing corresponding to the current image, retains the image identifier of the current image to be recognized, and sends the input feature map to the subsequent modules after re-completing feature extraction.

[0107] The reference sampling grid corresponds to the sampling specifications of the feature extraction operator, and the reference sampling points are represented by coordinates relative to the sampling center. The feature map acquisition module assigns reference sampling point identifiers to each reference sampling point according to their arrangement in the reference sampling grid, and establishes adjacency relationships between sampling nodes based on horizontal, vertical, and diagonal adjacency. These adjacency relationships are represented by an adjacency matrix, where rows and columns correspond to the reference sampling point identifiers. The input feature map and the set of reference sampling points are sent to the geometric parameter generation module, and the adjacency relationships between sampling nodes are sent to the spatial consistency constraint module, remaining unchanged within the processing cycle of the same input feature map.

[0108] The geometric parameter generation module 02, connected to the feature map acquisition module, is used to generate scale parameters, rotation parameters, and local deformation parameters associated with each reference sampling point based on the input feature map, and to form a geometric sampling configuration result according to the input feature map, the set of reference sampling points, the adjacency relationship of the sampling nodes, the scale parameters, the rotation parameters, and the local deformation parameters. The geometric parameter generation module receives the input feature map and the set of reference sampling points from the feature map acquisition module, performs a convolution operation on the input feature map, and forms a parameter tensor corresponding to each spatial position of the input feature map. The parameter tensor is divided into multiple parameter channel groups according to the reference sampling point identifier. Each parameter channel group contains the scale parameter, rotation parameter, and local deformation parameter associated with the corresponding reference sampling point. The scale parameter represents the distance adjustment of the reference sampling point relative to the sampling center, the rotation parameter represents the direction adjustment of the reference sampling point relative to the sampling center, and the local deformation parameter includes the horizontal and vertical components in the input feature map coordinate system.

[0109] The geometric parameter generation module reads the parameter channels in the parameter tensor according to the reference sampling point identifier and establishes an association between the same parameter channel group and the corresponding reference sampling point. For scale parameters exceeding the preset scale range, scale limiting is performed; for rotation parameters exceeding the preset angle range, angle limiting is performed; for parameter channel groups containing non-numerical elements, the scale parameter is set to a unit scale, the rotation parameter is set to zero, and the lateral and longitudinal components of the local deformation parameter are set to zero. The processed input feature map, the reference sampling point set, the adjacency relationship of sampling nodes, and the three types of parameters corresponding to each reference sampling point together form the geometric sampling configuration result. The geometric sampling configuration result is sent to the sampling point transformation module, and the scale and rotation parameters are also sent to the region aggregation kernel generation module for subsequent calculation of the direction and range of the region aggregation kernel.

[0110] The sampling point transformation module 03, connected to the geometric parameter generation module, is used to adjust the distance of the reference sampling point relative to the sampling center according to the scale parameter and the rotation parameter, based on the geometric sampling configuration result, and to superimpose the local deformation parameter to generate an initial adaptive sampling point set. After receiving the geometric sampling configuration result, the sampling point transformation module reads the sampling center, the reference sampling point set, and the corresponding three types of parameters position by position according to the output spatial position in the input feature map. For the current reference sampling point, the horizontal coordinate difference and vertical coordinate difference between the reference sampling point and the sampling center are first calculated, and then the horizontal coordinate difference and the vertical coordinate difference are converted into radius and polar angle. The scale parameter acts on the radius, and the rotation parameter acts on the polar angle. The corrected radius and polar angle are then converted into input feature map coordinates to form direction-scale transformed coordinates.

[0111] After completing the direction and distance adjustments, the sampling point transformation module superimposes the lateral components of the local deformation parameters onto the abscissa of the orientation-scale transformation coordinates, and superimposes the ordinate components of the local deformation parameters onto the ordinate of the orientation-scale transformation coordinates. The superimposed coordinates are rearranged according to the reference sampling point identifier, forming an initial adaptive sampling point subset corresponding to the current output spatial position; all initial adaptive sampling point subsets corresponding to the output spatial positions together form the initial adaptive sampling point set. If an initial adaptive sampling point exceeds the boundary of the input feature map, the sampling point transformation module retains the corresponding boundary state; if the distance exceeds a preset boundary range, the initial adaptive sampling point is truncated to the boundary of the input feature map. The initial adaptive sampling point set, along with the reference sampling point identifier and the orientation-scale transformation coordinates, is sent to the spatial consistency constraint module.

[0112] The spatial consistency constraint module 04 connects the sampling point transformation module and the feature map acquisition module. It is used to determine the difference between each initial adaptive sampling point in the initial adaptive sampling point set and the corresponding direction-scale transformation coordinate as the local deformation displacement, constrain the local deformation displacement using the graph Laplacian operator based on the adjacency relationship of the sampling nodes, and superimpose the constrained local deformation displacement with the corresponding direction-scale transformation coordinate to generate a constrained sampling point set.

[0113] The spatial consistency constraint module constructs a graph Laplacian operator based on the adjacency matrix and uses this operator to constrain each local deformation displacement, obtaining the constrained local deformation displacement. Each constrained local deformation displacement is then superimposed with its corresponding direction-scale transformation coordinates to generate a set of constrained sampling points. This processing only applies to local deformation displacements and does not directly change the direction-scale transformation coordinates formed by the scale and rotation parameters. If the distance between adjacent constrained sampling points is less than a preset lower limit or greater than a preset upper limit, the corresponding constrained sampling point is identified as an anomalous sampling point, and the anomalous sampling point is backtracked to its corresponding direction-scale transformation coordinates. If the backtracking still exceeds the boundary of the input feature map, coordinate truncation is performed.

[0114] The region aggregation kernel generation module 05, connected to the spatial consistency constraint module and the geometric parameter generation module, is used to generate a set of region aggregation kernels. It uses constraint sampling points in the constraint sampling point set as the center of the region aggregation kernel, determines the direction of the region aggregation kernel based on the rotation parameters associated with the constraint sampling points, and determines the range of the region aggregation kernel based on the scale parameters associated with the constraint sampling points. The region aggregation kernel generation module matches the constraint sampling points with the scale parameters and rotation parameters output by the geometric parameter generation module according to the reference sampling point identifier. Each constraint sampling point corresponds to one region aggregation kernel. The coordinates of the constraint sampling point serve as the center of the region aggregation kernel, the rotation parameters serve as the main direction input of the region aggregation kernel, and the scale parameters serve as the range input of the region aggregation kernel.

[0115] In one implementation, an elliptic Gaussian kernel is used as the region aggregation kernel. The region aggregation kernel generation module determines the major axis direction of the elliptic Gaussian kernel based on rotation parameters, and determines the major and minor axes based on scale parameters and preset base axis lengths. Then, a covariance matrix is ​​constructed based on the major axis direction, major axis, and minor axis. If the major or minor axis is less than a preset lower limit, the corresponding axis length is replaced with the preset lower limit. If the covariance matrix does not meet the positive definite condition, a preset stabilizing factor is added to the diagonal elements of the covariance matrix before reconstructing the region aggregation kernel. The region aggregation kernels corresponding to all constrained sampling points are arranged according to their output spatial location and reference sampling point identifiers to form a region aggregation kernel set, which is then sent to the region feature aggregation module.

[0116] The region feature aggregation module 06, connected to the region aggregation kernel generation module and the feature map acquisition module, is used to perform weighted aggregation of neighborhood features in the input feature map based on the region aggregation kernel set, generating a deformation-adaptive feature map. The region feature aggregation module receives the region aggregation kernel set from the region aggregation kernel generation module and reads the input feature map from the feature map acquisition module. For the current region aggregation kernel, candidate neighborhoods in the input feature map are first determined based on the center and range of the region aggregation kernel. Then, the spatial weights corresponding to each candidate feature position are calculated based on the direction of the region aggregation kernel and the coordinate differences between each candidate feature position and the center of the region aggregation kernel. Feature positions located outside the coverage area of ​​the region aggregation kernel do not participate in the current local aggregation.

[0117] The regional feature aggregation module sums the spatial weights within the coverage area of ​​the regional aggregation kernel, normalizes each spatial weight based on the sum, and then multiplies and sums the feature values ​​of multiple feature locations with their corresponding spatial weights to form local aggregated features corresponding to the constrained sampling points. When the regional aggregation kernel crosses the boundary of the input feature map, feature locations outside the boundary are excluded, and the spatial weights within the boundary are renormalized. When the sum of effective spatial weights is lower than a preset value, the current regional aggregation kernel is reconstructed using the basic axis length. Multiple local aggregated features are arranged according to the reference sampling point identifiers and form a deformation-adaptive feature map according to the output spatial position. The deformation-adaptive feature map is sent to the target recognition module, and its spatial position maintains a mapping relationship with the corresponding receptive region in the input feature map.

[0118] The target recognition module 07, connected to the region feature aggregation module, is used to predict target category, target location, and target key points based on the deformation adaptation feature map, and associates the target category, target location, target key points, and recognition confidence to generate a target recognition result. After receiving the deformation adaptation feature map, the target recognition module calls the target category prediction branch, target location prediction branch, and target key point prediction branch according to the output spatial position in the deformation adaptation feature map. The target category prediction branch outputs the category score corresponding to each target category, the target location prediction branch outputs the target center coordinates and target size, and the target key point prediction branch outputs the target key point coordinates and key point confidence.

[0119] The target recognition module associates the target category, target location, target key points, and recognition confidence score based on the output spatial location and target candidate identifier. When multiple target candidates correspond to the same image region, the module filters the candidates based on the degree of overlap between target locations and the recognition confidence score. For target candidates with a recognition confidence score lower than a preset output threshold, the output of that candidate is stopped. The filtered target locations and key points are converted into coordinates of the image to be recognized based on the coordinate mapping relationship between the input feature map and the image to be recognized, and together with the target category and recognition confidence score, constitute the target recognition result. The target recognition result is output to the upstream vision control device or subsequent target processing unit. After the current image to be recognized is processed, the next image to be recognized triggers the feature map acquisition module to execute a new recognition cycle.

Claims

1. A method for recognizing visual target deformation features, characterized in that, Includes the following steps: S100. Obtain the input feature map of the image to be recognized, and determine the set of reference sampling points and the adjacency relationship of sampling nodes according to the reference sampling grid; Based on the input feature map, scale parameters, rotation parameters, and local deformation parameters associated with each reference sampling point are generated; S200. Adjust the distance of the reference sampling point relative to the sampling center according to the scale parameter, and adjust its direction according to the rotation parameter to obtain the direction-scale transformation coordinates; superimpose the local deformation parameter onto the corresponding direction-scale transformation coordinates to generate an initial adaptive sampling point set. S300. The difference between each initial adaptive sampling point in the initial adaptive sampling point set and the corresponding direction-scale transformation coordinate is determined as the local deformation displacement. The difference between adjacent local deformation displacements is calculated based on the adjacency relationship of the sampling nodes, and the local deformation displacement is constrained using the graph Laplacian operator. The constrained local deformation displacement is superimposed with the corresponding direction-scale transformation coordinates to generate a set of constrained sampling points; S400. Using each constraint sampling point in the constraint sampling point set as the center of the elliptical Gaussian kernel, the rotation parameters and scale parameters associated with each constraint sampling point are reused to determine the direction and range of the corresponding elliptical Gaussian kernel; the neighborhood features in the input feature map are weighted and aggregated according to the elliptical Gaussian kernel to generate a deformation adaptation feature map, and a target recognition result is generated based on the deformation adaptation feature map.

2. The method according to claim 1, characterized in that, The generation of the scale parameters, rotation parameters, and local deformation parameters includes: The input feature map is convolved to generate a parameter tensor that is divided into parameter channel groups according to the reference sampling point identifier; The scale parameters, rotation parameters, and local deformation parameters of the corresponding reference sampling points are extracted from each parameter channel group. Positive bounded mapping is performed on the scale parameters, angular bounded mapping is performed on the rotation parameters, and displacement limiting is performed on the local deformation parameters.

3. The method according to claim 1, characterized in that, The step of adjusting the distance of the reference sampling point relative to the sampling center according to the scale parameter, and adjusting the orientation of the reference sampling point relative to the sampling center according to the rotation parameter, includes: Convert the coordinates of the reference sampling point relative to the sampling center into radius and polar angle; The radius is corrected according to the scale parameter, and the polar angle is corrected according to the rotation parameter; The corrected radius and polar angle are converted into input feature map coordinates.

4. The method according to claim 3, characterized in that, The determination of the initial adaptive sampling point and local deformation displacement includes: The lateral and longitudinal components of the local deformation parameters are superimposed onto the orientation-scale transformation coordinates to obtain initial adaptive sampling points; the difference between the initial adaptive sampling points and the corresponding orientation-scale transformation coordinates is determined as the local deformation displacement, and they are arranged according to the reference sampling point identifiers.

5. The method according to claim 1, characterized in that, The determination of the adjacency relationship of the sampling nodes includes: An adjacency matrix is ​​established based on the horizontal, vertical, and diagonal adjacency relationships of the reference sampling points in the reference sampling grid; The adjacent initial adaptive sampling points in the initial adaptive sampling point set are determined based on the adjacency matrix.

6. The method according to claim 5, characterized in that, The method of constraining the local deformation displacement using the graph Laplacian operator includes: A graph Laplacian operator L is constructed based on the adjacency matrix, and each local deformation displacement is arranged into a local deformation displacement vector U according to the reference sampling point identifier; The constrained local deformation displacement vector U' is obtained by using (I + λL)U' = U, where I is the identity matrix and λ is the constraint weight. Add each constrained local deformation displacement in the constrained local deformation displacement vector U' to the corresponding direction-scale transformation coordinate to generate a set of constrained sampling points; The distance between adjacent constraint sampling points in the constraint sampling point set is detected; when the distance exceeds a preset range, the adjacent constraint sampling point corresponding to the distance is identified as an abnormal sampling point, and each abnormal sampling point is backed up to its corresponding direction-scale transformation coordinate.

7. The method according to claim 1, characterized in that, The determination of the elliptic Gaussian kernel includes: The constrained sampling point is taken as the center of the elliptical Gaussian kernel; The major axis direction is determined based on the same rotation parameter involved in generating the orientation-scale transformation coordinates, and the major and minor axes are determined based on the same scale parameter involved in generating the orientation-scale transformation coordinates. Construct a covariance matrix based on the major axis direction, major axis, and minor axis.

8. The method according to claim 7, characterized in that, The weighted aggregation includes: The Mahalanobis distance between the candidate feature positions and the center of the elliptical Gaussian kernel is calculated based on the covariance matrix of the elliptical Gaussian kernel, and the feature positions whose Mahalanobis distance is not greater than a preset cutoff distance are retained. The spatial weights of each preserved feature location are calculated based on the Mahalanobis distance, and the spatial weights are then normalized. Multiply the features at each retained feature location with their corresponding spatial weights and sum them to generate local aggregated features; The local aggregated features are arranged according to the reference sampling point identifier and the output spatial position to generate the deformation adaptation feature map.

9. The method according to claim 1, characterized in that, The generation of target recognition results based on the deformation adaptation feature map includes: The deformation adaptation feature map is input into the target recognition module to perform target category prediction, target location prediction, and target key point prediction, respectively. The target category, target location, target key points, and recognition confidence are correlated to generate the target recognition result.

10. A visual target deformation feature recognition system, used to perform the method according to any one of claims 1 to 9, characterized in that, include: The feature map acquisition module is used to acquire the input feature map, the set of reference sampling points, and the adjacency relationship of the sampling nodes; The geometric parameter generation module is used to generate scale parameters, rotation parameters, and local deformation parameters associated with each reference sampling point; The sampling point transformation module is used to generate orientation-scale transformation coordinates based on the scale parameters and rotation parameters, and to superimpose the local deformation parameters onto the corresponding orientation-scale transformation coordinates to generate an initial adaptive sampling point set. The spatial consistency constraint module is used to determine the difference between each initial adaptive sampling point in the initial adaptive sampling point set and the corresponding direction-scale transformation coordinate as the local deformation displacement, constrain the local deformation displacement using the graph Laplacian operator based on the adjacency relationship of the sampling nodes, and superimpose the constrained local deformation displacement with the corresponding direction-scale transformation coordinate to generate a constrained sampling point set. The region aggregation kernel generation module is used to determine the direction and range of the corresponding elliptical Gaussian kernel by reusing the rotation parameters and scale parameters associated with each constraint sampling point in the set of constraint sampling points as the center of the elliptical Gaussian kernel. The region feature aggregation module is used to perform weighted aggregation of neighborhood features in the input feature map based on the elliptical Gaussian kernel to generate a deformation adaptation feature map. The target recognition module is used to generate target recognition results based on the deformation adaptation feature map.

Images