Feature representation analysis method and device for self-supervised image, equipment and medium

Abstract

The invention relates to the technical field of image detection, can be applied to business system platforms of financial science and technology, medical health and the like, and discloses a feature representation analysis method, device and equipment of a self-supervised image and a medium, and the method comprises the steps: extracting basic features of a target image, and recognizing a spatial position corresponding to the basic features; generating a measurement tensor corresponding to each spatial position, and adjusting a local geometric perception range of the convolution kernel through the measurement tensor; determining a geometric propagation distance threshold value of the target image according to the local geometric perception range, and determining geometric consistent features of feature vectors in the basic features through the geometric propagation distance threshold value; performing geometric regularization on the metric tensor to obtain geometric regularization loss, inputting geometric consistent features into a self-supervision task head, and outputting target feature representation; and optimizing the target feature representation according to the task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image. And the accuracy of image feature representation is improved.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a method, apparatus, device, and medium for feature representation analysis of self-supervised images. Background Technology

[0002] Self-supervised image representation learning has become an important research direction in the field of image detection in recent years. Its core goal is to learn discriminative and robust feature representations from unlabeled data. Therefore, in order to improve the accuracy of image feature representation, it is necessary to optimize deep learning for geometric perception.

[0003] In the field of healthcare, medical diagnosis relies on subtle differences, but self-supervised learning often focuses on global structures, such as organ outlines, and may ignore these high-value subtle features, resulting in insufficient feature representation for diagnosis and thus low accuracy in the feature representation of medical images.

[0004] In the fintech business, images often contain noise, such as blurry surveillance footage or wrinkled bills. Self-supervised tasks, such as predicting the order of image blocks, may learn noise-related features, such as the texture of wrinkles, rather than key semantic features. This leads to the failure of feature representation in noisy scenarios, making it difficult to capture the specific features of image samples, resulting in low accuracy of feature representation for financial image data.

[0005] Traditional methods typically rely on large-scale labeled datasets, while self-supervised learning significantly reduces labeling costs by designing agent tasks to mine supervisory signals from the data itself. However, when dealing with spatial transformations such as rotation, translation, and scaling, existing methods often rely on explicit data augmentation to force the model to learn invariance. This not only increases computational overhead but may also cause the feature representation to lose geometric structural information, resulting in lower accuracy of image feature representation. Summary of the Invention

[0006] This invention provides a self-supervised image feature representation analysis method, apparatus, device, and medium to solve the technical problem of low accuracy in image feature representation.

[0007] Firstly, a self-supervised image feature representation analysis method is provided, including: Acquire a target image, extract the basic features of the target image using a trained isovariant convolutional layer, and identify the spatial location corresponding to the basic features; A lightweight convolutional neural network module is used to generate a metric tensor corresponding to each spatial location, and the local geometric perception range of the convolution kernel in the isovariant convolutional layer is adjusted by the metric tensor. The geometric propagation distance threshold of the target image is determined based on the local geometric perception range, and the geometric consistency features of the feature vectors in the basic features are determined through the geometric propagation distance threshold. The metric tensor is geometrically regularized to obtain a geometric regularization loss. The geometrically consistent features are then input into a preset self-supervised task head, and the target feature representation is output. The target feature representation is optimized based on the preset task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image.

[0008] Secondly, a self-supervised image feature representation analysis apparatus is provided, comprising: A spatial location recognition module is used to acquire a target image, extract basic features of the target image using a trained isovariant convolutional layer, and identify the spatial location corresponding to the basic features. The local geometric perception range determination module is used to generate a metric tensor corresponding to each spatial location using a lightweight convolutional neural network module, and to adjust the local geometric perception range of the convolution kernel in the equivariant convolutional layer through the metric tensor. The geometric consistency feature determination module is used to determine the geometric propagation distance threshold of the target image based on the local geometric perception range, and to determine the geometric consistency features of the feature vector in the basic features through the geometric propagation distance threshold; The target feature representation output module is used to perform geometric regularization on the metric tensor to obtain the geometric regularization loss, input the geometrically consistent features into a preset self-supervised task head, and output the target feature representation. The feature representation optimization module is used to optimize the target feature representation based on the preset task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image.

[0009] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described self-supervised image feature representation analysis method.

[0010] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described self-supervised image feature representation analysis method.

[0011] In the aforementioned scheme implemented by the self-supervised image feature representation analysis method, apparatus, device, and medium, a target image can be acquired through a client. Basic features of the target image are extracted using a trained isovariant convolutional layer, and the spatial locations corresponding to these basic features are identified. A lightweight convolutional neural network module generates a metric tensor corresponding to each spatial location. The local geometric perception range of the convolutional kernel in the isovariant convolutional layer is adjusted using the metric tensor. A geometric propagation distance threshold for the target image is determined based on the local geometric perception range. The geometric consistency features of the feature vectors in the basic features are determined using the geometric propagation distance threshold. Geometric regularization is then applied to the metric tensor. The geometric regularization loss is obtained, and the geometrically consistent features are input into a preset self-supervised task head to output the target feature representation. The target feature representation is optimized according to the preset task loss and geometric regularization loss to obtain the feature representation of the self-supervised image. The feature representation is fed back to the client. In this invention, the transform group structure is directly encoded by normalized isovariant convolution, making the feature mapping covariant with the input transformation. The response mode of the convolution kernel is adaptively adjusted by learning the local Riemann metric tensor. Stepped approximate parallel transmission is used to meet the needs of large-scale training while maintaining geometric consistency, thereby solving the technical problem of low accuracy of image feature representation. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of an application environment for a self-supervised image feature representation analysis method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a self-supervised image feature representation analysis method according to an embodiment of the present invention; Figure 3 yes Figure 2 A flowchart illustrating a specific implementation method of step S3; Figure 4 yes Figure 2 A flowchart illustrating a specific implementation of step S4; Figure 5 This is a schematic diagram of a feature representation analysis device for self-supervised images according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 7 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] The self-supervised image feature representation analysis method provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can acquire the target image from the client, extract the basic features of the target image using trained isovariant convolutional layers, and identify the spatial locations corresponding to the basic features. A lightweight convolutional neural network module is used to generate a metric tensor corresponding to each spatial location. The metric tensor is used to adjust the local geometric perception range of the convolutional kernels in the isovariant convolutional layers. A geometric propagation distance threshold for the target image is determined based on the local geometric perception range, and the geometric consistency features of the feature vectors in the basic features are determined based on the geometric propagation distance threshold. Geometric regularization is applied to the metric tensor to obtain a geometric regularization loss. The geometric consistency features are input into a preset self-supervised task head, and the target feature representation is output. The target feature representation is optimized based on the preset task loss and geometric regularization loss to obtain the feature representation of the self-supervised image. The feature representation is fed back to the client. In this invention, the transform group structure is directly encoded by normalized isovariant convolution, making the feature mapping covariant with the input transform. The response mode of the convolutional kernel is adaptively adjusted by learning the local Riemann metric tensor. Stepped approximate parallel transmission is used to meet the needs of large-scale training while maintaining geometric consistency, thereby solving the technical problem of low accuracy in image feature representation. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.

[0016] Please see Figure 2 As shown, Figure 2 A flowchart illustrating the self-supervised image feature representation analysis method provided in this embodiment of the invention includes the following steps: S1. Obtain the target image, extract the basic features of the target image using a trained isovariant convolutional layer, and identify the spatial location corresponding to the basic features.

[0017] In this embodiment of the invention, the target image refers to the original input image that the encoder needs to process. It is a two-dimensional pixel array containing original visual information of the features to be extracted, such as medical images, natural scene images, and remote sensing images.

[0018] In detail, target images can be obtained from pre-stored storage areas using computer statements with data-grabbing capabilities (such as Java statements, Python statements, etc.), where the storage areas include, but are not limited to, databases and blockchains.

[0019] For example, in a medical setting, the target image can be a medical image, a three-dimensional image of the brain, or a brain MRI image that includes the location of the tumor and surrounding nerves and blood vessels. In addition, in fintech scenarios, target images are images of the actions and expressions of both parties, used to identify abnormal transaction behavior; target images are images related to the seals, signatures, and amount numbers on the bills, used to detect forgery and alteration traces.

[0020] Furthermore, the kernel parameters of traditional convolutional neural networks are fixed and sensitive to geometric transformations of images, such as rotation, translation, and slight deformation. The features extracted by traditional convolution may differ significantly from those before rotation, making it difficult for the model to recognize them. Therefore, it is necessary to parameterize the convolution kernel based on the SE (2) group (two-dimensional Euclidean motion group, which includes translation and rotation parameters) based on normalized equivariant convolution. When the input image undergoes geometric transformations such as rotation and translation, the basic features of the output will change synchronously according to the corresponding rules.

[0021] In this embodiment of the invention, the basic features refer to the semantic features extracted from the target image through the equivariant convolutional layer. They are an abstraction of the original pixel information, that is, the process of the image propagating forward in the network. Generally, shallow networks extract basic information, such as color and texture, while deep networks extract semantic information. In this embodiment of the invention, the extraction of basic features of the target image using a trained isovariant convolutional layer includes: Identify the two-dimensional image space corresponding to the target image, and extract the mapping features of the target image in the two-dimensional image space; Determine the spatial location of the mapped feature in the two-dimensional image space, and identify the geometric transformation corresponding to the spatial location; The original position of the geometric transformation corresponding to the spatial position is determined by using a preset inverse transformation, and the target features corresponding to the original position are extracted. Obtain the rotation and translation components of the kernel function stored in the kernel parameter register, and analyze the basic features corresponding to the spatial position based on the target features, the rotation components, and the translation components.

[0022] In detail, defined in two-dimensional image space Feature mapping on Its isovariant convolution operation is ,in: For SE(2) group elements (including translation) and rotation parameter), To create a learnable kernel function, sampling is discretized in the swarm parameter space. Represents coordinates Apply the inverse transformation for each possible geometric transformation. (Translation + Rotation), first... pass Inverse transform to find the corresponding original position Extract the target features at that location. Then with kernel function (Corresponding transformation) Multiply the weights together and sum them up to get the result. Output characteristics at the location.

[0023] Specifically, this operation ensures that when the input image is transformed When the output features satisfy Strict equivariance, that is, for the convolution result Apply transformation The output features are ,and First, process the input. Apply transformation get , and then with Convolution, result Instead, all groups are discretized directly on the SE (2) group. (Translation + Rotation) will cause the kernel function to... With too many parameters, such as 360 discrete values ​​for rotation angle and multiple step sizes for translation, the parameter scale explodes. Therefore, in practical implementation, a fast approximation method is used, expanding the kernel function through a harmonic basis: ,in Using radial basis functions significantly reduces the number of parameters; that is, the kernel function is decomposed into rotational and translational components. The rotational component... It is a complex exponential function (utilizing the periodicity of rotation, It is the order of the Fourier series, describing the kernel function with respect to rotation. Dependency, translation component These are radial basis functions (such as Gaussian functions), describing the kernel function's effect on translation. Dependence, These are learnable coefficients that control the weights of each component, reducing the kernel parameter from being proportional to the number of group elements to O(M), thus significantly reducing computational complexity.

[0024] Furthermore, the spatial location corresponding to the basic feature is identified. That is, the basic feature and its spatial part directly reflect the geometric structure of the image. Only after identifying the spatial location can we further analyze the geometric information such as the spatial distance and orientation relationship between features. The spatial location is the position index on the feature map. For example, the spatial location corresponding to each feature point in the feature map output after the convolutional layer. For example, if the target image is a 224×224 cat photo, the isovariant convolutional layer extracts the basic feature of the cat's ear edge. Its corresponding spatial location may be the region from (50, 30) to (80, 60) in the original image, or the region from (12, 7) to (20, 15) on the feature map.

[0025] Furthermore, while canonical equivariant convolution can guarantee equivariance under global transformations, it cannot adapt to local geometric differences in images. The role of the Riemann metric prediction network is to generate local geometric metric standards for each spatial location, so that the propagation of features at different locations must follow the metric rules of that location.

[0026] S2. A metric tensor corresponding to each spatial location is generated using a lightweight convolutional neural network module, and the local geometric perception range of the convolution kernel in the equivariant convolutional layer is adjusted using the metric tensor.

[0027] In this embodiment of the invention, the metric tensor is a parameter that describes the local spatial geometric characteristics of an image, that is, a corresponding positive definite matrix (SPD matrix). Through the eigenvalues ​​and eigenvectors of the matrix, the metric tensor can reflect the scale and dominant direction of the local region.

[0028] In this embodiment of the invention, generating the metric tensor corresponding to each spatial location using a lightweight convolutional neural network module includes: The basic features are extracted by the convolutional layer of the lightweight convolutional neural network module, and the spatial features corresponding to each spatial location are decomposed to obtain the lower triangular matrix. The lower triangular matrix is ​​transposed to obtain the lower triangular transpose matrix, and a symmetric positive definite matrix is ​​determined based on the lower triangular matrix and the lower triangular transpose matrix. Error analysis is performed on the symmetric positive definite matrix to obtain the metric tensor corresponding to each spatial location.

[0029] In detail, in location metric tensor at point Predict using lightweight networks: ,in: It is a lower triangular matrix, obtained by Cholesky decomposition of the output of a 3×3 convolution. To ensure positive definiteness, the convolutional layers of lightweight networks further extract local spatial characteristics from the basic features of each spatial location, such as the gradient direction of surrounding pixels, texture distribution, and neighborhood correlation, i.e., spatial features. These spatial features are then decomposed, essentially mapping the features to a lower triangular matrix using Cholesky decomposition (a matrix factorization algorithm). The characteristic of a lower triangular matrix is ​​that the elements above the main diagonal are 0, and the elements below the main diagonal are non-zero. Therefore, any symmetric positive definite matrix (SPD) can be uniquely decomposed into the product of a lower triangular matrix and its transpose.

[0030] Specifically, for lower triangular matrices Transpose the matrix to obtain the transpose matrix. , lower triangular matrix AND matrix Multiply to get ,because Since it is a lower triangular matrix, its product must be a symmetric positive definite matrix, satisfying the positive definiteness property, and based on... Positive definiteness error analysis is performed on the symmetric positive definite matrix to obtain the metric tensor corresponding to each spatial location.

[0031] Furthermore, under the assumption that the geometry of all regions in an image is uniform under a fixed perception range, but in reality, there are structural differences in a large number of geometric transformations in an image. By measuring tensors, local geometric specificity can be accurately described. By adjusting the perception range, the range can be expanded in complex geometric regions to capture details, and the range can be reduced in simple regions to reduce redundancy.

[0032] In this embodiment of the invention, the local geometric perception range refers to the range of which spatial points around the point need to be examined in order to accurately extract the features of that point. The range formed by these examined spatial points is the local geometric perception range.

[0033] In this embodiment of the invention, adjusting the local geometric perception range of the convolution kernel in the isovariant convolutional layer using the metric tensor includes: Identify the first spatial point position and the second spatial point position corresponding to the spatial location; Calculate the position difference between the first spatial point and the second spatial point; Determine the local metric tensor between the first spatial point position and the second spatial point position based on the metric tensor; Calculate the metric spatial distance between the first spatial point position and the second spatial point position based on the position difference and the local metric tensor; The local sensing range of the convolution kernel in the equivariant convolutional layer is determined based on the metric spatial distance.

[0034] In detail, the local geometric perception range of the tensor-adjusted convolution kernel is defined as the distance in the metric space as: This allows the convolution kernel to adapt to the stretching / compression of local image structures (such as anisotropy at blood vessel branches).

[0035] Specifically, the location of the first spatial point is The location of the second spatial point is ,but The position difference between the first spatial point and the second spatial point is determined, and the transpose of the position difference is determined. The local metric tensors of the first and second spatial points are also determined. This allows for the calculation of the metric spatial distance between the positions of the first and second spatial points. And then measure the tensor By adjusting the distance calculation method, the convolution kernel can focus on pixels of different ranges in different regions, such as in texture-dense regions (such as blood vessel branches). It is an anisotropic matrix (off-diagonal elements are not zero), the distance along the texture direction is compressed, and the perpendicular direction is stretched, making the model pay more attention to texture-related local features, i.e., at blood vessel branches. This allows the kernel function to focus more on adjacent pixels along the blood vessel direction and ignore irrelevant pixels in the vertical direction, thus achieving geometrically adaptive feature extraction.

[0036] For example, in a medical setting, in brain MRI, the geometric structure of the tumor region differs significantly from that of normal tissue (gray matter, white matter, ventricles). The spatial distance is calculated and measured, and the convolution kernel is adjusted. Specifically, the perception range of the convolution kernel in the tumor edge region is reduced to capture boundary details, while the perception range in the white matter region is expanded to capture the overall structure.

[0037] Furthermore, in fintech scenarios, stock trading heatmaps (horizontal axis for time, vertical axis for price, and color for trading volume) show significant differences in geometric structure between periods of high volatility (such as limit up / limit down) and stable periods. By calculating and measuring spatial distance and adjusting the convolution kernel, the perception range is reduced during periods of high volatility to capture minute-level price jumps, while the range is expanded during stable periods to capture hourly trends.

[0038] Furthermore, canonical isovariant convolution guarantees the isovariance of features under a single transformation, but the propagation of features at different spatial locations (such as from position x to y) still needs to consider local geometric differences. For example, different parts of the same object may appear at different scales in the image due to perspective.

[0039] S3. Determine the geometric propagation distance threshold of the target image based on the local geometric perception range, and determine the geometric consistency feature of the feature vector in the basic feature through the geometric propagation distance threshold.

[0040] In this embodiment of the invention, the geometric propagation distance threshold is a rule or indicator used to quantify and standardize the propagation of geometric features in an image between different spatial locations.

[0041] In this embodiment of the invention, reference is made to Figure 3 As shown, determining the geometric propagation distance threshold of the target image based on the local geometric perception range includes: S31. Determine the target range pixels of the target image based on the local geometric perception range; S32. Identify the target spatial location corresponding to the pixels in the target range; S33. Calculate the target gradient of the metric tensor corresponding to the target spatial location through the intermediate layer gradient of the preset metric analysis network; S34. Determine the geometric propagation distance threshold of the target image based on the target gradient.

[0042] In detail, the target range pixels refer to all pixels that fall within the local geometric perception range, that is, the set of pixels that are geometrically related to the current spatial location being processed. The focus is on pixels that have a geometric impact on the current location, excluding irrelevant areas, reducing redundant calculations, and retaining the key pixel information required for geometric propagation. Thus, each pixel corresponds to a specific spatial coordinate in the image, binding the pixel to a specific spatial location and providing a spatial coordinate reference for subsequent analysis of changes in geometric features in space.

[0043] Specifically, the metric analysis network is a sub-network used to analyze the relationship between the metric tensor and geometric features. The intermediate layer gradient refers to the partial derivative of the output of the intermediate layer with respect to the input, that is, the gradient of the metric tensor at the target spatial location. The target gradient refers to the rate of change of the metric tensor at the target spatial location, reflecting the trend of the metric tensor changing with spatial location. The metric tensor itself describes the local geometric structure, such as the measurement method of distance and angle, and its gradient reveals the evolution law of this geometric structure in space. The target gradient reflects the direction and intensity of the spatial change of the geometric structure. For example, the region with a large gradient may be a place where the geometric structure changes abruptly, such as the edge of an object; the region with a small gradient may be a place where the geometric structure is gentle, such as the interior of an object. Based on this change law, a geometric propagation distance threshold is determined to determine how geometric features propagate from one location to another in the image.

[0044] Furthermore, the local metrics generated by the Riemann metric prediction network must be compatible with the SE (2) group transform on which the equivariant convolution is based. Otherwise, there may be a contradiction where the local geometric adaptation violates the global transformation rules. For example, after rotating the image, the change in local metrics does not match the rotation operation. In this case, the metric tensor needs to be geometrically regularized so that the global consistency of the metric tensor is forced through the loss function.

[0045] In this embodiment of the invention, the geometrically consistent feature refers to the feature vector that, in the basic features, remains consistent in essential geometric attributes, such as direction, structural correlation, and topological relationship, even though it exhibits apparent differences due to image geometric deformation, such as scaling, rotation, and distortion, during spatial transmission, such as transmission along object edges or curved surfaces, after geometric correction.

[0046] In this embodiment of the invention, determining the geometrically consistent features of the feature vectors in the basic features through the geometric propagation distance threshold includes: Identify the transmission path of the feature vector in the basic features, and determine the transmission start point and transmission end point based on the transmission path; Calculate the displacement vector between the transmission endpoint position and the transmission start position; The target starting point gradient of the transmission starting point position is determined based on the geometric propagation distance threshold. The parallel transmission operator of the transmission path is calculated based on the target starting point gradient and the displacement vector, and the geometric consistency features of the feature vector are determined using the parallel transmission operator.

[0047] In detail, eigenvectors arrive The transmission is approximated by the Schild ladder: ,in Obtained by measuring the intermediate gradients of the prediction network. From Transmitted to Parallel transmission operator; this linear approximation reduces the computational complexity of traditional ODE solving from Down to .

[0048] Specifically, feature vector from Transmitted to The transmission path is Transmitted to , For the starting position of transmission, To determine the transmission endpoint location, the displacement vector between the transmission endpoint location and the transmission start location is calculated. and the transpose data of the displacement vector. And because Local changes ( ), The direction will deviate, and the second term in the formula is the correction for this deviation, through... Given the gradient and displacement vectors, estimate the magnitude of the deviation and correct it. To ensure that the transmitted vector is in The local geometry still conforms to the rules.

[0049] For example, in medical scenarios, the basic features of MRI (such as the texture and density feature vector of the tumor) need to maintain consistency during spatial transmission (such as the texture change from the center of the tumor to the edge should be continuous) to avoid feature distortion due to position changes. Parallel transmission operators are calculated using the gradient of the target starting point and the displacement vector to ensure that the feature vector is not distorted during transmission and to obtain geometrically consistent features, such as the continuous feature where the texture gradually becomes sparser from the center of the tumor to the ventricle.

[0050] Furthermore, in fintech scenarios, transaction feature vectors, such as the volume-price ratio, need to maintain consistency over time. For example, the feature of a sharp drop in the morning session should be logically consistent with the feature of a continued drop in the afternoon session. Parallel transmission operators are calculated using the target starting point gradient and displacement vector to obtain geometrically consistent features. For instance, the volume-price ratio feature of a sharp drop in the morning session should maintain trend consistency in the afternoon session.

[0051] Furthermore, due to the Riemannian metric tensor It needs to adapt to the local geometry, but its changes must be coordinated with the SE (2) group transformation on which the canonical equivariant convolution is based. Otherwise, if the changes of the metric tensor under group action (such as rotation and translation) are chaotic, it will destroy the overall geometric consistency. Therefore, geometric regularization constrains the metric tensor to maintain approximately isometry under group action through the loss function, that is, the covariance relationship between the transformed metric and the original metric under group transformation is consistent, ensuring that the local adaptability of the Riemann metric does not violate the global group transformation rules.

[0052] S4. Perform geometric regularization on the metric tensor to obtain the geometric regularization loss, input the geometrically consistent features into the preset self-supervised task head, and output the target feature representation.

[0053] In this embodiment of the invention, the geometric regularization loss constrains the Riemann metric field through a loss function. The rationality under geometric transformation ensures that the metric itself conforms to geometric laws, that is, the transformed metric is consistent with the original metric after transformation and adaptation.

[0054] In this embodiment of the invention, reference is made to Figure 4 As shown, the geometric regularization of the metric tensor to obtain the geometric regularization loss includes: S41. Identify the geometric transformation corresponding to the metric tensor and extract the Jacobian matrix corresponding to the geometric transformation; S42. Determine the constraint logic of the metric tensor based on the Jacobian matrix; S43. Generate a geometric transformation loss function based on the constraint logic and the metric tensor; S44. Calculate the geometric regularization loss of the metric tensor using the geometric transformation loss function.

[0055] In detail, the geometric transformation loss function corresponding to the transformation behavior of the constraint metric tensor under group action is: ,in This is the total number of pixels in the image; the average is calculated to avoid the influence of image size. For transformation Jacobian matrix, It is the Frobenius norm, and log (•) is the matrix logarithm, which maps the SPD matrix to a symmetric matrix, making it convenient to measure the difference. This loss is forced. To ensure that the metric remains covariant under group action, the constraint logic is as follows: That is, the position after transformation The measurement should be approximately equal to the original position. The result of the metric after Jacobian matrix adaptation is used to calculate the geometric regularization loss of the metric tensor based on the geometric transformation loss function.

[0056] Specifically, if a transformation is applied to the image If rotation occurs, local metrics should also be transformed. For example, the metrics at a blood vessel branch after rotation should be consistent with the metrics before rotation after adaptation using a rotation matrix to avoid metric distortion after transformation. This is achieved through penalty... and The deviation ensures that the metric field itself is compatible with geometric transformations, providing a reliable geometric reference for feature propagation.

[0057] Furthermore, the features obtained after correction by the parallel transmission operator, although they may exhibit apparent differences in spatial transmission due to image geometric deformations such as rotation, scaling, and distortion, are essentially of the same geometric structure and maintain consistency in geometric properties.

[0058] In this embodiment of the invention, the self-supervised task head is a specially designed sub-network module. Its function is to force the input geometrically consistent features to learn richer semantic information, such as the overall structure of the object and the relationship between local and global elements, through a proxy task that does not require manual annotation. The final output is a discriminative feature. The structure of the task head and the proxy task are pre-designed. The pre-designed tasks include rotation prediction, jigsaw puzzle reconstruction, feature reconstruction, and contrastive learning. They are composed of a small number of fully connected layers, convolutional layers, or Transformer layers. The input is geometrically consistent features (high-dimensional feature vectors or feature maps), and the output is the prediction result of the proxy task (such as the classification probability of the rotation angle and the arrangement probability of the jigsaw puzzle order).

[0059] In detail, target features refer to the feature representations obtained after processing by the self-supervised task head. They not only inherit the geometric robustness of geometrically consistent features, but also learn stronger semantic discriminative power through the self-supervised task, enabling them to distinguish between different objects and structures. The self-supervised task head uses the loss function of the proxy task to filter out geometrically consistent features that are useful for the task, such as consistency features related to the structure of the object, suppress irrelevant information, and improve the discriminative power of the features.

[0060] Furthermore, by constraining semantic discriminative power through task loss and geometrical regularization loss to constrain geometric robustness, the feature representation of the final self-supervised image achieves a unity of geometric stability and semantic clarity.

[0061] S5. Optimize the target feature representation based on the preset task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image.

[0062] In this embodiment of the invention, the feature representation of a self-supervised image refers to the high-dimensional vector / tensor obtained after the target image is processed by a neural network. It is an abstract encoding of the original data, which can distinguish different categories of information and is not sensitive to geometric deformation of the image.

[0063] In detail, the task loss is a loss function bound to the self-supervised task head, used to measure the error of the target feature representation in completing the self-supervised proxy task. For example, if the self-supervised task is rotation prediction, the task loss can be cross-entropy loss; if the task is jigsaw puzzle reconstruction, the task loss can be ranking loss. The task loss ensures that the feature learns semantic information related to the proxy task, ensuring that the feature has the discriminative power to complete the task. The geometric consistency loss function is used to prevent the feature from losing geometric robustness during the optimization process, forcing the feature to remember geometric attributes during optimization, ensuring that the improvement of semantic discriminative power does not come at the expense of geometric robustness. The feature obtained after joint optimization of the two losses represents a feature that has both semantic discriminative power and geometric robustness, learned solely through self-supervision without the need for manually labeled data.

[0064] Specifically, a total loss function is constructed by weighting the task loss and geometric regularization loss. Then, the network parameters, including lightweight networks, isovariant convolutional layers, and self-supervised task heads, are updated through backpropagation. Ultimately, the target feature representation satisfies both constraints simultaneously. Based on the error calculated between the output of the self-supervised task head and the surrogate task label, the total loss is calculated as: Total Loss = Task Loss Weight × Task Loss + Geometric Regularization Weight × Geometric Regularization Loss. The total loss is then propagated back through backpropagation to all network parameters, including lightweight networks, convolutional layers, and task heads, adjusting the parameters to minimize the total loss. After multiple rounds of iterative optimization (loss convergence), the target feature representation is corrected to simultaneously satisfy both constraints, efficiently completing the self-supervised surrogate task and maintaining geometric consistency. The features at this point are the feature representation of the self-supervised image.

[0065] As can be seen, in the above scheme, the feature mapping is made covariant with the input transformation by directly encoding the transform group structure through normalized equivariant convolution; the response mode of the convolution kernel is adaptively adjusted by learning the local Riemannian metric tensor; and the step-approximate parallel transmission is used to meet the needs of large-scale training while maintaining geometric consistency, thereby solving the technical problem of low accuracy of image feature representation.

[0066] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0067] In one embodiment, a feature representation analysis apparatus for self-supervised images is provided, which corresponds one-to-one with the feature representation analysis method for self-supervised images described in the above embodiments. For example... Figure 5 As shown, the self-supervised image feature representation analysis device includes a spatial location recognition module 101, a local geometric perception range determination module 102, a geometrically consistent feature determination module 103, a target feature representation output module 104, and a feature representation optimization module 105. Detailed descriptions of each functional module are as follows: The spatial location recognition module 101 is used to acquire a target image, extract basic features of the target image using a trained isovariant convolutional layer, and identify the spatial location corresponding to the basic features. The local geometric perception range determination module 102 is used to generate a metric tensor corresponding to each spatial location using a lightweight convolutional neural network module, and adjust the local geometric perception range of the convolution kernel in the equivariant convolutional layer through the metric tensor. The geometric consistency feature determination module 103 is used to determine the geometric propagation distance threshold of the target image based on the local geometric perception range, and to determine the geometric consistency features of the feature vector in the basic features through the geometric propagation distance threshold; The target feature representation output module 104 is used to perform geometric regularization on the metric tensor to obtain geometric regularization loss, input the geometrically consistent features into a preset self-supervised task head, and output the target feature representation. The feature representation optimization module 105 is used to optimize the target feature representation according to the preset task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image.

[0068] In one embodiment, the spatial location recognition module 101, when performing the extraction of basic features of the target image using a trained isovariant convolutional layer, is used to: Identify the two-dimensional image space corresponding to the target image, and extract the mapping features of the target image in the two-dimensional image space; Determine the spatial location of the mapped feature in the two-dimensional image space, and identify the geometric transformation corresponding to the spatial location; The original position of the geometric transformation corresponding to the spatial position is determined by using a preset inverse transformation, and the target features corresponding to the original position are extracted. Obtain the rotation and translation components of the kernel function stored in the kernel parameter register, and analyze the basic features corresponding to the spatial position based on the target features, the rotation components, and the translation components.

[0069] In one embodiment, the local geometry perception range determination module 102, when executing the generation of the metric tensor corresponding to each spatial location using the lightweight convolutional neural network module, is used to: The basic features are extracted by the convolutional layer of the lightweight convolutional neural network module, and the spatial features corresponding to each spatial location are decomposed to obtain the lower triangular matrix. The lower triangular matrix is ​​transposed to obtain the lower triangular transpose matrix, and a symmetric positive definite matrix is ​​determined based on the lower triangular matrix and the lower triangular transpose matrix. Error analysis is performed on the symmetric positive definite matrix to obtain the metric tensor corresponding to each spatial location.

[0070] In one embodiment, the local geometric perception range determination module 102, when performing the adjustment of the local geometric perception range of the convolution kernel in the isovariant convolutional layer through the metric tensor, is further configured to: Identify the first spatial point position and the second spatial point position corresponding to the spatial location; Calculate the position difference between the first spatial point and the second spatial point; Determine the local metric tensor between the first spatial point position and the second spatial point position based on the metric tensor; Calculate the metric spatial distance between the first spatial point position and the second spatial point position based on the position difference and the local metric tensor; The local sensing range of the convolution kernel in the equivariant convolutional layer is determined based on the metric spatial distance.

[0071] In one embodiment, the geometric consistency feature determination module 103, when performing the determination of the geometric propagation distance threshold of the target image based on the local geometric perception range, is configured to: The target range pixels of the target image are determined based on the local geometric perception range; Identify the target spatial location corresponding to the pixels within the target range; The target gradient of the metric tensor corresponding to the target spatial location is calculated by using the intermediate layer gradient of the preset metric analysis network. The geometric propagation distance threshold of the target image is determined based on the target gradient.

[0072] In one embodiment, the geometric consistency feature determination module 103, when performing the determination of the geometric consistency features of the feature vectors in the basic features through the geometric propagation distance threshold, is further configured to: Identify the transmission path of the feature vector in the basic features, and determine the transmission start point and transmission end point based on the transmission path; Calculate the displacement vector between the transmission endpoint position and the transmission start position; The target starting point gradient of the transmission starting point position is determined based on the geometric propagation distance threshold. The parallel transmission operator of the transmission path is calculated based on the target starting point gradient and the displacement vector, and the geometric consistency features of the feature vector are determined using the parallel transmission operator.

[0073] In one embodiment, the target feature representation output module 104, when performing geometric regularization on the metric tensor to obtain the geometric regularization loss, is used to: Identify the geometric transformation corresponding to the metric tensor and extract the Jacobian matrix corresponding to the geometric transformation; The constraint logic for the metric tensor is determined based on the Jacobian matrix. Generate a geometric transformation loss function based on the constraint logic and the metric tensor; The geometric regularization loss of the metric tensor is calculated using the geometric transformation loss function.

[0074] This invention provides a self-supervised image feature representation analysis device. It directly encodes the transform group structure through normalized equivariant convolution, making the feature mapping covariant with the input transformation. By learning the local Riemannian metric tensor, it adaptively adjusts the response mode of the convolution kernel. By utilizing staircase approximate parallel transmission, it meets the requirements of large-scale training while maintaining geometric consistency, thereby solving the technical problem of low accuracy of image feature representation.

[0075] Specific limitations regarding the feature representation analysis device for self-supervised images can be found in the limitations of the feature representation analysis method for self-supervised images described above, and will not be repeated here. Each module in the aforementioned feature representation analysis device for self-supervised images can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0076] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a self-supervised image feature representation analysis method on the server side.

[0077] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of a self-supervised image feature representation analysis method.

[0078] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Acquire a target image, extract the basic features of the target image using a trained isovariant convolutional layer, and identify the spatial location corresponding to the basic features; A lightweight convolutional neural network module is used to generate a metric tensor corresponding to each spatial location, and the local geometric perception range of the convolution kernel in the isovariant convolutional layer is adjusted by the metric tensor. The geometric propagation distance threshold of the target image is determined based on the local geometric perception range, and the geometric consistency features of the feature vectors in the basic features are determined through the geometric propagation distance threshold. The metric tensor is geometrically regularized to obtain a geometric regularization loss. The geometrically consistent features are then input into a preset self-supervised task head, and the target feature representation is output. The target feature representation is optimized based on the preset task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image.

[0079] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Acquire a target image, extract the basic features of the target image using a trained isovariant convolutional layer, and identify the spatial location corresponding to the basic features; A lightweight convolutional neural network module is used to generate a metric tensor corresponding to each spatial location, and the local geometric perception range of the convolution kernel in the isovariant convolutional layer is adjusted by the metric tensor. The geometric propagation distance threshold of the target image is determined based on the local geometric perception range, and the geometric consistency features of the feature vectors in the basic features are determined through the geometric propagation distance threshold. The metric tensor is geometrically regularized to obtain a geometric regularization loss. The geometrically consistent features are then input into a preset self-supervised task head, and the target feature representation is output. The target feature representation is optimized based on the preset task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image.

[0080] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0081] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0083] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.

[0084] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A self-supervised image feature representation analysis method, characterized in that, include: Acquire a target image, extract the basic features of the target image using a trained isovariant convolutional layer, and identify the spatial location corresponding to the basic features; A lightweight convolutional neural network module is used to generate a metric tensor corresponding to each spatial location, and the local geometric perception range of the convolution kernel in the isovariant convolutional layer is adjusted by the metric tensor. The geometric propagation distance threshold of the target image is determined based on the local geometric perception range, and the geometric consistency features of the feature vectors in the basic features are determined through the geometric propagation distance threshold. The metric tensor is geometrically regularized to obtain a geometric regularization loss. The geometrically consistent features are then input into a preset self-supervised task head, and the target feature representation is output. The target feature representation is optimized based on the preset task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image.

2. The self-supervised image feature representation analysis method as described in claim 1, characterized in that, The basic features of the target image are extracted using trained isovariant convolutional layers, including: Identify the two-dimensional image space corresponding to the target image, and extract the mapping features of the target image in the two-dimensional image space; Determine the spatial location of the mapped feature in the two-dimensional image space, and identify the geometric transformation corresponding to the spatial location; The original position of the geometric transformation corresponding to the spatial position is determined by using a preset inverse transformation, and the target features corresponding to the original position are extracted. Obtain the rotation and translation components of the kernel function stored in the kernel parameter register, and analyze the basic features corresponding to the spatial position based on the target features, the rotation components, and the translation components.

3. The self-supervised image feature representation analysis method as described in claim 1, characterized in that, The step of generating the metric tensor corresponding to each spatial location using a lightweight convolutional neural network module includes: The basic features are extracted by the convolutional layer of the lightweight convolutional neural network module, and the spatial features corresponding to each spatial location are decomposed to obtain the lower triangular matrix. The lower triangular matrix is ​​transposed to obtain the lower triangular transpose matrix, and a symmetric positive definite matrix is ​​determined based on the lower triangular matrix and the lower triangular transpose matrix. Error analysis is performed on the symmetric positive definite matrix to obtain the metric tensor corresponding to each spatial location.

4. The self-supervised image feature representation analysis method as described in claim 1, characterized in that, The method of adjusting the local geometric perception range of the convolution kernel in the isovariant convolutional layer using the metric tensor includes: Identify the first spatial point position and the second spatial point position corresponding to the spatial location; Calculate the position difference between the first spatial point and the second spatial point; Determine the local metric tensor between the first spatial point position and the second spatial point position based on the metric tensor; Calculate the metric spatial distance between the first spatial point position and the second spatial point position based on the position difference and the local metric tensor; The local sensing range of the convolution kernel in the equivariant convolutional layer is determined based on the metric spatial distance.

5. The self-supervised image feature representation analysis method as described in claim 1, characterized in that, Determining the geometric propagation distance threshold of the target image based on the local geometric perception range includes: The target range pixels of the target image are determined based on the local geometric perception range; Identify the target spatial location corresponding to the pixels within the target range; The target gradient of the metric tensor corresponding to the target spatial location is calculated by using the intermediate layer gradient of the preset metric analysis network. The geometric propagation distance threshold of the target image is determined based on the target gradient.

6. The self-supervised image feature representation analysis method as described in claim 1, characterized in that, The step of determining the geometrically consistent features of the feature vectors in the basic features through the geometric propagation distance threshold includes: Identify the transmission path of the feature vector in the basic features, and determine the transmission start point and transmission end point based on the transmission path; Calculate the displacement vector between the transmission endpoint position and the transmission start position; The target starting point gradient of the transmission starting point position is determined based on the geometric propagation distance threshold. The parallel transmission operator of the transmission path is calculated based on the target starting point gradient and the displacement vector, and the geometric consistency features of the feature vector are determined using the parallel transmission operator.

7. The self-supervised image feature representation analysis method as described in claim 1, characterized in that, The geometric regularization of the metric tensor to obtain the geometric regularization loss includes: Identify the geometric transformation corresponding to the metric tensor and extract the Jacobian matrix corresponding to the geometric transformation; The constraint logic for the metric tensor is determined based on the Jacobian matrix. Generate a geometric transformation loss function based on the constraint logic and the metric tensor; The geometric regularization loss of the metric tensor is calculated using the geometric transformation loss function.

8. A feature representation analysis device for self-supervised images, characterized in that, include: A spatial location recognition module is used to acquire a target image, extract basic features of the target image using a trained isovariant convolutional layer, and identify the spatial location corresponding to the basic features. The local geometric perception range determination module is used to generate a metric tensor corresponding to each spatial location using a lightweight convolutional neural network module, and to adjust the local geometric perception range of the convolution kernel in the equivariant convolutional layer through the metric tensor. The geometric consistency feature determination module is used to determine the geometric propagation distance threshold of the target image based on the local geometric perception range, and to determine the geometric consistency features of the feature vector in the basic features through the geometric propagation distance threshold; The target feature representation output module is used to perform geometric regularization on the metric tensor to obtain the geometric regularization loss, input the geometrically consistent features into a preset self-supervised task head, and output the target feature representation. The feature representation optimization module is used to optimize the target feature representation based on the preset task loss and the geometric regularization loss to obtain the feature representation of the self-supervised image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the feature representation analysis method for self-supervised images as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the feature representation analysis method for self-supervised images as described in any one of claims 1 to 7.

Images