Image feature determination method and related equipment

Abstract

The invention provides an image feature determination method and related equipment, and the method comprises the steps: extracting image features of a target image under at least one scale, and obtaining target parameters corresponding to the image features, the target parameters comprising one or more of gradient intensity, local variance and high-frequency energy response parameters; according to the target parameters, the complexity of the image features under each scale is determined, the weight of each scale under the self-attention mechanism is determined according to the complexity of the image features, and the complexity and the weight are in positive correlation; determining the attention energy of the image features under each scale, and determining the weight of each scale under a self-attention mechanism according to the complexity of the image features; and according to the self-attention mechanism, each weight and each attention energy, fusing each image feature to obtain an image fusion feature. According to the invention, the extraction accuracy of the image features is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image feature determination method and related equipment. Background Technology

[0002] With the development of the field of computer vision, the requirements for image processing are becoming increasingly demanding.

[0003] In the exemplary technology, a self-attention mechanism is used to adaptively capture long-distance dependencies in the image feature sequence, thereby exhibiting better feature representation energy in image processing tasks such as semantic segmentation, object detection, and image super-resolution.

[0004] However, the self-attention mechanism uses a uniform attention weight calculation method for the input image features. When multi-scale image feature fusion is performed based on the self-attention mechanism, the uniform attention weighting method causes semantic layer features to become overly dominant, thereby suppressing the detailed features at the lower level. This leads to attention being focused on the salient areas of the image, while neglecting edge areas, small objects, or local texture information. As a result, the fused image features cannot accurately express the texture and semantics of the image, which means there is a problem of inaccurate image feature extraction. Summary of the Invention

[0005] Based on the above-mentioned technological status, this application provides an image feature determination method and related equipment to solve the problem of inaccurate image feature extraction.

[0006] To achieve the above-mentioned technical objectives, this application proposes the following technical solution: In a first aspect, this application provides an image feature determination method, including: Extract image features at at least one scale from the target image and obtain the target parameters corresponding to the image features. The target parameters include one or more of gradient strength, local variance, and high-frequency energy response parameters. Based on the target parameters, the complexity of the image features at each scale is determined, and based on the complexity of the image features, the weight of each scale under the self-attention mechanism is determined, wherein the complexity and the weight are positively correlated. Determine the attention energy of the image features at each scale, and determine the weight of each scale under the self-attention mechanism based on the complexity of the image features; Based on the self-attention mechanism, each of the weights, and each of the attention energies, the image features are fused to obtain image fusion features.

[0007] In some implementations, determining the attention energy of the image features at each scale includes: The attention energy matrix is ​​determined based on the attention energy of the image features at each scale. The attention energy of image features at the specified scale is determined based on a preset scaling factor, the attention energy matrix, the weights, and a preset functional relationship.

[0008] In some implementations, determining the attention energy of the image features at each scale includes: The average energy value of the image feature is determined based on the spatial size and number of channels of the image feature; The average energy value is normalized to obtain the attention energy of the image feature.

[0009] In some implementations, fusing the image features based on the self-attention mechanism, the weights of each scale, and the attention energy of the image features at each scale to obtain image fusion features includes: Each of the image features is upsampled to obtain features to be processed with the same spatial resolution; Based on the weights of each scale under the self-attention mechanism, the attention energy corresponding to each feature to be processed is weighted and balanced to obtain the image fusion feature.

[0010] In some implementations, the target parameter includes gradient intensity, and obtaining the target parameter corresponding to the image feature includes: Channel aggregation is performed on the image features to obtain single-channel feature mapping parameters; Based on the single-channel feature mapping parameters, determine the first gradient of the image feature in the horizontal direction and the second gradient in the vertical direction; The gradient intensity corresponding to the image feature is determined based on the first gradient and the second gradient.

[0011] In some implementations, the target parameter includes local variance, and obtaining the target parameter corresponding to the image feature includes: A sliding window is set for the image features, and the image features are slid based on the sliding window to obtain local features; Based on the feature values ​​of each feature in the local features, the feature mean of the local features is determined, and the target variance is determined based on the feature mean and each feature value. The target variance is normalized to obtain the local variance corresponding to the image feature.

[0012] In some implementations, the target parameters include high-frequency energy response parameters, and obtaining the target parameters corresponding to the image features includes: The image features are subjected to two-dimensional discrete wavelet decomposition to obtain the first high-frequency detail subband parameters in the horizontal direction, the second high-frequency detail subband parameters in the vertical direction, and the third high-frequency detail subband parameters in the diagonal direction. Based on the first high-frequency detail subband parameter, the second high-frequency detail subband parameter, and the second high-frequency detail subband parameter, determine the high-frequency energy response parameter to be processed; The high-frequency energy response parameters to be processed are normalized to obtain the high-frequency energy response parameters corresponding to the image features.

[0013] Secondly, this application provides an image feature determination apparatus, comprising: An extraction module is used to extract image features at at least one scale from a target image and obtain target parameters corresponding to the image features. The target parameters include one or more of gradient strength, local variance, and high-frequency energy response parameters. The first determining module is used to determine the complexity of image features at each scale according to the target parameters, and to determine the weight of each scale under the self-attention mechanism according to the complexity of the image features, wherein the complexity and the weight are positively correlated. The second determining module is used to determine the attention energy of the image features at each scale, and to determine the weight of each scale under the self-attention mechanism based on the complexity of the image features. The fusion module is used to fuse the image features according to the self-attention mechanism, the weights, and the attention energy of the image features at each scale to obtain image fusion features.

[0014] Thirdly, this application provides an electronic device, including a memory and a processor, wherein, The memory is connected to the processor and is used to store programs; The processor is used to implement the image feature determination method as described in the first aspect or any implementation thereof by running a program in the memory.

[0015] Fourthly, this application provides a computer program product, which, when executed by a processor, implements the image feature determination method as described in the first aspect or any implementation thereof.

[0016] Fifthly, this application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image feature determination method as described in the first aspect or any implementation thereof.

[0017] This application provides an image feature determination method and related apparatus. It extracts image features at multiple scales from a target image and obtains target parameters such as gradient intensity, local variance, and high-frequency energy response parameters of the image features. Based on these target parameters, it determines the complexity of the image features at each scale, determines the weight of each scale under a self-attention mechanism based on the complexity of the image features, and determines the attention energy of the image features at each scale. Then, it fuses the various image features using the self-attention mechanism, the weights of each scale, and the attention energy of the image features at each scale to obtain a fused image feature. In this application, the weight of each scale under the self-attention mechanism is determined by determining the complexity of the image features at different scales, and the weight and complexity are positively correlated. That is, a larger complexity results in a larger weight to enhance the attention response of local features, while a smaller complexity results in a smaller weight to suppress unnecessary attention diffusion. This achieves a self-balancing of the attention energy of image features at different scales, enabling the fused image features to accurately express the texture and semantics of the image, thus improving the accuracy of image feature extraction. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0019] Figure 1 A flowchart of an image feature determination method provided in this application embodiment Figure 1 .

[0020] Figure 2 A flowchart of an image feature determination method provided in this application embodiment Figure 2 .

[0021] Figure 3 A flowchart of an image feature determination method provided in this application embodiment Figure 3 .

[0022] Figure 4 A flowchart of an image feature determination method provided in this application embodiment Figure 4 .

[0023] Figure 5 A flowchart of an image feature determination method provided in this application embodiment Figure 5 .

[0024] Figure 6 A flowchart of an image feature determination method provided in this application embodiment Figure 6 .

[0025] Figure 7 A flowchart of an image feature determination method provided in this application embodiment Figure 7 .

[0026] Figure 8 This is a schematic diagram of the structure of an image feature determination device provided in an embodiment of this application.

[0027] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0029] It should be noted that the user information (including but not limited to electrical equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0030] With the development of the field of computer vision, the requirements for image processing are becoming increasingly demanding.

[0031] In the exemplary technology, a self-attention mechanism is used to adaptively capture long-distance dependencies in the image feature sequence, thereby exhibiting better feature representation energy in image processing tasks such as semantic segmentation, object detection, and image super-resolution.

[0032] However, the self-attention mechanism uses a uniform attention weight calculation method for the input image features. When multi-scale image feature fusion is performed based on the self-attention mechanism, the uniform attention weighting method causes semantic layer features to become overly dominant, thereby suppressing the detailed features at the lower level. This leads to attention being focused on the salient areas of the image, while neglecting edge areas, small objects, or local texture information. As a result, the fused image features cannot accurately express the texture and semantics of the image, which means there is a problem of inaccurate image feature extraction.

[0033] In view of this, embodiments of this application aim to provide an image feature determination method and related apparatus. This method extracts image features at multiple scales from a target image and obtains target parameters such as gradient intensity, local variance, and high-frequency energy response parameters of the image features. Based on these target parameters, the complexity of the image features at each scale is determined. The weight of each scale under a self-attention mechanism is determined based on the complexity of the image features, and the attention energy of the image features at each scale is also determined. Finally, the image features are fused using the self-attention mechanism, the weights of each scale, and the attention energy of the image features at each scale to obtain fused image features. In this application, the weight of each scale under the self-attention mechanism is determined by determining the complexity of the image features at different scales. The weight and complexity are positively correlated; that is, a larger weight enhances the attention response of local features when the complexity is higher, and a smaller weight suppresses unnecessary attention diffusion when the complexity is lower. This achieves self-balancing of the attention energy of image features at different scales, enabling the fused image features to accurately express the texture and semantics of the image, thus improving the accuracy of image feature extraction.

[0034] Figure 1 A flowchart of an image feature determination method provided in this application embodiment Figure 1 .like Figure 1 As shown, the image feature determination method provided in this embodiment includes: Step S101: Extract image features at at least one scale from the target image and obtain the target parameters corresponding to the image features. The target parameters include one or more of gradient strength, local variance, and high-frequency energy response parameters.

[0035] In this embodiment, the execution entity is an image feature determination device, which is the device housing the visual model, or the image feature determination device represents the visual model. The visual model can be a SegFormer (Simple and Efficient Design for Semantic Segmentation with Transformers) semantic segmentation network structure, where Transformer refers to a deep learning model based on a self-attention mechanism. SegFormer employs a lightweight Mix Transformer encoder and a simple MLP (Multilayer Perceptron) decoder architecture, balancing accuracy and speed. SegFormer exhibits good robustness in handling multi-scale feature fusion and semantic region boundary recognition. For ease of description, the term "device" will be used below to refer to the image feature determination device.

[0036] The device acquires a target image and extracts image features at at least one scale from the target image. For example, by inputting the target image into the SegFormer backbone network, Transformer features at different scales can be obtained; these Transformer features are defined as image features. For instance, image features at different scales are represented as follows: ; in, High resolution, detail layer; Low resolution, semantic layer; intermediate layer This indicates the characteristics of the transition scale.

[0037] After obtaining image features at different scales, target parameters for each image feature are obtained. The target parameters include at least one of gradient strength, local variance, and high-frequency energy response parameters.

[0038] Step S102: Based on the target parameters, determine the complexity of the image features at each scale, and based on the complexity of the image features, determine the weight of each scale under the self-attention mechanism, where complexity and weight are positively correlated.

[0039] Step S103: Determine the attention energy of image features at each scale, and determine the weight of each scale under the self-attention mechanism based on the complexity of the image features.

[0040] The device includes a feature complexity awareness module. This module evaluates the complexity distribution of image features at different spatial locations and outputs a complexity response map C(x, y). In one example, the complexity response map C(x, y) is calculated by combining the local gradient rate of change, feature variance, and high-frequency energy response. Its core idea is to measure the non-smoothness of features using a non-parametric statistical method, as shown in the following formula:

[0041] Among them, || F|| represents the gradient strength; Var represents the local variance; HF represents the high-frequency energy response after wavelet processing; λ i λ is an adjustable coefficient. i That is, λ1, λ2, and λ3 are obtained from fixed empirical values. For example, λ1, λ2, and λ3 take values ​​of 0.4, 0.3, and 0.3, respectively.

[0042] Furthermore, the complexity of image features can be the average complexity, which is:

[0043] in, Let i be the spatial dimension of image feature i, which is the average complexity of image feature calculated based on the complexity of image feature in different spaces.

[0044] The device then determines the attention energy for image features at each scale. For example, the attention energy is the similarity matrix. ,in, , For query matrix (Query). The input feature map is the feature output of the previous Transformer encoder or convolutional layer. K is the key matrix. For the linear mapping matrix of the query, Let be the linear mapping matrix of the keys. It is learned through training on the network (SegFormer backbone network).

[0045] In another example, the input feature map After initial feature extraction via convolutional layers, the complexity of image features is calculated using a combination of local variance and entropy.

[0046] in, Represented in pixels The variance of local window features centered on; Represents the entropy value of the pixels within the window; is the weighting coefficient, with a value range of [0,1]. Local variance refers to the local feature obtained after applying a sliding window to the image features.

[0047] After determining the image complexity, the weight of each scale under the self-attention mechanism can be determined. For example, complexity and weight are positively correlated; that is, the greater the complexity of the image features, the greater the weight. Using this positive correlation and complexity, the weight of each scale under the self-attention mechanism can be determined. For example, the weights are:

[0048] in, This represents the average complexity of image features at the current scale. The average complexity of image features at other scales or the current scale. This represents the weights at the current scale under the self-attention mechanism.

[0049] Step S104: Based on the self-attention mechanism, the attention energy of each weight and the image features at each scale, the image features are fused to obtain the image fusion features.

[0050] After determining the weights corresponding to each scale, the image features are fused based on the self-attention mechanism, the attention capabilities of each weight, and the image features at each scale, thus obtaining the image fusion feature. In one example, the image fusion feature is:

[0051] in, For image fusion features, For attention energy, For weights.

[0052] In this embodiment, image features at multiple scales are extracted from the target image, and target parameters such as gradient intensity, local variance, and high-frequency energy response parameters of the image features are obtained. Based on the target parameters, the complexity of the image features at each scale is determined. The weight of each scale under the self-attention mechanism is determined by the complexity of the image features, and the attention energy of the image features at each scale is determined. Then, the image features are fused by the self-attention mechanism, the weights of each scale, and the attention energy of the image features at each scale to obtain the image fusion feature. In this application, the weight of each scale under the self-attention mechanism is determined by determining the complexity of the image features at different scales, and the weight and complexity are positively correlated. That is, if the complexity is large, a larger weight is used to enhance the attention response of local features, and if the complexity is small, a smaller weight is used to suppress unnecessary attention diffusion. This achieves self-balancing of the attention energy of image features at different scales, so that the fused image features can accurately express the texture and semantics of the image, improving the accuracy of image feature extraction.

[0053] Figure 2 A flowchart of an image feature determination method provided in this application embodiment Figure 2 ,based on Figure 1 In the embodiment shown, step S104 includes: Step S201: Determine the attention energy matrix based on the attention energy of image features at each scale.

[0054] Step S202: Based on the preset scaling factor, attention energy matrix, image feature complexity, and preset functional relationship, the various image features are fused to obtain image fusion features.

[0055] SegFormer's original self-attention mechanism follows the Transformer structure, and its basic computation process is as follows: Let the input features be... ,in , These represent the spatial dimensions of the feature map, Represents the number of channels. A query, key, and value matrix is ​​obtained through linear mapping: ; in: is a trainable weight matrix; For attention feature dimensions; ,, This indicates the number of spatial positions after flattening.

[0056] Therefore, the calculation formula for the self-attention mechanism in the exemplary technology is as follows: ; in: This represents the similarity between the query and the key (i.e., the attention energy matrix); This is a scaling factor used to prevent values ​​from becoming too large; This is used to normalize the similarity weights, making the sum of all attention weights equal to 1; finally, it is multiplied by... Implement weighted feature aggregation.

[0057] In this embodiment, a dynamic adjustment coefficient generated from the feature complexity graph C(x,y) is introduced. The attention energy is weighted and balanced, that is, the attention calculation formula in this embodiment is: ; in, (Q,K) = f(C), where C is the complexity-aware weight factor; f(·) represents a non-linear mapping function (e.g., Sigmoid, ReLU, Tanh, or a 1×1 convolutional layer); when the complexity of the input feature region is high, the value of C is large, and the result after transformation by f(·) is... The value of C is also relatively large, thus enhancing the attention response to local features; when the feature region is smooth, the value of C is smaller, corresponding to... It is also smaller, thus inhibiting unnecessary diffusion of attention; this mechanism achieves energy self-balancing of attention.

[0058] Furthermore, Adjusted to: ； in, , is the result of a complexity graph C through a nonlinear mapping function (such as Sigmoid or a trainable function). The adjustment coefficients (weights) obtained from convolution. When the feature region is highly complex... Larger areas require improved local attention resolution in the model; when the region is smooth, Smaller size inhibits the spread of ineffective attention, thereby achieving self-balancing regulation of characteristic energy.

[0059] Understandably, the device is based on a preset scaling factor. Attention Energy Matrix Scale weight and preset function relationships The attention energy is used to determine the image features at different scales.

[0060] In this embodiment, a complexity-aware weighting factor is introduced based on the original self-attention mechanism of SegFormer. It is used to dynamically adjust the distribution of attention energy.

[0061] Figure 3 A flowchart of an image feature determination method provided in this application embodiment Figure 3 ,based on Figure 1 In the embodiment shown, step S102 includes: Step S301: Determine the average energy value of the image features based on the spatial size and number of channels of the image features.

[0062] In this embodiment, an average energy is calculated for each image feature at each scale to measure the overall amplitude level of the image feature at that scale. The average energy value is:

[0063] Where H and W are spatial dimensions, and C is the number of channels. The energy in the i-th space is the average energy value of the image features, which is determined by the spatial size of the image features and the number of channels.

[0064] Step S302: Normalize the average energy value to obtain the attention energy of the image features.

[0065] After obtaining the average energy value, normalizing the average energy value yields the attention energy of the image features. The normalization process is performed using the following formula: ; in, To prevent small constants from being divided by zero (such as...) ).

[0066] In this embodiment, the average energy value of the image features is obtained and then normalized to ensure that the energy of image features at different scales is at the same level, thus avoiding the dominance of the energy of high-level image features under the self-attention mechanism.

[0067] Figure 4A flowchart of an image feature determination method provided in this application embodiment Figure 4 .based on Figures 1 to 3 In any of the embodiments shown, step S104 includes: Step S401: Upsample each image feature to obtain features to be processed with the same spatial resolution.

[0068] In this embodiment, when fusing multi-scale image features, it is necessary to upsample the image features at each scale to the same spatial resolution, that is, to obtain each feature to be processed with the same spatial resolution.

[0069] Step S402: Based on the weights of each scale under the self-attention mechanism, the attention energy corresponding to each feature to be processed is weighted and balanced to obtain the image fusion features.

[0070] After determining each feature to be processed, the attention energy corresponding to each feature is weighted and balanced based on the weights of each scale under the self-attention mechanism, thus obtaining the image fusion feature. The attention energy of the image features before sampling is the attention energy corresponding to the features to be processed obtained from the sampling of the image features. The weighted fusion is calculated using the following formula:

[0071] in, Indicates the upsampling function (such as bilinear interpolation or deconvolution); The fused result It preserves both high-level semantics and low-level detailed features; dynamic weights ensure that high-resolution layer features are used more in complex regions and semantic layer information is used more in smooth regions.

[0072] The following examples illustrate multi-scale image features: Taking an image with an input resolution of 512 × 512 as an example: Step 1: The SegFormer encoder outputs 4 layers of features: ; ; ; .

[0073] Step 2: Calculate the energy separately and normalize features . Step 3: Obtain the complexity graph using the FCPM module Calculate the average complexity: , , , ; The dynamic weights are obtained after normalization: , , , . Step 4: Upsampled to 512 × 512, and weighted and summed according to the above weights, the fused features are obtained:

[0074] Final output It preserves the detailed information at the edges of the text and integrates the deep semantic context, achieving multi-scale dynamic balance feature fusion.

[0075] In this embodiment, by upsampling image features of different scales to the same spatial resolution, multi-scale image features are fused based on dynamic weights calculated by complexity. This allows the image fusion features to retain detailed information about the edges of the text while integrating deep semantic context.

[0076] Figure 5 A flowchart of an image feature determination method provided in this application embodiment Figure 5 .based on Figures 1 to 4 In any of the embodiments shown, step S101 includes: Step S501: Channel aggregation is performed on the image features to obtain single-channel feature mapping parameters.

[0077] In this embodiment, the target parameter includes gradient intensity. Gradient intensity reflects the magnitude of local change of the input feature map F(x,y) at spatial location (x,y), and is an indicator of the saliency of edges and details. The feature map F(x,y) is the image feature.

[0078] First, channel aggregation is performed on the input feature map F to obtain the single-channel feature mapping parameters f(x,y):

[0079] Where C is the number of channels for the image features.

[0080] Step S502: Determine the first gradient of the image features in the horizontal direction and the second gradient in the vertical direction based on the single-channel feature mapping parameters.

[0081] After obtaining the single-channel feature mapping parameters, the gradients in the horizontal and vertical directions are determined based on these parameters. The first gradient in the horizontal direction is:

[0082] The second gradient in the horizontal direction: ; Where "*" represents convolution operation, S x , S y For Sobel or Prewitt operators, for example: .

[0083] Step S503: Determine the gradient intensity corresponding to the image feature based on the first gradient and the second gradient.

[0084] After obtaining the first and second gradients, the gradient strength can be calculated as follows: .

[0085] The gradient strength is then normalized so that its range falls within [0, 1], specifically: ; Where ε is a small constant to prevent division by zero (e.g., 10). -8 ).

[0086] In this embodiment, the gradient intensity of the image features is accurately determined by performing channel aggregation on the image features and calculating the gradients in the horizontal and vertical directions.

[0087] Figure 6 A flowchart of an image feature determination method provided in this application embodiment Figure 6 .based on Figures 1 to 5 In any of the embodiments shown, step S101 includes: Step S601: Set a sliding window for the image features, and slide the image features based on the sliding window to obtain local features.

[0088] In this embodiment, the target parameter includes local variance. Local variance measures the degree of variation of features within a local region and reflects texture complexity and non-uniformity within the region. First, select a sliding window W(x,y) for the feature map f(x,y). Based on the sliding window, slide the image features to obtain local features, such as a 3×3 region.

[0089] Step S602: Determine the mean of the local features based on the feature values ​​of each feature in the local features, and determine the target variance based on the mean and each feature value.

[0090] Based on the feature values ​​of each feature in the local features, the feature mean of the local features is determined. The feature mean is:

[0091] Among them, f These are the eigenvalues.

[0092] The device then determines the target variance using eigenvalues ​​and the eigenmean, whereby the target variance is:

[0093] Step S603: Normalize the target variance to obtain the local variance corresponding to the image features.

[0094] After obtaining the target variance, normalizing it yields the local variance. The normalization process is as follows:

[0095] Here, ε represents the numerical stability term. Furthermore, convolution can be used to accelerate the calculation of local variance:

[0096] AvgPool represents window averaging.

[0097] In this embodiment, by setting a sliding window for image features, the local variance can be quickly and accurately determined based on the local features obtained by sliding the window.

[0098] Figure 7 A flowchart of an image feature determination method provided in this application embodiment Figure 7 .based on Figures 1 to 6 In any of the embodiments shown, step S101 includes: Step S701: Perform two-dimensional discrete wavelet decomposition on the image features to obtain the first high-frequency detail subband parameters in the horizontal direction, the second high-frequency detail subband parameters in the vertical direction, and the third high-frequency detail subband parameters in the diagonal direction.

[0099] In this embodiment, the target parameters include high-frequency energy response parameters, which are used to characterize the distribution of details and texture information in an image or feature map.

[0100] The device performs two-dimensional discrete wavelet decomposition on the image features to obtain the first high-frequency detail sub-band parameters in the horizontal direction, the second high-frequency detail sub-band parameters in the vertical direction, and the third high-frequency detail sub-band parameters in the diagonal direction. The specific process is as follows:

[0101] in, This is a low-frequency approximate sub-band. , , These are the high-frequency detail subband parameters for the horizontal, vertical, and diagonal directions, respectively.

[0102] Step S702: Determine the high-frequency energy response parameters to be processed based on the first high-frequency detail subband parameters, the second high-frequency detail subband parameters, and the second high-frequency detail subband parameters.

[0103] The device determines the high-frequency energy response parameters to be processed based on the first high-frequency detail subband parameters, the second high-frequency detail subband parameters, and the third high-frequency detail subband parameters. The specific process is as follows: ; Alternatively, summation after taking the absolute value: .

[0104] Step S703: Normalize the high-frequency energy response parameters to be processed to obtain the high-frequency energy response parameters corresponding to the image features.

[0105] After obtaining the high-frequency energy response parameters to be processed, normalization is performed on these parameters to obtain the high-frequency energy response parameters corresponding to the image features, specifically: .

[0106] In this embodiment, gradient intensity The gradients in the horizontal and vertical directions are calculated using the Sobel operator, and their squares and square roots are obtained. The local variance (Var) is calculated using the mean and variance formulas within a sliding window to reflect the local texture complexity. The high-frequency energy response (HF) is extracted by performing discrete wavelet transform on the feature map to obtain high-frequency subbands. Then, its energy sum is calculated; the three results are normalized and then weighted and fused to obtain the final image feature complexity C(x,y).

[0107] In this application, the SegFormer self-attention dynamic balancing method based on feature complexity awareness is based on the SegFormer network structure. By introducing a feature complexity-aware module (FCAM) ​​in the encoder stage, the computational ratio of local and global attention mechanisms is dynamically balanced. In the decoding stage, a spatial-semantic joint weighting mechanism is introduced, thereby improving the model's feature representation ability in complex scenes and multi-scale text images.

[0108] The overall structure of the device includes the following main modules: Feature complexity evaluation submodule: Calculates the complexity of intermediate features of the input image and outputs a complexity score matrix; Local-Global Dynamic Fusion Module: Adaptively adjusts the fusion ratio of local and global attention based on complexity scores; Spatial-semantic joint weighting module: integrates location information and semantic information to improve the spatial consistency of features; Dynamic Weight Adjustment Module (MLP): Generates weight parameters for each submodule to achieve adaptive adjustment.

[0109] An example of a Feature Complexity Aware Module (FCAM) ​​is as follows: In this embodiment, the input feature map After extracting preliminary features through convolutional layers, the feature complexity is calculated by combining local variance and entropy.

[0110] in: Represented in pixels The variance of local window features centered on; Represents the entropy value of the pixels within the window; This represents the weighting coefficient, with a value range of [0,1]. Complexity score. After normalization, it is used as a dynamic control signal input to the local-global fusion module to balance the attention ratio of different types of features.

[0111] An example of a local-to-global dynamic fusion module is as follows: The input features are divided into N non-overlapping windows, each of size P × P. Local self-attention computation is performed within each window:

[0112] Simultaneously, global attention is performed using global features sampled across windows:

[0113] The fusion of local and global features adopts a dynamic weighting approach:

[0114] Wherein, β(x, y) is obtained by mapping the complexity score C(x, y) through the Sigmoid function:

[0115] This allows more global attention weights to be assigned to complex regions, while smooth regions tend to be modeled locally, thereby improving the overall feature balance.

[0116] An example of the spatial-semantic joint weighting module is as follows: To enhance the model's understanding of spatial structure and semantic consistency, this invention designs a joint space-semantic weighting mechanism: Add a two-dimensional learnable positional code P to the input features; Semantic features are extracted using a pre-trained semantic segmentation network (DeepLabV3+). ; The integration of the three: .in Learnable weights; Calculate the weighted joint attention: This allows for the simultaneous capture of spatial relationships and semantic consistency on the feature map.

[0117] An example of a dynamic weight generation module (MLP controller) is as follows: To enable the fusion ratio and weight allocation to adapt to different input images, this invention introduces a dynamic weight generation module: Features after global average pooling of the image For input; The weight vector is output through a two-layer MLP: ; will generate These are used to control the local-global fusion coefficients respectively. Spatial-semantic weighting coefficients And the combination ratio of variance and entropy in the complexity perception module. This design enables the model to adaptively adjust attention allocation based on image complexity and structural characteristics, achieving a dynamic balance.

[0118] In complex scenes (such as images with dense text, images with document ghosting, or images with varying illumination), the above-mentioned image feature determination method can significantly improve the stability and global consistency of feature representation. Experimental results show that: On the standard dataset ICDAR2019, the model's character recognition accuracy improved by approximately 3.5%; The inference time only increases by 8% compared to the original SegFormer, while maintaining high computational efficiency; The model responds more accurately to complex textures and weak edge areas, resulting in a smoother visual effect.

[0119] Corresponding to the image feature determination method described above, this application also provides an image feature determination device. Figure 8 This is a schematic diagram of a module of an image feature determination device provided in an embodiment of this application. The image feature determination device 800 provided in this embodiment includes: The extraction module 810 is used to extract image features at at least one scale from the target image and obtain the target parameters corresponding to the image features. The target parameters include one or more of gradient intensity, local variance, and high-frequency energy response parameters. The first determining module 820 is used to determine the complexity of image features at each scale based on the target parameters, and to determine the weight of each scale under the self-attention mechanism based on the complexity of the image features, wherein the complexity and weight are positively correlated. The second determining module 830 is used to determine the attention energy of image features at each scale, and to determine the weight of each scale under the self-attention mechanism based on the complexity of the image features. The fusion module 840 is used to fuse various image features based on the self-attention mechanism, the attention energy of each weight and the image features at each scale, to obtain image fusion features.

[0120] In some implementations, the image feature determination device 800 is also used for: The attention energy matrix is ​​determined based on the attention energy of image features at each scale; The attention energy of image features at a given scale is determined based on a preset scaling factor, attention energy matrix, weights, and preset functional relationships.

[0121] In some implementations, the image feature determination device 800 is further used to: determine the average energy value of the image features based on the spatial size and number of channels of the image features; The average energy value is normalized to obtain the attention energy of the image features.

[0122] In some implementations, the image feature determination device 800 is also used for: Each image feature is upsampled to obtain features to be processed with the same spatial resolution; Based on the weights of each scale under the self-attention mechanism, the attention energy corresponding to each feature to be processed is weighted and balanced to obtain the image fusion features.

[0123] In some implementations, the image feature determination device 800 is also used to: perform channel aggregation on image features to obtain single-channel feature mapping parameters; Based on the single-channel feature mapping parameters, determine the first gradient of the image features in the horizontal direction and the second gradient in the vertical direction; The gradient strength corresponding to the image feature is determined based on the first gradient and the second gradient.

[0124] In some implementations, the image feature determination device 800 is also used for: A sliding window is set for image features, and the image features are slid across the sliding window to obtain local features; Based on the eigenvalues ​​of each feature in the local features, determine the eigenvalue mean of the local features, and then determine the target variance based on the eigenvalue mean and each eigenvalue. The target variance is normalized to obtain the local variance corresponding to the image features.

[0125] In some implementations, the image feature determination device 800 is also used for: Two-dimensional discrete wavelet decomposition is performed on the image features to obtain the first high-frequency detail subband parameters in the horizontal direction, the second high-frequency detail subband parameters in the vertical direction, and the third high-frequency detail subband parameters in the diagonal direction. Based on the first high-frequency detail subband parameter, the second high-frequency detail subband parameter, and the second high-frequency detail subband parameter, determine the high-frequency energy response parameter to be processed; The high-frequency energy response parameters to be processed are normalized to obtain the high-frequency energy response parameters corresponding to the image features.

[0126] The image feature determination apparatus and image feature determination method provided in the above embodiments of this application belong to the same concept and can execute the image feature determination method provided in any of the above embodiments of this application. They have the corresponding functional modules and beneficial effects for executing the image feature determination method. Technical details not described in detail in this embodiment can be found in the specific processing content of the image feature determination method provided in the above embodiments of this application, and will not be repeated here.

[0127] The functions implemented by each module in the image feature determination device can be implemented by the same or different processors, and this application embodiment does not limit this.

[0128] It should be understood that the modules in the above image feature determination device can be implemented in the form of processor calling firmware. For example, the system includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of each module of the device. The processor can be a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal to the device or external to the system. Alternatively, the modules in the system can be implemented in the form of hardware circuits. By designing the hardware circuits, some or all of the module functions can be implemented. The hardware circuits can be understood as one or more processors. For example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above modules are implemented by designing the logical relationships of the components within the circuit. In another implementation, the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through a configuration file, thereby implementing the functions of some or all of the above modules. All modules of the above image feature determination device can be implemented entirely by processor calling firmware, or entirely by hardware circuits, or partially by processor calling firmware with the remaining parts implemented by hardware circuits.

[0129] In this application embodiment, a processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP. In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above modules. Furthermore, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.

[0130] As can be seen, each module in the above image feature determination device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor types.

[0131] Furthermore, the modules in the above image feature determination device can be integrated in whole or in part, or they can be implemented independently. In one implementation, these modules are integrated together and implemented in the form of a System-on-Chip (SoC). The SoC may include at least one processor for implementing any of the above methods or implementing the functions of the modules of the device. The at least one processor may be of different types, such as CPU and FPGA, CPU and artificial intelligence processor, CPU and GPU, etc.

[0132] This application provides another structural schematic diagram of an electronic device, see [link to schematic diagram]. Figure 9 As shown, the electronic device includes a memory 900 and a processor 910; wherein the memory 900 is connected to the processor 910 and is used to store programs; the processor 910 is used to implement the image feature determination method disclosed in any of the above embodiments by running the programs stored in the memory 900.

[0133] Specifically, the aforementioned electronic device may further include: a bus, a communication interface 920, an input device 930, and an output device 940. The electronic device may also include a data transceiver module, an image monitoring module, and a signal monitoring module.

[0134] The processor 910, memory 900, communication interface 920, input device 930, and output device 940 are interconnected via a bus. Among them: A bus can include a pathway for transmitting information between various components in an electronic device.

[0135] The processor 910 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present invention. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0136] The processor 910 may include a main processor, as well as a baseband chip, modem, etc.

[0137] The memory 900 stores a program that executes the technical solution of this invention, and may also store an operating system and other key business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 900 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0138] Input device 930 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0139] Output device 940 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0140] The communication interface 920 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0141] The processor 910 executes the program stored in the memory 900 and calls other devices, which can be used to implement each step of any of the image feature determination methods provided in the above embodiments of this application.

[0142] It should be noted that the electronic device can be an in-vehicle terminal, a mobile phone, a wearable device, or a server, etc.; or it can be an image feature determination device including an in-vehicle terminal, etc.

[0143] This application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in the memory through the data interface to execute the image feature determination method described in any of the above embodiments. For the specific processing procedure and its beneficial effects, please refer to the above-described embodiments of the image feature determination method.

[0144] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the image feature determination methods according to various embodiments of this application as described in any of the above embodiments of this specification.

[0145] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the power device, as a standalone firmware package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0146] Furthermore, embodiments of this application may also be storage media storing a computer program, which is executed by a processor to perform the steps of the image feature determination method according to various embodiments of this application described in any of the above embodiments of this specification, specifically implementing the steps of the image feature determination method as described above.

[0147] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0148] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0149] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0150] The units of the apparatus in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0151] It should be understood that the disclosed terminals, devices, and methods can be implemented in other ways, given the several embodiments provided in this application. For example, the terminal embodiments described above are merely illustrative. For instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0152] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0153] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or as firmware functional modules or sub-modules.

[0154] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer firmware, or a combination of both. To clearly illustrate the interchangeability of hardware and firmware, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or firmware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0155] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly using hardware, firmware units executed by a processor, or a combination of both. The firmware unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0156] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0157] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image feature determination method characterized by, The method comprises the following steps: extracting image features at at least one scale from a target image, and obtaining target parameters corresponding to the image features, the target parameters comprising one or more of gradient intensity, local variance, and high-frequency energy response parameter; determining the complexity of the image features at each scale according to the target parameters, and determining the weight of each scale under the self-attention mechanism according to the complexity of the image features, wherein the complexity and the weight are in a positive correlation; determining the attention energy of the image features at each scale, and determining the weight of each scale under the self-attention mechanism according to the complexity of the image features; fusing the image features according to the self-attention mechanism, the weights, and the attention energies of the image features at each scale, to obtain image fusion features.

2. The image feature determination method according to claim 1, characterized in that, The method further comprises the following steps: determining an attention energy matrix according to the attention energies of the image features at each scale; determining the attention energy of the image features at each scale according to a preset scaling factor, the attention energy matrix, the weights, and a preset function relationship.

3. The image feature determination method according to claim 1, characterized in that, The method further comprises the following steps: determining an average energy value of the image features according to the spatial size and the number of channels of the image features; normalizing the average energy value to obtain the attention energy of the image features.

4. The image feature determination method according to claim 1, characterized in that, The method further comprises the following steps: upsampling the image features to obtain to-be-processed features with the same spatial resolution; weighting and balancing the attention energies corresponding to the to-be-processed features according to the weights to obtain the image fusion features.

5. The image feature determination method according to claim 1, characterized in that, The target parameters comprise gradient intensity, and the method further comprises the following steps: performing channel aggregation on the image features to obtain a single-channel feature mapping parameter; determining a first gradient in the horizontal direction and a second gradient in the vertical direction of the image features according to the single-channel feature mapping parameter; determining the gradient intensity corresponding to the image features according to the first gradient and the second gradient.

6. The image feature determination method of claim 1, wherein, The target parameters comprise local variance, and the method further comprises the following steps: setting a sliding window for the image features, and sliding the image features based on the sliding window to obtain local features; determining a feature mean value of the local features according to the feature values of the features in the local features, and determining a target variance according to the feature mean value and the feature values; normalizing the target variance to obtain the local variance corresponding to the image features.

7. The image feature determination method of claim 1, wherein, The target parameters comprise high-frequency energy response parameter, and the method further comprises the following steps: performing two-dimensional discrete wavelet decomposition on the image feature to obtain a first high-frequency detail sub-band parameter of the image feature in a horizontal direction, a second high-frequency detail sub-band parameter of the image feature in a vertical direction, and a third high-frequency detail sub-band parameter of the image feature in a diagonal direction; determining a high-frequency energy response parameter to be processed according to the first high-frequency detail sub-band parameter, the second high-frequency detail sub-band parameter, and the third high-frequency detail sub-band parameter; performing normalization processing on the high-frequency energy response parameter to be processed to obtain a high-frequency energy response parameter corresponding to the image feature.

8. An image feature determination apparatus characterized by comprising: Comprise: an extraction module configured to extract image features at at least one scale from a target image, and obtain target parameters corresponding to the image features, the target parameters comprising one or more of gradient intensity, local variance, and high-frequency energy response parameter; a first determination module configured to determine a complexity of the image features at each of the scales according to the target parameters, and determine a weight of each of the scales under a self-attention mechanism according to the complexity of the image features, wherein the complexity and the weight are in a positive correlation relationship; a second determination module configured to determine an attention energy of the image features at each of the scales, and determine a weight of each of the scales under the self-attention mechanism according to the complexity of the image features; a fusion module configured to fuse the image features according to the self-attention mechanism, the weights, and the attention energy of the image features at each of the scales to obtain image fusion features.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is run by a processor to implement the image feature determination method in any one of claims 1-7.

10. A computer program product, the computer program being executed by a processor to implement the image feature determination method in any one of claims 1-7.

Images