Intelligent image processing method and device for AI chip and medium

By employing intelligent image processing methods based on AI chips, K-MEANS filtering and regularization are used to generate image data matrices. Combined with residual learning and non-uniform quantization, the problems of regional importance differences and multimodal data collaboration in image processing by AI chips are solved, achieving efficient image processing and accurate recognition in dynamic environments.

CN121616544APending Publication Date: 2026-03-06SHANGHAI QIANYI INFORMATION TECHNOLOGY CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202511793077.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-01
Publication Date
2026-03-06

AI Technical Summary

Technical Problem

Existing AI chips fail to adequately consider the differences in importance of image content regions during image processing, resulting in the loss of details in key areas or redundant calculations in background areas. Furthermore, the lack of multimodal data collaboration mechanisms affects the accuracy of decision-making in dynamic scenarios.

Method used

Image data matrices are generated using K-MEANS filtering and regularization. Similarity scores are calculated using a query-key-value matrix. Attention weight matrices and key region mask matrices are extracted. Error compensation is performed using residual learning to achieve non-uniform quantization. Multi-scale similarity matching with a sample library is performed through hierarchical feature fusion to calculate confidence scores and finally generate an image processing report.

Benefits of technology

It enables intelligent switching between high-precision quantization in critical areas and low-precision quantization in non-critical areas, improving the deployment efficiency of AI chips on edge devices and enhancing target recognition accuracy in dynamic environments, thus providing reliable technical support for safety-critical applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121616544A_ABST
    Figure CN121616544A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent image processing method and device for an AI chip and a medium, and relates to the technical field of intelligent image processing, and the method comprises the steps: collecting original image data, carrying out K-MEANS filtering and regularization processing, and generating an image data matrix; based on the image data matrix, calculating a similarity score matrix through a query-key-value matrix, extracting an image key region, and generating a key region mask matrix and an attention weight matrix; calculating a quantization precision level parameter according to the attention weight matrix and the key area mask matrix, and performing error compensation in combination with residual learning to obtain a quantization compression feature vector; and based on the quantized compressed feature vector, performing multi-scale similarity matching with a sample library through hierarchical feature fusion, and calculating a confidence score in combination with an attention weight. According to the method, the identification precision is ensured, and the deployment efficiency of the AI chip on the edge equipment is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent image processing technology, and in particular to an intelligent image processing method, device and medium for AI chips. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, AI chips, as hardware platforms specifically designed to accelerate machine learning algorithms, have been widely used in the field of computer vision. Particularly in image processing tasks, AI chips, through parallel computing architectures and dedicated instruction sets, have improved the processing efficiency of complex models such as convolutional neural networks (CNNs) and attention mechanisms. Current technologies mostly employ end-to-end deep learning frameworks, combining multi-scale feature extraction and attention weighting mechanisms to automate the process from image acquisition to feature matching. In recent years, researchers have optimized computational efficiency by introducing techniques such as quantization compression and dynamic thresholding.

[0003] Existing technologies suffer from the following main shortcomings: First, during feature quantization, most schemes employ static or uniform quantization strategies, failing to adequately consider the regional differences in image content importance, leading to the loss of details in key regions or computational redundancy in background regions. Second, in the similarity matching stage, traditional methods lack multimodal data collaboration mechanisms, failing to effectively integrate environmental parameter data with historical information, thus limiting their adaptability to dynamic scenes. Especially under complex conditions such as changes in illumination and occlusion, the confidence assessment of existing schemes is often unreliable, affecting the accuracy of the final decision. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an intelligent image processing method for AI chips to solve the problem of balancing image processing efficiency and accuracy in complex scenarios.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides an intelligent image processing method for an AI chip, comprising, Raw image data is acquired and processed using K-MEANS filtering and regularization to generate an image data matrix. Based on the image data matrix, a similarity score matrix is ​​calculated using a query-key-value matrix, and key regions of the image are extracted to generate an attention weight matrix and a key region mask matrix. Based on the attention weight matrix and the key region mask matrix, quantization precision level parameters are calculated, and error compensation is performed using residual learning to obtain a quantized compressed feature vector. Based on the quantized compressed feature vector, multi-scale similarity matching is performed with a sample database through hierarchical feature fusion, and a confidence score is calculated using attention weights. Based on the confidence score and dynamic decision values, image content analysis and judgment are performed to generate an image processing report.

[0007] As a preferred embodiment of the intelligent image processing method for AI chips described in this invention, the original image data includes RGB image data and environmental parameter data.

[0008] As a preferred embodiment of the intelligent image processing method for AI chips described in this invention, the specific steps for acquiring raw image data, performing preprocessing, and generating an image data matrix are as follows: The raw image data is acquired and K-MEANS filtering is performed to obtain the filtered image data. The filtered image data is then regularized to obtain regularized image data. The structure of regularized image data is optimized and the format is converted by multi-scale feature fusion and dimensionality normalization methods to generate an image data matrix.

[0009] In a preferred embodiment of the intelligent image processing method for AI chips described in this invention, the step of calculating a similarity score matrix based on an image data matrix using a query-key-value matrix involves the following specific steps. Multi-scale feature extraction is performed on the image data matrix to generate a multi-scale feature map set; The NPU unit of the AI ​​chip performs linear projection operations on the multi-scale feature map set to form a query matrix, a key matrix, and a value matrix. The similarity score matrix is ​​calculated using the query matrix and the key matrix, and the spatial focus weight matrix is ​​generated using the softmax function.

[0010] As a preferred embodiment of the intelligent image processing method for AI chips described in this invention, the specific steps for extracting key regions of the image and generating an attention weight matrix and a key region mask matrix are as follows: Key regions of the image are extracted from the spatial focusing weight matrix, and a key region mask matrix is ​​generated by the DBSCAN clustering algorithm. Morphological post-processing is performed on the key region mask matrix, and attention weight matrix is ​​generated by combining the value matrix and the spatial focus weight matrix.

[0011] As a preferred embodiment of the intelligent image processing method for AI chips described in this invention, the steps of calculating quantization precision level parameters based on the attention weight matrix and the key region mask matrix, and performing error compensation by combining residual learning to obtain a quantized compressed feature vector are as follows: Based on the attention weight matrix and the key region mask matrix, the quantization precision level parameter is calculated. Calculate the dynamic quantization threshold based on the quantization accuracy level parameter and the attention weight matrix; Based on the dynamic quantization threshold, the attention weight matrix is ​​subjected to non-uniform quantization to obtain the quantized matrix; Error compensation is performed on the quantized matrix and attention weight matrix through residual learning to generate compensated feature vectors. Sparse coding is performed on the compensated feature vector to generate a sparse feature vector. The sparse feature vectors are subjected to scalar quantization and entropy encoding to generate quantized compressed feature vectors.

[0012] As a preferred embodiment of the intelligent image processing method for AI chips described in this invention, the steps of performing multi-scale similarity matching with a sample library based on quantized compressed feature vectors through hierarchical feature fusion and combining attention weights to calculate confidence scores are as follows: The quantized compressed feature vector is dequantized to generate a reconstructed feature vector. The reconstructed feature vectors are decomposed at multiple scales to generate a multi-scale feature set. Extract multi-scale template features from the sample library that correspond to the multi-scale feature set; Attention-weighted similarity matching is performed on multi-scale feature sets and multi-scale template features, and multi-scale similarity scores are calculated using Gaussian integral similarity. The multi-scale similarity scores are hierarchically fused to generate an overall similarity score, and the confidence score is calculated using the Sigmoid function.

[0013] As a preferred embodiment of the intelligent image processing method for AI chips described in this invention, the steps of analyzing and judging image content based on confidence scores and dynamic decision values ​​to generate an image processing report are as follows: Based on confidence scores, and combined with multimodal historical data and environmental parameter data, a multimodal dataset is generated through feature concatenation and weighted fusion. Based on the multimodal dataset, dynamic decision values ​​are calculated by dynamically adjusting the multimodal integral. The confidence score and dynamic decision value are compared, and image content analysis is performed to generate a binary decision state and adjustment value. The multimodal dataset, binary decision states, and adjustment values ​​are logically combined and the report is compiled to generate an image processing report.

[0014] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the intelligent image processing method for an AI chip as described in the first aspect of the present invention.

[0015] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the intelligent image processing method for an AI chip as described in the first aspect of the present invention.

[0016] The beneficial effects of this invention are as follows: By using non-uniform quantization processing and residual learning for error compensation, intelligent switching between high-precision quantization in critical areas and low-precision quantization in non-critical areas is achieved, which improves the deployment efficiency of AI chips on edge devices while ensuring recognition accuracy; The use of a dynamic adjustment integral algorithm to calculate decision values ​​solves the problem of insufficient decision reliability in dynamic environments, effectively improves the target recognition accuracy under adverse weather conditions, and provides reliable technical support for safety-critical applications. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of an intelligent image processing method used in AI chips.

[0019] Figure 2 A flowchart for generating an image data matrix.

[0020] Figure 3 A flowchart for generating the attention weight matrix and the key region mask matrix.

[0021] Figure 4 A flowchart for generating quantized compressed feature vectors. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides an intelligent image processing method for an AI chip, comprising the following steps: S1. Acquire raw image data and preprocess it to generate an image data matrix.

[0026] The original image data is acquired and processed by K-MEANS filtering to obtain the filtered image data.

[0027] It should be noted that RGB image data is acquired using an image acquisition device. The RGB image data consists of three components: red, green, and blue, which together constitute the color information of each pixel. Each pixel corresponds to three values ​​in the RGB image data, representing the red component value, green component value, and blue component value, respectively. Each component value ranges from 0 to 255, forming complete RGB image data.

[0028] When collecting environmental parameter data, environmental sensor devices are used to acquire environmental parameter data, including temperature, humidity, atmospheric pressure, and light intensity. The environmental sensor devices detect the temperature, humidity, air pressure, and light intensity in the environment through corresponding physical sensing elements, and convert these physical quantities into corresponding analog electrical signals. After the analog electrical signals are filtered and amplified by the signal conditioning circuit, they are input into the analog-to-digital converter to be converted into digital signals and packaged to form environmental parameter data.

[0029] Furthermore, the K-MEANS filtering process treats each pixel in the original image data as a data point, randomly selects several initial cluster centers, and performs iterative allocation and update operations. The allocation operation assigns each pixel to the nearest cluster center, and the update operation re-acquires the average color value of each cluster center as the pixel's value. The allocation and update operations are repeated until the cluster centers no longer change. Then, the color value of each pixel is replaced with the color value of the cluster center to which it belongs, thus obtaining the filtered image data.

[0030] The filtered image data is then regularized to obtain regularized image data.

[0031] The specific process includes performing zero-mean, unit-variance transformation on the red, green, and blue component values ​​of each pixel in the filtered image data. The zero-mean, unit-variance transformation obtains the mean and standard deviation of the red component values ​​in the filtered image data. Each red component value is then adjusted to a value centered on the mean and scaled according to the standard deviation. The green and blue component values ​​are processed in the same way, so that the red, green, and blue component values ​​each exhibit zero-mean and unit-variance distribution characteristics as a whole, thereby eliminating the influence of scale differences between channels and illumination changes, and obtaining regularized image data.

[0032] The structure of regularized image data is optimized and the format is converted by multi-scale feature fusion and dimensionality normalization methods to generate an image data matrix.

[0033] The specific process includes performing convolution or downsampling operations at different scales on the regularized image data to extract feature representations at multiple scales, forming a multi-scale feature set. The features at each scale in the multi-scale feature set are then aligned and stitched together in the channel dimension or spatial dimension to complete the multi-scale feature fusion. The fused features are then processed using a dimension normalization method, and the mean and variance of each feature dimension are obtained independently. All values ​​of the feature dimension are mapped to a unified numerical distribution range to make the values ​​of different dimensions comparable. Finally, the processed data is rearranged into a two-dimensional or three-dimensional regular array form to generate an image data matrix.

[0034] S2. Based on the image data matrix, calculate the similarity score matrix through the query-key-value matrix, extract key regions of the image, and generate the attention weight matrix and key region mask matrix.

[0035] Multi-scale feature extraction is performed on the image data matrix to generate a multi-scale feature map set.

[0036] The specific process includes using convolution kernels of different sizes or downsampling operations of different levels to perform multi-scale feature extraction on the image data matrix, thereby capturing the local details and global structural information of the image data matrix at different spatial resolutions. Each level of operation outputs a feature map of the corresponding scale, and the feature maps of all scales together form a multi-scale feature map set.

[0037] The NPU unit of the AI ​​chip performs linear projection operations on the multi-scale feature map set to form a query matrix, a key matrix, and a value matrix.

[0038] The specific process includes: unfolding the feature map of each scale in the multi-scale feature map set into a two-dimensional tensor in the channel dimension, where each row corresponds to a feature vector of a spatial location; performing matrix multiplication on the two-dimensional tensor of each scale with three independent learnable weight matrices, the size of which matches the input feature dimension and the target projection dimension, and being used to generate the query matrix, key matrix, and value matrix, respectively; after matrix multiplication, the input features of each scale are mapped to a unified embedding space, forming the query matrix, key matrix, and value matrix of the corresponding scale.

[0039] The similarity score matrix is ​​calculated using the query matrix and the key matrix, and the spatial focusing weight matrix is ​​generated using the softmax function, expressed as follows: ; in, Represents the similarity score matrix. Represents the query matrix. Represents the key matrix, This represents the transpose of the key matrix. This represents the dimension of the key vector.

[0040] The specific process includes performing matrix multiplication on the transposes of the query matrix and the key matrix to obtain an intermediate result reflecting the similarity between each query vector in the query matrix and each key vector in the key matrix. This intermediate result is the similarity score matrix. Then, the softmax function is applied to each row of the similarity score matrix to convert the elements in each row into non-negative real numbers and make the sum of the elements in the row equal to one. This highlights the weight of high similarity positions and suppresses the response of low similarity positions, ultimately generating a spatial focusing weight matrix.

[0041] Key regions of the image are extracted from the spatial focus weight matrix, and a key region mask matrix is ​​generated using the DBSCAN clustering algorithm.

[0042] The specific process includes treating the value of each position in the spatial focus weight matrix as a measure of the importance of that position in the image, selecting positions with importance measures higher than a dynamic threshold as candidate key points, and inputting the spatial coordinates of the candidate key points into the DBSCAN clustering algorithm. The DBSCAN clustering algorithm groups nearby and dense candidate key points into the same cluster based on the spatial distance and neighborhood density between points. Each cluster corresponds to a continuous key region of the image, and a mask value of 1 is assigned to the spatial position covered by each cluster, while the remaining positions are assigned a mask value of 0, thereby forming a key region mask matrix.

[0043] It should be noted that the DBSCAN clustering algorithm is a density-based clustering method. Using two parameters—neighborhood radius and minimum number of neighbors—it groups nearby points with sufficient density into the same cluster, while points in sparse regions are considered noise. The DBSCAN clustering algorithm starts from any unvisited point. If the number of points in its neighborhood reaches the minimum number of neighbors, it becomes the core point and expands to form a cluster. This process recursively adds all density-reachable points until the cluster can no longer expand, ultimately outputting multiple clusters and noise points.

[0044] The dynamic threshold refers to the decision boundary value obtained through the statistical characteristics of the spatial focusing weight matrix. The acquisition process is as follows: based on the overall distribution characteristics of the spatial focusing weight matrix and combined with the sensitivity parameter, it is obtained in real time through percentile statistics or the mean-standard deviation weighted formula.

[0045] Morphological post-processing is performed on the key region mask matrix, and attention weight matrix is ​​generated by combining the value matrix and the spatial focus weight matrix.

[0046] The specific process includes performing morphological closing and opening operations sequentially on the key region mask matrix during morphological post-processing. The morphological closing operation is used to fill small holes in the regions with a mask value of one in the key region mask matrix and connect adjacent regions with a mask value of one in the key region mask matrix. The morphological opening operation is used to remove isolated and small regions with a mask value of one in the key region mask matrix, thereby obtaining a binary mask with smooth boundaries and a complete structure. The morphologically post-processed key region mask matrix is ​​then multiplied element-wise with the spatial focus weight matrix to obtain the first intermediate matrix. The first intermediate matrix is ​​then multiplied element-wise with the value matrix so that the value at each spatial location simultaneously reflects the focus intensity in the spatial focus weight matrix and the feature content in the value matrix, generating the attention weight matrix.

[0047] S3. Based on the attention weight matrix and the key region mask matrix, calculate the quantization precision level parameters, and combine residual learning to perform error compensation to obtain the quantization compressed feature vector.

[0048] Based on the attention weight matrix and the key region mask matrix, the quantization precision level parameter is calculated using dynamic parameters, expressed as follows: ; in, This indicates the quantization precision level parameter. Indicates the index of a matrix row. Indicates the index of a matrix column. The attention weight matrix is ​​represented in the th... line, number The element values ​​of the column, The key region mask matrix represents the first... line, number The element values ​​of the column.

[0049] The specific process includes multiplying the element values ​​of each row and column in the attention weight matrix with the element values ​​of the corresponding row and column in the key region mask matrix to obtain the weighted value at the corresponding position, and summing the weighted response values ​​corresponding to all row and column index positions to form the numerator; at the same time, summing the element values ​​of all row and column index positions in the key region mask matrix to form the denominator; finally, the numerator and denominator are compared, and the resulting ratio is the quantization precision level parameter.

[0050] The dynamic quantization threshold is calculated based on the quantization precision level parameter and the attention weight matrix, expressed as follows: ; in, Indicates the dynamic quantization threshold. This represents the natural exponential function. The attention weight matrix is ​​represented in the th... line, number The gradient vector of the column, This represents the attenuation constant.

[0051] The specific process includes calculating the gradient vector of the attention weight matrix at each row and column position, substituting the magnitude of the gradient vector into the natural exponential function, scaling it according to the decay constant as the denominator to obtain the exponentially decayed response value at the corresponding position, and then multiplying the exponentially decayed response value at each position with the quantization precision level parameter to finally obtain the dynamic quantization threshold.

[0052] It should be noted that the decay constant is a parameter that controls the decay rate of the exponential function. Its value is obtained through grid search optimization, and its typical range is 0.1 to 1.0.

[0053] Based on the dynamic quantization threshold, the attention weight matrix is ​​subjected to non-uniform quantization to obtain the quantized matrix.

[0054] The specific process includes: when performing non-uniform quantization on the attention weight matrix according to the dynamic quantization threshold, comparing the element value at each position in the attention weight matrix with the dynamic quantization threshold, and mapping the element value to the nearest quantization level based on the comparison result. The interval between quantization levels is dynamically adjusted according to the dynamic quantization threshold, so that the position corresponding to the key area adopts a finer quantization interval, and the position corresponding to the non-key area adopts a coarser quantization interval, thereby reducing data redundancy while retaining important information, and finally obtaining the quantized matrix.

[0055] Error compensation is performed on the quantized matrix and attention weight matrix through residual learning to generate compensated feature vectors.

[0056] The specific process includes subtracting the attention weight matrix from the quantized matrix element-wise at each position in the same row and column to obtain a residual matrix of the same size. Each element of the residual matrix represents the original value lost due to non-uniform quantization at the corresponding position. The residual matrix is ​​used as input to a residual learning structure containing multiple convolutional operations. The residual learning structure is composed of several convolutional layers connected in sequence. Each convolutional layer uses a convolutional kernel of a fixed size to locally perceive the residual matrix and enhances its expressive power through a non-linear activation function, thereby outputting a corrected feature map with the same size as the input residual matrix. The corrected feature map captures the spatial correlation and structural patterns of the quantization error (e.g., local error clustering patterns, cross-regional error propagation paths, spatial continuity preservation characteristics, and multi-scale error distribution patterns). The corrected feature map is then added element-wise to the quantized matrix at the same position to obtain a compensated feature vector.

[0057] The compensated feature vectors are subjected to sparse encoding to generate sparse feature vectors.

[0058] The specific process includes: when performing sparse encoding on the compensated feature vector, obtaining a pre-trained overcomplete dictionary; performing projection decomposition on the compensated feature vector through the overcomplete dictionary; and finding the sparse linear combination that best represents the structural features of the compensated feature vector. During the projection decomposition, L1 norm regularization is used to constrain the sparsity of the coefficient vector, ensuring that only a small number of dictionary atoms are activated. Activated atoms correspond to the main structural components of the compensated feature vector, while non-activated atom coefficients are compressed to zero. The product of the sparse coefficient vector and the overcomplete dictionary reconstructs a sparse feature vector that retains the main features. The sparse feature vector reduces dimensionality while preserving discriminative information.

[0059] Furthermore, the training process of the overcomplete dictionary takes a large number of image feature samples as input and initializes a random matrix as the initial overcomplete dictionary. This random matrix has more columns than rows, and each column is called an atom. Then, sparse coding and dictionary update are performed alternately. Sparse coding identifies an optimization problem with L1 norm constraints for each image feature sample in the current overcomplete dictionary, and obtains the corresponding sparse coefficient vector. Dictionary update updates the atoms of the overcomplete dictionary column by column, while keeping all sparse coefficient vectors fixed, so that the total reconstruction error between all image feature samples reconstructed from the current sparse coefficient vector and the overcomplete dictionary and the original image feature samples is minimized. After repeated iterations, the overcomplete dictionary converges, and the trained overcomplete dictionary is obtained.

[0060] The sparse feature vectors are subjected to scalar quantization and entropy encoding to generate quantized compressed feature vectors.

[0061] The specific process includes: independently performing scalar quantization on each non-zero element in the sparse feature vector; mapping continuously valued elements to a finite number of discrete quantization levels, with the quantization level determined by a dynamic quantization threshold; and performing entropy coding on the quantized sparse feature vector, using Huffman coding or arithmetic coding methods, assigning variable-length codewords based on the frequency of occurrence of quantized elements, assigning short codewords to quantized values ​​with high occurrence frequency, and assigning long codewords to quantized values ​​with low occurrence frequency, ultimately generating a quantized compressed feature vector.

[0062] S4. Based on quantized compressed feature vectors, multi-scale similarity matching is performed with the sample library through hierarchical feature fusion, and confidence scores are calculated by combining attention weights.

[0063] The quantized compressed feature vector is dequantized to generate a reconstructed feature vector.

[0064] The specific process includes restoring each quantization level value in the quantization compression feature vector to a continuous value according to the inverse mapping relationship corresponding to the scalar quantization process. This inverse mapping relationship is restored based on the midpoint or reconstructed value of the quantization interval determined by the dynamic quantization threshold. The restored continuous values ​​are then rearranged according to the positional structure of the original sparse feature vector, and zero values ​​are filled in the original sparse positions, thereby restoring a floating-point representation with the same dimension as the sparse feature vector, and finally generating a reconstructed feature vector.

[0065] It should be noted that the inverse mapping relationship refers to the mathematical correspondence rule for restoring discrete quantization level values ​​to continuous values ​​during the dequantization process. Based on the quantization interval mapping table established during scalar quantization, the discrete values ​​are accurately restored to continuous values ​​through linear or nonlinear reconstruction functions.

[0066] The reconstructed feature vectors are decomposed into multi-scale features to generate multi-scale feature sets.

[0067] The specific process includes the following steps: when performing multi-scale decomposition on the reconstructed feature vector, the reconstructed feature vector is sequentially input into multiple convolutional kernels of different scales or downsampling operations of different levels. Each level of operation extracts the feature representation of the reconstructed feature vector at a specific spatial resolution. The shallower levels retain high-resolution detail information, while the deeper levels capture low-resolution semantic information. The feature representations output by all levels together constitute a multi-scale feature set.

[0068] Extract multi-scale template features from the sample library that correspond to the multi-scale feature set.

[0069] The specific process includes: when extracting multi-scale template features corresponding to the multi-scale feature set from the sample library, firstly, determining the category label or content semantics corresponding to the feature at each scale in the multi-scale feature set, and searching for a pre-stored template that matches the category label or content semantics in the sample library. This pre-stored template is also processed by multi-scale decomposition and stored as feature representations at multiple scales. The feature representations that correspond one-to-one with each scale of the multi-scale feature set in the pre-stored template are extracted one by one and combined to form multi-scale template features.

[0070] It should be noted that the sample library is a pre-stored collection containing multi-scale template features, obtained by performing the same multi-scale decomposition and feature extraction operations on a large amount of labeled image data as the current processing flow.

[0071] Pre-stored templates refer to a set of multi-scale feature representations that are pre-stored in a sample library. These templates are obtained by performing the same multi-scale decomposition and feature extraction operations on a large amount of labeled image data as the current processing flow. Each template corresponds to a standardized feature representation of a specific category or content.

[0072] Attention-weighted similarity matching is performed on the multi-scale feature set and multi-scale template features, and the multi-scale similarity score is calculated using Gaussian integral similarity, expressed as: ; in, Representing scale The similarity score below Indicates scale index. Represents spatial coordinates in the horizontal direction. Represents spatial coordinates in the vertical direction. Representing scale Below is the spatial position eigenvalues, Representing scale Below is the spatial position template feature values, Indicates spatial location Attention weight value at the location.

[0073] in, Indicated by The Gaussian kernel function with bandwidth parameter in spatial location The weighted value at the position is expressed as: ; in, Indicates bandwidth parameter, It represents pi (π).

[0074] The specific process includes feeding the feature values ​​of the multi-scale feature set and the multi-scale template features at the same spatial location into the Gaussian integral similarity function at each scale. The Gaussian integral similarity function generates a local similarity response based on the difference between the two, and multiplies the local similarity response with the weight value of the attention weight matrix at the corresponding position to highlight the matching contribution of the key region. Finally, the weighted similarity responses of all spatial locations are accumulated to obtain the multi-scale similarity score at each scale.

[0075] It should be noted that in the Gaussian integral similarity function, the bandwidth parameter is a scalar parameter that controls the decay rate of the Gaussian function, used to adjust the sensitivity of similarity to feature differences. The larger the bandwidth parameter, the smoother the Gaussian function, the higher the tolerance for feature differences, and the slower the similarity decreases; the smaller the bandwidth parameter, the steeper the Gaussian function, the more sensitive it is to feature differences, and the faster the similarity decreases.

[0076] The multi-scale similarity scores are hierarchically fused to generate an overall similarity score, and the confidence score is calculated using the Sigmoid function, expressed as follows: ; in, This represents the confidence score. Represents the slope parameter. This represents the overall similarity score. This represents the decision offset.

[0077] The specific process includes: when performing hierarchical fusion of multi-scale similarity scores, the multi-scale similarity scores at all scales are weighted and summed according to spatial hierarchy to obtain the overall similarity score. The overall similarity score is then substituted into the Sigmoid function. The Sigmoid function controls the steepness of the response curve with a slope parameter and adjusts the position of the curve on the horizontal axis with a decision offset. Finally, a value between zero and one is output, which represents the confidence score.

[0078] It should be noted that the slope parameter is obtained by fitting and optimizing the overall similarity score of the labeled samples with the corresponding true labels during the training phase. Gradient descent or maximum likelihood estimation is usually used to adjust the value so that the confidence score can more accurately distinguish between positive and negative samples.

[0079] Spatial hierarchical relationships refer to the spatial correspondence and semantic dependencies between feature maps of different scales in multi-scale feature representation. These relationships are established through downsampling ratios between scales, receptive field alignment, and feature transfer paths, ensuring a coherent representation from fine-grained details to coarse-grained semantics.

[0080] S5. Based on the confidence score and combined with the dynamic decision value, perform image content analysis and judgment, and generate an image processing report.

[0081] Based on confidence scores, multimodal historical data and environmental parameter data are combined to generate a multimodal dataset through feature concatenation and weighted fusion.

[0082] The specific process includes aligning the confidence scores with the multimodal historical data in the time dimension to ensure that each modality data corresponds to the same event or state as the current confidence score, and associating the environmental parameter data with the corresponding modality in the multimodal historical data according to their physical meaning, such as associating the light intensity parameter with the visual modality data and the temperature parameter with the infrared modality data; using fusion methods such as feature splicing or weighted fusion to uniformly map the confidence scores, multimodal historical data, and environmental parameter data to the same feature space; and finally forming a multimodal dataset containing the confidence scores, multimodal historical data, and environmental parameter data.

[0083] It should be noted that multimodal historical data refers to a collection of information in various modalities, including images, audio, text, and sensor signals, collected at past points in time. This information is synchronously recorded and stored in historical tasks or scenarios by sensing units deployed on different devices or platforms.

[0084] Based on the multimodal dataset, the dynamic decision value is calculated by dynamically adjusting the multimodal integral, and the expression is: ; in, Represents dynamic decision values. Indicates a time index. Indicates the size of the time window. Indicates at a point in time The weighting coefficient value, Indicates at a point in time The confidence score.

[0085] The specific process includes extracting the confidence score for each time point from the multimodal dataset and generating time-related weight coefficients by combining the environmental parameter data corresponding to this time point with the multimodal historical data; then multiplying the confidence score for each time point with the corresponding weight coefficient to obtain the weighted confidence score; finally, accumulating the weighted confidence scores for all time points within the time window, and this accumulated result is the dynamic decision value.

[0086] Dynamic decision value refers to the adaptive decision threshold obtained by multimodal dynamic adjustment integral algorithm, which is generated by fusing historical confidence sequence, environmental parameter data and real-time context features. It is used to dynamically adjust the judgment criteria for image content analysis and realize intelligent decision adaptation under different environmental conditions.

[0087] The confidence score and dynamic decision value are compared, and image content analysis is performed to generate a binary decision state and adjustment value.

[0088] The specific process includes comparing the confidence score with the dynamic decision value. If the confidence score is greater than or equal to the dynamic decision value, the current image content is considered to have sufficient matching degree with the multimodal template features, and the image content is determined to belong to the target category. At this time, a binary decision state of one is generated. If the confidence score is less than the dynamic decision value, the current image content is considered to have insufficient matching with the multimodal template features, and the image content is determined not to belong to the target category. At this time, a binary decision state of zero is generated. Then, the absolute value of the difference between the confidence score and the dynamic decision value is obtained, and this absolute value of the difference is used as the adjustment value.

[0089] The multimodal dataset, binary decision states, and adjustment values ​​are logically combined and the report is compiled to generate an image processing report.

[0090] The specific process includes: extracting confidence scores, historical observation records for each modality, and environmental parameter data from the multimodal dataset, and organizing these contents into complete contextual information in chronological order; then embedding the binary decision state as the core judgment result into the contextual information to clearly identify whether the image content conforms to the target category; associating the adjustment value with the difference between the confidence score and the dynamic decision value to indicate the reliability of the current decision and whether further adjustment is needed, and logically integrating it according to semantic rules. For example, when the binary decision state is one and the adjustment value is small, the judgment result is stable; when the binary decision state is zero and the adjustment value is large, it indicates that there may be boundary conditions; finally, the integrated structured information is compiled into a complete document containing the decision conclusion, the supporting data, the confidence level, and the adjustment suggestions according to the standard format, generating an image processing report.

[0091] In summary, this invention achieves intelligent switching between high-precision quantization in critical areas and low-precision quantization in non-critical areas through non-uniform quantization processing and residual learning for error compensation. This ensures recognition accuracy while improving the deployment efficiency of AI chips on edge devices. Furthermore, the use of a dynamically adjusted integral algorithm to calculate decision values ​​solves the problem of insufficient decision reliability in dynamic environments, effectively improving target recognition accuracy under adverse weather conditions and providing reliable technical support for safety-critical applications.

[0092] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An intelligent image processing method for an AI chip, characterized in that: The application relates to an image content analysis method and device. Collect original image data, perform K-MEANS filtering and regularization processing, and generate an image data matrix; Based on the image data matrix, a similarity score matrix is calculated through a query-key-value matrix, and an image key region is extracted to generate an attention weight matrix and a key region mask matrix; According to the attention weight matrix and the key region mask matrix, a quantization precision level parameter is calculated, and error compensation is performed through residual learning to obtain a quantized compressed feature vector; Based on the quantized compressed feature vector, multi-scale similarity matching is performed through hierarchical feature fusion and a sample library, and a confidence score is calculated in combination with attention weight; Based on the confidence score, image content analysis is judged in combination with a dynamic decision value to generate an image processing report. 2.The intelligent image processing method for an AI chip of claim 1, wherein: The original image data includes RGB image data and environmental parameter data. 3.The intelligent image processing method for an AI chip of claim 2, wherein: The original image data is collected, preprocessed, and an image data matrix is generated, and the specific steps are as follows, The original image data is collected, K-MEANS filtering is performed, and filtered image data is obtained; The filtered image data is subjected to regularization processing to obtain regularized image data; The regularized image data is subjected to structure optimization and format conversion through a multi-scale feature fusion and dimension normalization method to generate an image data matrix. 4.The intelligent image processing method for an AI chip of claim 3, wherein: Based on the image data matrix, a similarity score matrix is calculated through a query-key-value matrix, and the specific steps are as follows, Multi-scale feature extraction is performed on the image data matrix to generate a multi-scale feature map set; The multi-scale feature map set is subjected to linear projection operation through an NPU unit of an AI chip to form a query matrix, a key matrix and a value matrix; The similarity score matrix is calculated through the query matrix and the key matrix, and a spatial focus weight matrix is generated using a softmax function. 5.The intelligent image processing method for an AI chip of claim 4, wherein: The image key region is extracted from the spatial focus weight matrix, and a key region mask matrix is generated through a DBSCAN clustering algorithm; The key region mask matrix is subjected to morphological post-processing, and an attention weight matrix is generated in combination with the value matrix and the spatial focus weight matrix. According to the attention weight matrix and the key region mask matrix, a quantization precision level parameter is calculated, and error compensation is performed through residual learning to obtain a quantized compressed feature vector, and the specific steps are as follows, 6.The intelligent image processing method for an AI chip of claim 5, wherein: Based on the attention weight matrix and the key region mask matrix, a quantization precision level parameter is calculated; According to the quantization precision level parameter and the attention weight matrix, a dynamic quantization threshold is calculated; According to the dynamic quantization threshold, the attention weight matrix is subjected to non-uniform quantization processing to obtain a quantized matrix; Error compensation is performed on the quantized matrix and the attention weight matrix through residual learning to generate a compensated feature vector; The compensated feature vector is subjected to sparse coding processing to generate a sparse feature vector; The sparse feature vector is subjected to scalar quantization and entropy coding processing to generate a quantized compressed feature vector. Based on the quantized compressed feature vector, multi-scale similarity matching is performed through hierarchical feature fusion and a sample library, and a confidence score is calculated in combination with attention weight, and the specific steps are as follows, 7.The intelligent image processing method for an AI chip of claim 6, wherein: ​ The quantized compressed feature vector is dequantized to generate a reconstructed feature vector; The reconstructed feature vector is subjected to multi-scale decomposition processing to generate a multi-scale feature set; Multi-scale template features corresponding to the multi-scale feature set are extracted from a sample library; The multi-scale feature set and the multi-scale template features are subjected to attention-weighted similarity matching, and a multi-scale similarity score is calculated through Gaussian integral similarity; The multi-scale similarity score is subjected to hierarchical fusion to generate an overall similarity score, and a confidence score is calculated through a Sigmoid function. 8.The intelligent image processing method for an AI chip of claim 7, wherein: Based on the confidence score, the image content analysis judgment is performed in combination with a dynamic decision value to generate an image processing report, and the specific steps are as follows, Based on the confidence score, multi-modal data sets are generated through feature splicing and weighted fusion in combination with multi-modal historical data and environmental parameter data; According to the multi-modal data sets, a dynamic decision value is calculated through multi-modal dynamic adjustment integration; The confidence score and the dynamic decision value are compared, and the image content analysis judgment is performed to generate a binary decision state and an adjustment value; The multi-modal data sets, the binary decision state and the adjustment value are logically combined and reported to generate an image processing report. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the intelligent image processing method for the AI chip according to any one of claims 1-8.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the intelligent image processing method for the AI chip according to any one of claims 1-8.

Citation Information

Cited By

  • Visual task-oriented linear attention mechanism method and device and electronic equipment

    CN122047306A

  • Linear attention mechanism methods, devices and electronic devices for vision tasks

    CN122047306B