Image recognition and processing system based on deep learning

Abstract

The invention provides an image recognition and processing system based on deep learning. The image recognition and processing system comprises an image acquisition module for acquiring original image data; the image preprocessing module performs noise suppression, edge enhancement and color correction on the image; the multi-scale feature extraction module extracts multi-dimensional features through a cascade convolutional layer and a self-attention mechanism; the self-adaptive classification and identification module performs multi-class classification based on a dynamic weight distribution algorithm; the dynamic optimization control module adjusts hyper-parameters in real time through a multi-objective optimization algorithm; the result output module outputs a classification result and a confidence score; and the feedback learning module updates the network weight according to the confidence score. All the modules work cooperatively, and efficient and dynamic image recognition and processing are achieved. According to the method, efficient and accurate image recognition and processing can be realized, and the adaptability, classification accuracy and robustness of the system are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning technology, and more specifically, to an image recognition and processing system based on deep learning. Background Art

[0002] In the field of image recognition and processing, with the rapid development of computer vision technology, deep learning methods have gradually become mainstream. Existing image recognition systems usually rely on traditional convolutional neural network (CNN) architectures to extract image features through multi-layer convolution and pooling operations and perform classification and recognition. However, these systems have limitations when processing complex images, such as sensitivity to noise, insufficient extraction of edge information, and difficulty in effectively fusing multi-scale features. In addition, traditional systems have poor adaptability in dynamic environments, and it is difficult to adjust network parameters in real time to cope with image changes in different scenarios.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: the existing image recognition system is not perfect in image preprocessing, resulting in inaccurate feature extraction; the multi-scale feature fusion capability is insufficient and cannot fully utilize global and local information; the classification and recognition module lacks a dynamic adjustment mechanism and is difficult to adapt to diverse image categories and complex scenes; the network optimization process lacks a real-time feedback learning mechanism, making it difficult to continuously improve model performance. Summary of the invention

[0004] The present invention provides an image recognition and processing system based on deep learning, comprising: An image acquisition module, used for acquiring raw image data; The image preprocessing module is connected to the image acquisition module to perform noise suppression, edge enhancement and color correction on the original image; The multi-scale feature extraction module is connected to the image preprocessing module to extract local and global multi-dimensional features through cascaded convolutional layers and self-attention mechanism; The adaptive classification and recognition module is connected to the multi-scale feature extraction module to classify features into multiple categories based on a dynamic weight allocation algorithm; The dynamic optimization control module is connected to the adaptive classification and recognition module to adjust the network hyperparameters in real time through a multi-objective optimization algorithm; The result output module is connected to the dynamic optimization control module to output the classification results and confidence scores; The feedback learning module is connected to the result output module, updates the network weights according to the confidence score, and is synchronized to the multi-scale feature extraction module and the adaptive classification and recognition module.

[0005] Furthermore, the image preprocessing module includes: The noise suppression unit uses an asymmetric Gaussian filtering algorithm, and its filtering kernel function is:

[0006] in, is the horizontal and vertical standard deviation, is a learnable offset used to dynamically adjust the filtering direction; It is the coordinate offset relative to the center point of the filter kernel; The edge enhancement unit enhances the high-frequency edge components based on the adaptive Laplacian operator, and its operator weight matrix is:

[0007] The color correction unit adjusts the color deviation through the nonlinear mapping function of the CIELab color space. The mapping function is:

[0008] in, is the dynamic compensation coefficient based on image histogram statistics; , , It is the three-channel value of CIELab color space after correction.

[0009] Furthermore, the multi-scale feature extraction module performs the following operations in sequence: Step S1: Extract multi-scale local features through parallel 3×3, 5×5, and 7×7 convolution kernels and output feature map groups ; Step S2: Input the feature maps of each scale into the improved self-attention mechanism and calculate the cross-scale feature correlation matrix:

[0010] in, , are the query and key vectors, is the trainable projection matrix, is the preset scale-priority mask matrix; , It is the feature map output by the i-th and j-th scale layers; Step S3: Generate a global descriptor through a gated fusion mechanism:

[0011] in, is the Sigmoid function, is the fusion weight matrix, Represents element-wise multiplication; The feature map The attention weight matrix.

[0012] Furthermore, the dynamic weight allocation algorithm of the adaptive classification and recognition module satisfies the iterative formula:

[0013] in, is the exponential moving average term, is a learnable attenuation coefficient that satisfies , is the category confidence difference measure; is the category confidence difference; is the original weight at the current moment; is the EMA weight of the previous moment.

[0014] Furthermore, the category confidence difference The calculation method is:

[0015] in, For the The predicted probability of the class, is the total number of categories, is the temperature coefficient, used to suppress the difference contribution of high confidence categories, is the predicted probability of the mth class.

[0016] Furthermore, the multi-objective optimization algorithm of the dynamic optimization control module is defined as follows: Objective function:

[0017] Dynamic constraints:

[0018] in, is the cross entropy classification loss, is the bounding box regression loss, For the prediction distribution With the prior distribution The KL divergence of is the regularization coefficient; It means to find the λ that minimizes the objective function within the range of the variable λ. represents the Euclidean norm of vector λ; express The square of .

[0019] Furthermore, the dynamic adjustment coefficient in the dynamic constraint condition The update rule is:

[0020] in, is the comprehensive error, is the regularization coefficient of the jth loss term, is an adaptive learning rate based on the gradient magnitude: .

[0021] Furthermore, the feedback learning module performs the following operations: Step F1: Generate weight update according to classification confidence:

[0022] in, are the momentum and noise injection coefficients, is the confidence suppression factor; Step F2: Update parameters using the Nesterov accelerated momentum strategy:

[0023]

[0024] in, is the second-order momentum coefficient, is the momentum accumulation of the tth iteration; is the staged momentum cache, is the dynamic step size: .

[0025] Furthermore, the scale-first mask matrix The generation rules are:

[0026] in, is the average activation intensity of the feature map.

[0027] Furthermore, the prior distribution The generation method is:

[0028] in, To distribute the smooth temperature, is the number of samples in the current batch, is the indicator function.

[0029] According to the above-mentioned embodiment of the present invention, at least the following beneficial effects are achieved: the image recognition and processing system of the present invention can effectively improve the accuracy and robustness of image recognition through the collaborative work of multiple modules. The image preprocessing module adopts asymmetric Gaussian filtering, adaptive Laplace edge enhancement and correction technology based on CIELab color space, which can improve the image quality in a targeted manner and provide clearer and more accurate input data for subsequent feature extraction. The multi-scale feature extraction module combines parallel convolution kernels and improved self-attention mechanism to extract local and global features at the same time, and generates global descriptors through gated fusion mechanism, so as to make full use of multi-scale information and enhance the ability to understand complex image scenes. The adaptive classification recognition module is based on a dynamic weight allocation algorithm, which can be flexibly adjusted according to the importance of features of different categories to further improve the classification accuracy.

[0030] In addition, the dynamic optimization control module adjusts the network hyperparameters in real time through a multi-objective optimization algorithm to ensure that the system can maintain optimal performance under different conditions. The feedback learning module dynamically updates the network weights based on the classification confidence score, enabling the system to continuously learn and optimize, further improving adaptability and generalization capabilities. This closed-loop feedback mechanism can also effectively reduce the model's reliance on prior knowledge and enhance its stability and reliability in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which: Figure 1 A schematic diagram of the structure of an image recognition and processing system based on deep learning provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0032] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0033] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0034] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0035] Reference below Figure 1 , Figure 1 This is a schematic diagram of the structure of an image recognition and processing system based on deep learning provided by an embodiment of the present invention. Figure 1 As shown, a deep learning-based image recognition and processing system 100 includes: An image acquisition module 101 is used to acquire original image data; The image preprocessing module 102 is connected to the image acquisition module to perform noise suppression, edge enhancement and color correction on the original image; The multi-scale feature extraction module 103 is connected to the image preprocessing module and extracts local and global multi-dimensional features through cascaded convolutional layers and a self-attention mechanism; The adaptive classification and recognition module 104 is connected to the multi-scale feature extraction module and classifies the features into multiple categories based on a dynamic weight allocation algorithm; The dynamic optimization control module 105 is connected to the adaptive classification and recognition module to adjust the network hyperparameters in real time through a multi-objective optimization algorithm; The result output module 106 is connected to the dynamic optimization control module to output the classification result and confidence score; The feedback learning module 107 is connected to the result output module, updates the network weight according to the confidence score, and synchronizes to the multi-scale feature extraction module and the adaptive classification and recognition module.

[0036] It should be noted that the entire system consists of multiple modules that work together. The image acquisition module is used to obtain raw image data. Raw image data refers to the initial image information that has not been processed by the system. These images may come from camera shooting, reading from storage devices, and other methods. The image preprocessing module is connected to the image acquisition module. Its main task is to perform noise suppression, edge enhancement, and color correction on the raw image. Noise suppression is to remove interference signals in the image to make the image clearer; edge enhancement aims to highlight the edges of objects in the image and make the object contours more obvious; color correction is to adjust the color deviation of the image to make the image color more accurate. The multi-scale feature extraction module is connected to the image preprocessing module, and extracts local and global multi-dimensional features through cascaded convolutional layers and self-attention mechanisms. The convolutional layer is an operation structure used to extract image features in deep learning. Different feature maps can be obtained through convolution operations; the self-attention mechanism allows the model to focus on the relationship between different parts of the image. The multi-dimensional features cover the features of the image at different scales, different positions, and different channels. The adaptive classification and recognition module is connected to the multi-scale feature extraction module, and uses the dynamic weight allocation algorithm to perform multi-category classification on the extracted features. Multi-category classification means classifying the image content into multiple preset categories. The dynamic weight allocation algorithm is a method that can adjust the weight according to different situations. The dynamic optimization control module is connected to the adaptive classification and recognition module, and uses the multi-objective optimization algorithm to adjust the network hyperparameters in real time. The network hyperparameters are parameters pre-set before model training. The role of the multi-objective optimization algorithm is to balance multiple different goals to improve the model performance. The result output module is connected to the adaptive classification and recognition module and is responsible for outputting the classification results and confidence scores. The classification result is the category to which the image is determined to belong, and the confidence score is used to measure the model's credibility in the classification result. The feedback learning module is connected to the result output module and the dynamic optimization control module, and updates the network weights according to the confidence score, and synchronizes the updated weights to the multi-scale feature extraction module and the adaptive classification and recognition module. The network weights are the key parameters in the model that affect its performance. Updating the weights helps the model perform better in the future.

[0037] Specifically, in the image acquisition module, if a camera is used to collect images, a camera with a suitable resolution can be selected according to the actual application scenario. For example, in a security monitoring scenario, a high-definition camera can be selected to clearly capture details; if the scene has high requirements for image real-time performance and relatively low requirements for details, a camera with a lower resolution but a higher frame rate can be selected. The noise suppression unit of the image preprocessing module adopts an asymmetric Gaussian filtering algorithm, and the filter kernel function and Respectively represent the standard deviation in the horizontal and vertical directions, which control the strength and range of filtering. The value is generally between 0.5 and 2 and can be adjusted according to the image noise situation; and It is a learnable offset used to dynamically adjust the filtering direction. The initial value can be set to 0 and is continuously optimized during the model training process. The adaptive Laplacian operator weight matrix of the edge enhancement unit middle, The value range is , The larger the value, the more obvious the edge enhancement effect, but it may also introduce more noise. In practical applications, the optimal value can be determined through multiple experiments. In the mapping function of the color correction unit, , , It is a dynamic compensation coefficient based on image histogram statistics. It is calculated by counting the distribution of pixel values ​​in different color channels of the image, and then adjusts the color deviation of the image. In the multi-scale feature extraction module, the parallel 3×3, 5×5, and 7×7 convolution kernels extract local features of different scales respectively. Small convolution kernels (such as 3×3) focus on extracting detailed features of the image, and large convolution kernels (such as 7×7) pay more attention to the overall structural features of the image. In the dynamic weight allocation algorithm of the adaptive classification and recognition module, , , , is a learnable decay coefficient, and , the initial value can be set to 0.25 and adjusted according to the model performance during training. They determine the degree of influence of different factors on weight update.

[0038] Preferably, in the noise suppression link of the image preprocessing module, in addition to the asymmetric Gaussian filtering algorithm, a bilateral filtering algorithm can also be used. Bilateral filtering can not only consider the spatial distance between pixels like Gaussian filtering, but also adjust the filtering weight according to the difference in pixel values, so that the image edge information can be better preserved while removing noise. When calculating the cross-scale feature correlation matrix, the multi-scale feature extraction module uses the preset scale priority mask matrix in addition to the preset scale priority mask matrix. In addition to the existing generation rules, the mask matrix can also be generated in combination with the semantic segmentation results of the image. For example, for images containing people and scenery, the person area and the scenery area are separated according to semantic segmentation, and different mask weights are set for different areas, so that the model can focus on the characteristics of different areas more specifically. When updating the network hyperparameters, the dynamic optimization control module can use genetic algorithms in addition to multi-objective optimization algorithms. Genetic algorithms find the optimal solution by simulating the process of natural selection and genetic variation. It can optimize in a larger search space and may find a better hyperparameter combination than traditional multi-objective optimization algorithms, thereby improving model performance. In some embodiments, the image preprocessing module includes: The noise suppression unit uses an asymmetric Gaussian filtering algorithm, and its filtering kernel function is:

[0039] in, and are the horizontal and vertical standard deviations, is a learnable offset used to dynamically adjust the filtering direction; It is the coordinate offset relative to the center point of the filter kernel; The edge enhancement unit enhances the high-frequency edge components based on the adaptive Laplacian operator, and its operator weight matrix is:

[0040] The color correction unit adjusts the color deviation through the nonlinear mapping function of the CIELab color space. The mapping function is:

[0041] in, is the dynamic compensation coefficient based on image histogram statistics; , , It is the three-channel value of CIELab color space after correction.

[0042] It should be noted that the image preprocessing module consists of a noise suppression unit, an edge enhancement unit, and a color correction unit. The noise suppression unit uses an asymmetric Gaussian filtering algorithm, which is a method for removing image noise. Its filter kernel function is used to perform weighted calculations on image pixels to achieve the purpose of noise reduction. and Represent the horizontal and vertical standard deviations respectively. These two parameters determine the degree of discreteness of the Gaussian distribution in the horizontal and vertical directions, affecting the strength and range of the filtering. and are learnable offsets that can change the direction of filtering and make filtering operations more targeted. The edge enhancement unit is based on the adaptive Laplacian operator. The Laplacian operator is a second-order differential operator commonly used in image processing to highlight edge information in images. Based on this, the adaptive Laplacian operator can adjust the enhancement effect according to different conditions of the image. Determines the weighting degree of the operator on different pixels. The color correction unit adjusts the color deviation through the nonlinear mapping function of the CIELab color space. The CIELab color space is a device-independent color model that can describe colors more accurately. The nonlinear mapping function is , , These motion compensation coefficients adjust the image color.

[0043] Specifically, in the noise suppression unit, and The value range of will be set according to the characteristics of the image noise. If the image noise distribution in the horizontal and vertical directions is very different, then and The value difference of should also increase accordingly. Generally speaking, and It can take values ​​between 0.5 and 2. and They can be set to 0 at the beginning of training, and their values ​​are adjusted through the back propagation algorithm as the model training progresses. In the edge enhancement unit, The value of is between [0.1,0.3]. When it is close to 0.1, the edge enhancement effect is relatively weak and the change to the image is small; When it is close to 0.3, the edge enhancement effect is more obvious, but some noise may be introduced. In practical applications, it is necessary to determine through experiments. For the color correction unit, calculate , , When calculating these dynamic compensation coefficients, we must first perform histogram statistics on each channel (L, a, b channels) of the image in the CIELab color space. By analyzing the distribution of pixel values ​​in the histogram, we can determine the degree to which each channel needs to be adjusted, and then calculate the corresponding coefficients.

[0044] Preferably, in the noise suppression unit, in addition to using the filter kernel function of the given asymmetric Gaussian filter algorithm, a median filter can also be combined. First, a median filter is used to remove isolated noise points such as salt and pepper noise in the image, and then an asymmetric Gaussian filter is used for further smoothing. This can more effectively remove various types of noise. In the edge enhancement unit, in addition to using the given adaptive Laplacian operator weight matrix, a median filter can also be dynamically adjusted. For example, adjust the value according to the local complexity of the image , for areas with complex textures, appropriately reduce To avoid excessive enhancement and loss of details; for areas with simple textures, increase In the color correction unit, in addition to calculating the dynamic compensation coefficient based on the image histogram statistics, a deep learning model can also be introduced. By training a neural network to learn the color deviation of different images and the corresponding optimal compensation coefficient, color correction can be performed more accurately. In some embodiments, the multi-scale feature extraction module performs the following operations in sequence: Step S1: Extract multi-scale local features through parallel 3×3, 5×5, and 7×7 convolution kernels and output feature map groups ; Step S2: Input the feature maps of each scale into the improved self-attention mechanism and calculate the cross-scale feature correlation matrix:

[0045] in, , are the query and key vectors, is the trainable projection matrix, is the preset scale-priority mask matrix; , It is the feature map output by the i-th and j-th scale layers; Step S3: Generate a global descriptor through a gated fusion mechanism:

[0046] in, is the Sigmoid function, is the fusion weight matrix, Represents element-wise multiplication; The feature map The attention weight matrix.

[0047] It should be noted that the operation of the multi-scale feature extraction module includes three key steps. In step S1, multi-scale local features are extracted by parallel 3×3, 5×5, and 7×7 convolution kernels. The convolution kernel is a matrix used to scan the image in the convolution operation. Convolution kernels of different sizes can capture details and features of the image at different scales. The 3×3 convolution kernel is suitable for extracting smaller local features and focusing on image details; the 5×5 convolution kernel has a larger field of view and can obtain features of a slightly larger range; the 7×7 convolution kernel focuses on overall features of a larger scale. The final output feature map group It is an image representation that contains local features of different scales. In step S2, the feature maps of each scale are input into the improved self-attention mechanism. The self-attention mechanism allows the model to automatically pay attention to the correlation between different parts of the image. Scale-priority mask matrix is pre-set and is used to adjust the association calculation weights between features of different scales. Step S3 generates a global descriptor through a gated fusion mechanism. The gated fusion mechanism uses the Sigmoid function And the fusion weight matrix Perform weighted fusion of feature maps of different scales and perform element-by-element multiplication operations Ensure that each element participates in the fusion calculation, and finally obtain a global descriptor G that can comprehensively reflect the multi-scale characteristics of the image.

[0048] Specifically, in step S1, the parameters of the convolution kernel should be set according to the size and complexity of the image. If a small-sized image with rich details is being processed, the step size of the 3×3 convolution kernel can be set to 1 and the padding to 1, so that the detailed features can be fully extracted; for the 5×5 and 7×7 convolution kernels, the step size can be set to 2 and the padding to 2, which can reduce the amount of calculation while ensuring the integrity of feature extraction. In step S2, the projection matrix can be trained The dimension d and It is usually determined by the number of channels in the feature map. It will be set to a value smaller than the number of feature map channels. For example, when the number of feature map channels is 64, It can be set to 16 to reduce the computational complexity. Scale-priority mask matrix Generated according to the average activation intensity of the feature map. When the average activation intensity is calculated, the size of the feature map Used to determine the range of summation. In step S3, the fusion weight matrix The initialization can be done by random initialization, such as using normal distribution random numbers for initialization, and continuously optimized through the back propagation algorithm during the model training process.

[0049] Preferably, in step S1, in addition to the fixed 3×3, 5×5, and 7×7 convolution kernels, a deformable convolution kernel can also be used. The deformable convolution kernel can adaptively adjust the sampling position of the convolution kernel according to the image content, so as to more accurately extract multi-scale local features. In step S2, position encoding information can be introduced when calculating the cross-scale feature association matrix. Because the position information of pixels in the image is also very important, adding position encoding to the calculation of query and key vectors can enable the model to better capture the relative position relationship of features of different scales in the image. In step S3, in addition to using the Sigmoid function, the gated fusion mechanism can also try to use the ReLU function or the Swish function. The ReLU function is simple to calculate and can effectively alleviate the problem of gradient disappearance. The Swish function combines the advantages of the Sigmoid function and the linear function, which may make the gated fusion effect better. In practical applications, the most suitable function can be selected through experimental comparison. In some embodiments, the dynamic weight allocation algorithm of the adaptive classification recognition module satisfies the iterative formula:

[0050] in, is the exponential moving average term, is a learnable attenuation coefficient that satisfies , is the category confidence difference measure; is the category confidence difference; is the original weight at the current moment; is the EMA weight of the previous moment.

[0051] It should be noted that the adaptive classification and recognition module adopts a dynamic weight allocation algorithm, which satisfies a specific iterative formula. In this formula, Represents the weight of the current moment (step t), which is obtained by the weight of the previous moment , gradient term , category confidence difference measure And the exponential moving average The gradient term is updated by working together. It reflects the rate of change of the loss function on the weights and is used to guide the weights to adjust in the direction of reducing the loss. It reflects the difference between the prediction probabilities of different categories, so as to adjust the impact of weights on predictions of different categories. By weighting the first two moments and It is obtained by weighted averaging, which can make the weight update smoother and reduce fluctuations. are learnable attenuation coefficients, which control the proportion of each part in the weight update, and their sum is 1 to ensure the rationality of the weight update.

[0052] Specifically, in practical applications, the gradient term The loss function between the model prediction result and the true label is calculated and then obtained using the back propagation algorithm. For example, the cross entropy loss function is often used in classification tasks, and its derivative with respect to the weight can be obtained For a learnable decay coefficient , the initial value can be set to 0.25. During the model training process, we can continuously try different values ​​and determine the optimal value based on the performance of the validation set. For example, if we find that the model does not classify well in some categories, we can appropriately increase the coefficients related to these categories (such as Related ), strengthen the focus on these categories. Category confidence difference measure When calculating, the predicted probability is needed And the total number of categories N, predicted probability The prediction output of each category by the model is processed by the Softmax function. The value is generally between 0.9 and 0.99, which determines the weight of the previous moment. The degree of dependence, The closer it is to 1, the more it depends on the weight of the previous moment.

[0053] Preferably, when calculating the class confidence difference metric In addition to the current calculation method, we can also consider introducing semantic similarity information between categories. For example, if there are some categories that are semantically similar, then when calculating the difference metric, we can appropriately reduce the difference weights between them so that the model can more reasonably distinguish these similar categories during classification. In addition to manual adjustment during training or updating through a fixed optimization algorithm, an adaptive update strategy can also be used. For example, according to the performance of the model at different training stages, the update step size of these coefficients can be dynamically adjusted. In the early stage of training, a larger step size can be used to quickly explore the appropriate coefficient value, and in the later stage of training, the step size can be reduced for more precise adjustments. In addition to using the weights at two moments, you can also consider using the weights at multiple moments for weighted averaging, which can better utilize historical information and make weight updates more stable. In some embodiments, the category confidence difference The calculation method is:

[0054] in, For the The predicted probability of the class, is the total number of categories, is the temperature coefficient, which is used to suppress the difference contribution of high confidence categories.

[0055] It should be noted that the category confidence difference is given here The calculation method of . Among them, Representative The predicted probability of a class is the model's prediction that an image belongs to The predicted value of the class probability. The total number of classes It is the number of all preset categories in the image classification task. Temperature coefficient It is a parameter used to adjust the calculation process. Its function is to suppress the difference contribution of high confidence categories, that is, to reduce the proportion of high confidence category differences in the calculation to avoid excessive impact on the results. The whole formula calculates the difference between the predicted probability of each category and the average predicted probability of all categories, and combines the exponential function and the temperature coefficient to finally obtain the category confidence difference. This value will be used in the dynamic weight allocation algorithm in the adaptive classification and recognition module to affect the update of the weight.

[0056] Specifically, in actual calculations, the predicted probability After the model processes the input image, it is obtained through the Softmax function and other methods. The Softmax function converts the original scores output by the model into probability values ​​corresponding to each category, and the sum of these probability values ​​is 1. Total number of categories It is determined according to the specific image classification task. For example, in a task of distinguishing three animals: cats, dogs, and rabbits, . Temperature coefficient The setting needs to be adjusted according to the actual situation. Generally speaking, its value range can be between 0.1-10. When the value is small, the inhibitory effect on the contribution of high confidence category differences is weaker; when When the value is larger, the inhibitory effect is stronger. When

[0057] That is, the average of all category prediction probabilities, and then calculate

[0058] Get the absolute value of the difference between the predicted probability of each category and the average value, and then multiply it by , and finally accumulate all categories.

[0059] Preferably, when calculating the class confidence difference When the temperature coefficient , dynamic adjustment can be adopted. For example, in the early stage of model training, the data distribution may be unstable. Set it to a smaller value to make the model pay more attention to the differences between categories and speed up the convergence. As the training progresses, when the model gradually stabilizes, increase The value of further suppresses the influence of high confidence category differences and makes the model more robust. In addition, when calculating the prediction probability In addition to the conventional Softmax function, you can also consider using an improved Softmax variant, such as Label-SmoothingSoftmax. It smoothes the labels when calculating the predicted probability to avoid the model's prediction of a certain category being too absolute, thus making the calculated More reasonable, which helps to improve the generalization ability of the model. In some embodiments, the multi-objective optimization algorithm of the dynamic optimization control module is defined as follows: Objective function:

[0060] Dynamic constraints:

[0061] in, is the cross entropy classification loss, is the bounding box regression loss, For the prediction distribution With the prior distribution The KL divergence of is the regularization coefficient; It means to find the λ that minimizes the objective function within the range of the variable λ. represents the Euclidean norm of vector λ; express The square of .

[0062] It should be noted that the dynamic optimization control module uses a multi-objective optimization algorithm to adjust the network hyperparameters in real time. This algorithm contains an objective function and dynamic constraints. The objective function is composed of the cross entropy classification loss , bounding box regression loss , KL divergence between predicted distribution P and prior distribution Q The weighted sum of these three parts is subtracted from the regularization term The cross entropy classification loss is used to measure the difference between the category distribution predicted by the model and the true category distribution. The smaller the difference, the better the model performs on the classification task. The bounding box regression loss is used to adjust the position and size of the bounding box in the target detection task so that the predicted bounding box can more accurately frame the target object. KL divergence It is used to measure the difference between two probability distributions, which here means the difference between the predicted distribution and the prior distribution. is the weight coefficient, which is used to adjust the relative importance of these three parts of loss in the objective function. Regularization coefficient Used to control the strength of the regularization term to prevent the model from overfitting. Dynamic constraints on weight coefficients The value range of is restricted and their sum is required to be 1 to ensure the stability and rationality of the model.

[0063] Specifically, in practical applications, the cross entropy classification loss It is usually obtained by calculating the cross entropy between the probability of each category predicted by the model and the true category label (usually one-hot encoded). Bounding box regression loss Different object detection algorithms have different calculation methods, such as the commonly used IoU (intersection over union) loss, which calculates the ratio of the intersection and union of the predicted bounding box and the true bounding box. The closer the ratio is to 1, the better the regression effect. For the predicted distribution P and the prior distribution Q, P is the model's predicted probability distribution for the current sample, and Q can be determined based on the statistical information or experience of the data. The generation method of the prior distribution Q is described in detail in other claims and will not be repeated here. Weight coefficient The initial value of can be set according to the focus of the task. For example, if the classification task is more important, it can be appropriately increased. The initial value of can generally be tried from a value close to the boundary of the value range, such as =0.3, =0.5, =0.2. Regularization coefficient It is usually between 0.001 and 0.1, and the optimal value can be determined through experiments on the validation set.

[0064] Preferably, when calculating the objective function, the weight coefficients can be dynamically adjusted according to the training stage of the model In the early stages of training, the model is not stable, so The values ​​of are relatively balanced, so that the model can learn and adjust classification, regression and distribution matching at the same time; as the training progresses, according to the performance of the model on the validation set, the weight coefficient corresponding to the task with poor performance can be appropriately increased to strengthen the optimization of the task. In addition to the commonly used IoU loss, you can also use GIoU (Generalized IoU) loss, DIoU (Distance - IoU) loss or CIoU (Complete - IoU) loss. These improved loss functions can better consider the relative position and scale relationship between bounding boxes and improve the accuracy of bounding box regression. When determining the prior distribution Q, in addition to the given calculation method, you can also combine domain knowledge and more historical data to generate a more reasonable prior distribution, so that The calculation is more meaningful and helps the model better learn the distribution characteristics of the data. In some embodiments, the dynamic adjustment coefficient The update rule is:

[0065] in, is the comprehensive error, is an adaptive learning rate based on the gradient magnitude:

[0066] It should be noted that the dynamic adjustment coefficient is given here The update rule of . Indicated in The coefficient of the iteration The value of The iteration To update. It is an exponential function, which is used in the formula to weight the gradient information. It is an adaptive learning rate based on the gradient amplitude. It dynamically adjusts the update step size according to the size of the gradient to avoid model instability caused by too large an update step size when the gradient is large, and to prevent the update from being too slow when the gradient is small. is the combined error right The partial derivative of The change of comprehensive error The degree of influence of Depend on It is calculated that Represents different loss items, such as classification loss, regression loss, etc. is the weight coefficient of the corresponding loss term. The whole formula is calculated by Perform exponential weighting and normalize the weighted sum of all coefficients to obtain the updated coefficients , in order to continuously optimize the performance of the model.

[0067] Specifically, in practical applications, calculation Need to be based on the specific loss function To perform the derivation. For example, if is the cross entropy loss function, which can be obtained according to the derivative formula of the cross entropy The partial derivative of . is the initial learning rate, which is a hyperparameter and usually ranges from 0.001 to 0.1. For example, in some simple image classification tasks, you can first try Set it to 0.01 and adjust it according to the convergence of the model training process. hour, represents the square root of the sum of the squares of the gradient magnitudes, which measures the overall size of the gradient. In each iteration, it is necessary to calculate the gradient based on the current loss function value and coefficients. calculate , and then get the adaptive learning rate , and then used to update . In the update When , it is necessary to traverse and calculate all j in the numerator and denominator from 1 to 3 to ensure that each item in the formula is calculated accurately.

[0068] Preferably, when calculating the adaptive learning rate When calculating , you can consider adding a momentum factor. The momentum factor can help the model converge faster and avoid falling into a local optimal solution. For example, when calculating Introducing the momentum term into the formula , the new formula can be modified as

[0069] in, is the adaptive learning rate of the previous iteration, The value is usually between 0.5 and 0.9. When updating, not only the current gradient information is considered, but also the last learning rate is combined to make the update more stable and efficient. When , a random update strategy can be adopted. That is, instead of updating all Instead of updating, a random part is selected Updates are made, which can reduce the amount of calculation and avoid model oscillation caused by updating all parameters at the same time. In large-scale data sets and complex models, this approach may improve training efficiency and model performance. In some embodiments, the feedback learning module performs the following operations: Step F1: Generate weight update according to classification confidence:

[0070] in, are the momentum and noise injection coefficients, is the confidence suppression factor; Step F2: Update parameters using the Nesterov accelerated momentum strategy:

[0071]

[0072] in, is the second-order momentum coefficient, is the dynamic step size:

[0073] It should be noted that the feedback learning module mainly performs two key steps to optimize the model. In step F1, the weight update amount is generated according to the classification confidence. The classification confidence refers to the credibility of the model for the image classification result. Represents the weight update amount, which consists of two parts. and is the momentum and noise injection coefficient, the momentum coefficient It can make the weight update have a certain inertia, accelerate convergence, and inject noise into the coefficient Used to introduce some randomness in the update process, which helps the model escape from the local optimal solution. is the confidence suppression factor, and , its function is to adjust the weight update amount according to the confidence level. The higher the confidence level, the stronger the inhibitory effect on the weight update. It is the gradient of the loss function with respect to the weights, which is used to guide the direction of weight update so that the model is optimized in the direction of reducing the loss. In step F2, the Nesterov accelerated momentum strategy is used to update the parameters. Represents the momentum at the current moment, which combines the momentum at the previous moment , weight update amount And the second-order momentum term . is the weight at the current moment, is the updated weight. is the second-order momentum coefficient, which is used to adjust the influence of the second-order momentum term. is the dynamic step size, which is based on the second norm of the gradient of the loss function and the second norm of the current momentum Dynamic adjustment, is the initial step size, is a small constant used to avoid the denominator being zero.

[0074] Specifically, in step F1, the momentum and noise injection coefficient and The value of needs to be adjusted according to the training situation of the model. Generally speaking, The value range of can be between 0.5 and 0.99. For example, in some simple image classification tasks, you can first Set it to 0.9, observe the convergence speed and stability of the model, and then make fine adjustments; Usually the value is small, such as 0.001-0.01. You can try different values ​​during the training process to see the impact on the generalization ability of the model. If you want to have stronger stability for high confidence classification results, you can is set to a larger value, such as 2 or 3. In step F2, the second-order momentum coefficient Generally, it is selected between 0-0.5, for example, it can be set to 0.2. The value range is usually between 0.001 and 0.1. For complex image recognition tasks, you may need to start with a smaller value, such as 0.001, and then adjust it according to the training effect. Usually take a very small positive number, such as , to ensure the stability of the calculation. When , it is necessary to accurately calculate the second norm of the loss function gradient and the second norm of the current momentum.

[0075] Preferably, in step F1, in addition to the fixed momentum and noise injection coefficient and , they can be adjusted in an adaptive way. For example, as the training progresses, the noise injection coefficient can be gradually reduced. , because in the early stage of training, the model needs more randomness to explore the solution space, and in the later stage, it needs more stable convergence. , can be adjusted dynamically according to the number of samples or the difficulty of samples in different categories. For example, for categories with fewer samples or more difficult classification, appropriately reduce The value of enables the model to update weights more flexibly in these categories. In step F2, when using the Nesterov accelerated momentum strategy, the learning rate decay strategy can be combined. For example, every certain number of training steps, the initial step size is reduced by a certain proportion. , such as every 50 epochs This will allow the model to adjust weights more finely in the later stages of training and improve model performance. In addition to using the second norm of the current momentum , you can also consider adding the average information of historical momentum to make the step size adjustment smoother and more reasonable. In some embodiments, the scale-first mask matrix The generation rules are:

[0076] in, is the average activation intensity of the feature map.

[0077] It should be noted that the scale-priority mask matrix is ​​specified here The generation rule of . Scale-first mask matrix It is used to adjust the correlation calculation weights between features of different scales when calculating the cross-scale feature correlation matrix in the multi-scale feature extraction module. and are feature maps of different scales in the multi-scale feature extraction module, which contain feature information of images at different scales. The scale of the feature map is determined by the average activation intensity and The average activation intensity reflects the overall activity of each pixel in the feature map. The scale Greater than or equal to The scale hour, The value of ;otherwise, The value of This rule makes it possible to highlight the importance of feature maps with larger scales when calculating cross-scale feature associations, and enhance the model's effective fusion of features of different scales.

[0078] Specifically, when calculating the average activation intensity hour, and Represents the feature maps The height and width of the feature map The absolute value of each pixel in Accumulate the rows and divide by the total number of pixels in the feature map , and the average activation strength is In practical applications, the size of the feature map , Determined by the convolution operation and network structure. For example, in a common convolutional neural network, after several layers of convolution and pooling operations, feature maps of different sizes are obtained. ,when When , the logarithmic function used here can be the natural logarithm , or the base 10 logarithm , the natural logarithm is generally more commonly used in mathematical calculations and model optimization. When setting for In actual calculations, since computers cannot accurately represent infinity, we can use a very large number (such as ) is used to approximate it, to ensure that when calculating the cross-scale feature correlation matrix, the influence of the smaller-scale feature map on the larger-scale feature map is significantly suppressed.

[0079] Preferably, when calculating the average activation intensity When , we can consider setting different weights for pixels in different positions. For example, we can give higher weights to pixels in the center of the image because the center often contains more critical information. We can define a weight matrix , the formula for calculating the average activation strength is modified to

[0080] For the scale-first mask matrix In addition to being based on the average activation intensity, the generation rules of can also be combined with the semantic information of the image. For example, for images with obvious hierarchical structures, the semantic importance of different levels can be adjusted. For feature maps corresponding to semantically more important layers, even if their average activation strength is smaller, their Value (if not set to , but is set to a relatively large but finite value), so that it can still play a certain role in the cross-scale feature association calculation, thereby better integrating semantic features of different scales and improving the performance of the model. In some embodiments, the prior distribution The generation method is:

[0081] in, To distribute the smooth temperature, is the number of samples in the current batch, is the indicator function.

[0082] It should be noted that the method for generating the prior distribution Q is given here. The prior distribution Q is a probability distribution determined based on certain assumptions and data statistics before model training, which is used to provide initial information for the model. Represents the probability value of the kth class in the prior distribution Q. is an intermediate variable, which is obtained by counting the number of samples belonging to the kth class in the current batch of samples. T is the distribution smoothing temperature, which is used to adjust the smoothness of the prior distribution and affect the difference between the probabilities of each category. B is the number of samples in the current batch, that is, the number of samples used in each training. Indicator function It is used to determine whether sample b belongs to the kth category. If it does, the function value is 1, otherwise it is 0. Perform exponential transformation and normalization to obtain the probability values ​​of each category in the prior distribution Q.

[0083] Specifically, in actual calculations, the number of samples in the current batch B is set according to the batch size during model training. For example, when using a small batch gradient descent algorithm for training, B usually takes values ​​between 16 and 256, and common values ​​are 32, 64, 128, etc. The specific value needs to be determined based on the size of the data set, hardware resources, and the complexity of the model. The distribution smoothing temperature T is generally a real number greater than 0, and its value range is between 0.1 and 10. When the value of T is small, the probability value difference of the prior distribution is relatively large, and the model distinguishes the categories more clearly; when the value of T is large, the prior distribution is smoother, and the probability values ​​of the categories are closer. In the calculation When , it is necessary to traverse all B samples in the current batch and determine the label of each sample b. Is it equal to k? If so, is 1, otherwise 0, then these values ​​are summed and divided by B. Finally, the calculated Substitute into the formula to calculate , perform this calculation for all categories to obtain the complete prior distribution Q.

[0084] Preferably, in calculating When , we can not only limit ourselves to the samples of the current batch, but also combine the sample information of multiple batches. For example, we can maintain a sliding window that contains the samples of the most recent batches. In this way, we can use more historical data to count the number of samples belonging to the kth class, so that The calculation is more accurate, and a more reasonable prior distribution is obtained. For the distribution smoothing temperature T, a dynamic adjustment strategy can be adopted. In the early stage of model training, since the model has little knowledge of the data, T can be set to a larger value to make the prior distribution smoother and give the model more room for exploration; as the training progresses, the model gradually learns the characteristics of the data. At this time, the value of T can be gradually reduced to make the prior distribution more focused on the true distribution of the data. In addition, when calculating In addition to using the exponential function for transformation, you can also try other nonlinear transformation functions, such as the Sigmoid function or the variant of the Softmax function. By comparing the impact of different functions on model performance through experiments, you can choose the optimal transformation method to generate the prior distribution Q to improve the overall performance of the model. The above-mentioned embodiments of the present invention have the following beneficial effects: the image recognition and processing system of the present invention can improve the accuracy and efficiency of image recognition through modular design and collaborative working mechanism. The image preprocessing module adopts an asymmetric Gaussian filtering algorithm, which can dynamically adjust the filtering direction according to the horizontal and vertical directions of the image, effectively suppress noise while retaining image details; the edge enhancement unit uses an adaptive Laplace operator to enhance high-frequency edge components to further highlight the key features of the image; the color correction unit adjusts the color deviation through a nonlinear mapping function to ensure the color consistency of the image under different lighting conditions. The multi-scale feature extraction module extracts multi-scale local features through parallel convolution kernels, and calculates the cross-scale feature association matrix in combination with an improved self-attention mechanism, which can fully capture the global and local features of the image and generate a more representative global descriptor. The adaptive classification recognition module is based on a dynamic weight allocation algorithm, which can flexibly adjust the weight allocation according to the difference in category confidence, thereby improving the accuracy and reliability of classification. The dynamic optimization control module adjusts the network hyperparameters in real time through a multi-objective optimization algorithm, and balances the classification loss, bounding box regression loss and KL divergence of the predicted distribution while meeting dynamic constraints, ensuring that the system can maintain optimal performance in different tasks and scenarios.

[0085] The feedback learning module dynamically updates the network weights according to the classification confidence score, and adopts the Nesterov accelerated momentum strategy and adaptive learning rate to quickly adjust the model parameters, further improving the adaptability and generalization ability of the system. The generation rule of the scale-priority mask matrix is ​​based on the average activation intensity of the feature map, which can dynamically adjust the priority of the feature map and further optimize the feature extraction process. The generation method of the prior distribution can flexibly adjust the shape of the prior distribution through smoothing temperature parameters and indicator functions, and enhance the adaptability of the model to different categories of distribution. Overall, the system of the present invention can effectively cope with complex image scenes, improve recognition accuracy, and at the same time have good dynamic adjustment capabilities and real-time feedback learning mechanisms, so that it has higher practicality and reliability in practical applications.

[0086] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0087] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. An image recognition and processing system based on deep learning, characterized in that: Includes the following modules that work together: An image acquisition module, used for acquiring raw image data; The image preprocessing module is connected to the image acquisition module to perform noise suppression, edge enhancement and color correction on the original image; The multi-scale feature extraction module is connected to the image preprocessing module to extract local and global multi-dimensional features through cascaded convolutional layers and self-attention mechanism; The adaptive classification and recognition module is connected to the multi-scale feature extraction module to classify features into multiple categories based on a dynamic weight allocation algorithm; The dynamic optimization control module is connected to the adaptive classification and recognition module to adjust the network hyperparameters in real time through a multi-objective optimization algorithm; The result output module is connected to the dynamic optimization control module to output the classification results and confidence scores; The feedback learning module is connected to the result output module, updates the network weights according to the confidence score, and is synchronized to the multi-scale feature extraction module and the adaptive classification and recognition module.

2. The image recognition and processing system according to claim 1, characterized in that: The image preprocessing module comprises: The noise suppression unit uses an asymmetric Gaussian filtering algorithm, and its filtering kernel function is: in, and are the horizontal and vertical standard deviations, respectively. is a learnable offset used to dynamically adjust the filtering direction; It is the coordinate offset relative to the center point of the filter kernel; The edge enhancement unit enhances the high-frequency edge components based on the adaptive Laplacian operator, and its operator weight matrix is: The color correction unit adjusts the color deviation through the nonlinear mapping function of the CIELab color space. The mapping function is: in, is the dynamic compensation coefficient based on image histogram statistics; , , It is the three-channel value of CIELab color space after correction.

3. The image recognition and processing system according to claim 1, characterized in that: The multi-scale feature extraction module performs the following operations in sequence: Step S1: Extract multi-scale local features through parallel 3×3, 5×5, and 7×7 convolution kernels and output feature map groups ; Step S2: Input the feature maps of each scale into the improved self-attention mechanism and calculate the cross-scale feature correlation matrix: in, , are the query and key vectors, is the trainable projection matrix, is the preset scale-priority mask matrix; , It is the feature map output by the i-th and j-th scale layers; Step S3: Generate a global descriptor through a gated fusion mechanism: in, is the Sigmoid function, is the fusion weight matrix, Represents element-wise multiplication; The feature map The attention weight matrix, For the The gating weight vector corresponding to the scale feature map.

4. The image recognition and processing system according to claim 1, characterized in that: The dynamic weight allocation algorithm of the adaptive classification recognition module satisfies the iterative formula: in, is the exponential moving average term, is a learnable attenuation coefficient that satisfies , is the category confidence difference measure; is the category confidence difference; is the original weight at the current moment; is the EMA weight of the previous moment.

5. The image recognition and processing system according to claim 4, characterized in that: The class confidence difference The calculation method is: in, For the The predicted probability of the class, is the total number of categories, is the temperature coefficient, which is used to suppress the difference contribution of high confidence categories, is the predicted probability of the mth class.

6. The image recognition and processing system according to claim 1, characterized in that: The multi-objective optimization algorithm of the dynamic optimization control module is defined as follows: Objective function: Dynamic constraints: in, is the cross entropy classification loss, is the bounding box regression loss, For the prediction distribution With the prior distribution The KL divergence of is the regularization coefficient; It means to find the λ that minimizes the objective function within the range of the variable λ. represents the Euclidean norm of vector λ; express The square of .

7. The image recognition and processing system according to claim 6, characterized in that: The dynamic constraint condition dynamically adjusts the coefficient The update rule is: in, is the comprehensive error, is the regularization coefficient of the jth loss term, is an adaptive learning rate based on the gradient magnitude: ； in, is the initial learning rate baseline value.

8. The image recognition and processing system according to claim 1, characterized in that: The feedback learning module performs the following operations: Step F1: Generate weight update based on classification confidence: in, are the momentum and noise injection coefficients, is the confidence suppression factor; Step F2: Update parameters using the Nesterov accelerated momentum strategy: in, is the second-order momentum coefficient, is the momentum accumulation of the tth iteration; is the staged momentum cache, is the dynamic step size: 。 9. The image recognition and processing system according to claim 3, characterized in that: The scale-first mask matrix The generation rules are: in, For the The average activation intensity of the feature map of the feature map of each scale, For the The average activation intensity of the feature maps at each scale.

10. The image recognition and processing system according to claim 6, characterized in that: The prior distribution The generation method is: in, To distribute the smooth temperature, is the number of samples in the current batch, is the indicator function, For the Prior statistics for the categories, For the Prior statistics for the categories, For the The true category labels of samples.

Images