Convolutional neural network optimization method based on autonomous learning

Abstract

The invention relates to the technical field of computer vision and artificial intelligence, in particular to a convolutional neural network optimization method based on autonomous learning, and the method comprises the steps: obtaining original image data; inputting the original image data into the convolutional neural network for forward propagation; in the forward propagation process, recording feature maps extracted from adjacent hierarchies; inputting the second feature map into a feedback prediction channel, and outputting a prediction feature map; calculating a difference value between the first feature map and the predicted feature map, and generating an error signal; inputting the error signal into a modulator, and outputting a modulated feature map; replacing the original first feature map with the modulated feature map, and performing forward propagation calculation from the first layer again; and applying the final identification result output by the network to decision control of the intelligent driving system. According to the method, the influence of input disturbance or interlayer information deviation on feature expression is effectively counteracted, so that the features output by the network are more stable.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a method for optimizing convolutional neural networks based on autonomous learning. Background Technology

[0002] Convolutional Neural Networks (CNNs) are one of the core models in the field of deep learning, widely used for their outstanding performance in computer vision tasks such as image recognition and object detection. Classic CNN models, such as AlexNet, VGGNet, GoogLeNet, and ResNet, typically employ a hierarchical feedforward structure. In this structure, input data (such as images) passes sequentially through convolutional layers, activation function layers (such as ReLU), and pooling layers, with information flowing unidirectionally from bottom to top. As the number of layers increases, the extracted features gradually transform from concrete details such as edges and textures at lower levels into abstract semantic information at higher levels.

[0003] The advantage of this feedforward structure lies in its ability to efficiently learn translation-invariant feature representations from images through "local receptive fields" and "parameter sharing" mechanisms. However, this structure also has inherent limitations. First, the unidirectional nature of the information flow means that high-level semantic features are difficult to effectively feed back and guide the optimization of lower-level features. After feature maps undergo downsampling operations such as pooling, some spatial details are inevitably lost, and subsequent layers cannot recover these lost details. Second, in complex real-world applications, such as the visual perception system for autonomous driving, models often need to handle image quality degradation caused by factors such as rain blur, low light at night, target occlusion, or motion blur. When faced with these input perturbations, existing feedforward CNNs may already have biases in the features extracted from the lower layers. These biases are amplified layer by layer during forward propagation, ultimately leading to errors in high-level semantic judgments, such as missed detections or misidentifications of pedestrians or vehicles. Summary of the Invention

[0004] The purpose of this invention is to enable specific layers of a CNN to receive feedback information from higher layers and compare it with their own features and adjust for errors, thereby effectively offsetting the impact of input disturbances (such as noise, changes in illumination) or inter-layer information deviations on feature expression, making the network output features more stable, reducing the risk of feature distortion caused by external interference, significantly enhancing overall robustness, and improving the stability and anti-interference ability of feature extraction.

[0005] In a first aspect, embodiments of the present invention provide a convolutional neural network optimization method based on autonomous learning, the method comprising: Acquire raw image data containing pedestrians, vehicles, or traffic signs captured by vehicle-mounted cameras; The original image data is input into a convolutional neural network for forward propagation. The network contains multiple sequentially connected layers, wherein the spatial resolution of the feature map decreases with each layer as the depth increases. During the forward propagation process, feature maps extracted from adjacent layers are recorded, wherein the feature maps extracted from adjacent layers include those from the previous layer with higher spatial resolution. The first feature map, and from the immediately following layer Then, the next level with lower spatial resolution +1 second feature map; The second feature map is input into a feedback prediction channel, which includes an upsampling layer and a learnable network module for detail enhancement. Through the transformation of the feedback prediction channel, the output is similar in spatial size and number of channels to the first feature map. Figure 1 A consistent predictive feature map that is enhanced in spatial detail; Calculate the difference between the first feature map and the predicted feature map to generate an error signal; The error signal is input to a modulator, which generates a modulated signal based on the error signal, adaptively adjusts the first feature map, and outputs a modulated feature map. The original first feature map is replaced with the modulated feature map, and the process is repeated from the first feature map. Layered forward propagation calculation; The final network output recognition results are used for decision-making and control in the intelligent driving system.

[0006] Optionally, the learnable network module for detail enhancement is a feature fusion module; The feedback prediction channel generates the prediction feature map through the following steps: The second feature map is upsampled to initially align its spatial size and number of channels with the first feature map. The upsampled features are compared with those from the convolutional neural network. The detailed features from the shallower hierarchical jumps are then fused together; The fused features are processed through at least one convolutional layer to output the predicted feature map.

[0007] Optionally, the learnable network module for detail enhancement is an attention mechanism module; The feedback prediction channel generates the prediction feature map through the following steps: The second feature map is upsampled to initially align its spatial size and number of channels with the first feature map. The upsampled features are input into the attention mechanism module to generate feature weights for enhancing key details; The upsampled features are weighted using the feature weights to output the predicted feature map.

[0008] Optionally, the modulator generates a modulation signal based on the error signal and adaptively adjusts the first feature map, specifically as follows: Channel-level modulation includes: generating a set of channel scaling coefficients and channel bias coefficients based on the error signal, and scaling and biasing each channel of the first feature map; or, Feature map-level modulation includes: generating a feature correction matrix with the same shape as the first feature map based on the error signal, and adding the matrix to the first feature map; or, Convolutional kernel-level modulation includes: generating a convolutional kernel adjustment amount based on the error signal, used to adjust the kernel level... The weight parameters of the convolution kernel are adjusted, and the first feature map is recalculated using the adjusted convolution kernel.

[0009] Optionally, based on the computing power of the deployed device, the implementation method of the modulator can be dynamically selected, including: When computational efficiency is a priority, the channel-level modulation with the lightest computational load is selected; When fine-tuning of the spatial details of the feature map is required, the feature map-level modulation with a computational load higher than channel-level modulation but lower than convolution kernel-level modulation is selected. When long-term optimization of core network parameters is required and computational resources allow, the convolutional kernel-level modulation with the heaviest computational load should be selected.

[0010] Optionally, the convolutional neural network is configured to include multiple feedback self-learning structures; Each of the aforementioned feedback self-learning structures is based on a hierarchical pair ( , +1) Establish; the hierarchy includes the previous level with higher spatial resolution. and immediately following the hierarchy Then, the next level with lower spatial resolution +1; Multiple feedback self-learning structures together constitute a multi-layered feedback self-learning system.

[0011] Optionally, the previous layer with higher spatial resolution , is the layer in the convolutional neural network whose spatial resolution of the first output feature map is lower than that of the input image.

[0012] Secondly, embodiments of the present invention provide a convolutional neural network optimization device based on autonomous learning, the device comprising: The image data acquisition module is used to acquire raw image data containing pedestrians, vehicles, or traffic signs captured by the vehicle-mounted camera. An image data input module is used to input the original image data into a convolutional neural network for forward propagation. The network contains multiple sequentially connected layers, wherein the spatial resolution of the feature map decreases layer by layer as the layer deepens. The feature map recording module is used to record feature maps extracted from adjacent layers during the forward propagation process, wherein the feature maps extracted from adjacent layers include those from the previous layer with higher spatial resolution. The first feature map, and from the immediately following layer Then, the next level with lower spatial resolution +1 second feature map; A predictive feature map generation module is used to input the second feature map into a feedback prediction channel. The feedback prediction channel includes an upsampling layer and a learnable network module for detail enhancement. Through the transformation of the feedback prediction channel, the output is similar in spatial size and number of channels to the first feature map. Figure 1 A consistent predictive feature map that is enhanced in spatial detail; An error signal calculation module is used to calculate the difference between the first feature map and the predicted feature map, and generate an error signal; The modulated feature map generation module is used to input the error signal to a modulator, which generates a modulation signal based on the error signal, adaptively adjusts the first feature map, and outputs the modulated feature map. The feature map adjustment module is used to replace the original first feature map with the modulated feature map and re-perform the self-adjustment. Layered forward propagation calculation; The recognition result output module is used to apply the final recognition results output by the network to the decision control of the intelligent driving system.

[0013] Thirdly, embodiments of the present invention provide an electronic device, including: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method described in the first aspect.

[0014] Fourthly, according to an embodiment of the present invention, a computer-readable storage medium is provided thereon storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0015] This invention constructs a multi-level, closed-loop feedback correction system by introducing an inter-layer feedback self-learning mechanism consisting of a feedback prediction channel and a modulator into the network. This mechanism utilizes high-level abstract semantic information as a guiding signal, refines the reconstruction of low-level features through a learnable feedback prediction channel, and dynamically corrects deviations in the low-level feature representation using a flexible modulator. This innovative design effectively overcomes the problems of feature distortion, detail loss, and semantic bias faced by traditional feedforward convolutional neural networks in complex scenarios, significantly improving the model's feature robustness and detail preservation capabilities under various interference environments such as sudden changes in illumination, motion blur, and rain / snow. This method not only effectively reduces missed and false detections of key targets such as pedestrians and vehicles but also improves the stability of inference while maintaining high accuracy. Ultimately, it provides more reliable and accurate recognition results for the perception module of intelligent driving systems, laying a solid technical foundation for vehicle decision-making and control, and significantly improving driving safety. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall architecture provided for an embodiment of the present invention; Figure 2 A schematic diagram illustrating three modulation schemes provided in embodiments of the present invention; Figure 3 This is a flowchart of a convolutional neural network optimization method based on autonomous learning, provided as an embodiment of the present invention. Detailed Implementation

[0017] The present invention will be described in detail below through embodiments.

[0018] Convolutional Neural Networks (CNNs), as a crucial model in deep learning, play a key role in fields such as image recognition and visual analysis. They stand out due to two core design features: first, the local receptive field mechanism allows the network to focus on features in local image regions without needing to consider globally redundant information; second, the parameter-sharing strategy allows neurons in the same convolutional layer to share a single set of parameters, significantly reducing network size. These two features together contribute to the high efficiency and strong feature extraction capabilities of CNNs when processing high-dimensional data such as images and speech.

[0019] The reason convolutional neural networks (CNNs) have become a core tool in computer vision is that they precisely meet the needs of visual tasks: they efficiently capture detailed features such as edges and textures through local receptive fields, significantly reduce computational burden through parameter sharing, stably identify targets at different locations due to translation invariance, and achieve deep feature mining from pixels to semantics through multi-layered structures. These capabilities allow them to automatically complete feature engineering without human intervention, easily handle large-scale data, and exhibit outstanding generalization performance. Furthermore, their flexible and scalable architecture adapts to various tasks such as image classification, object detection, and semantic segmentation, making their applications extremely widespread.

[0020] Traditional CNNs are feedforward structures: information is passed from the input layer through layers of convolution, non-linear activation, pooling, etc., gradually moving to higher layers. With each layer, the features become more abstract, but fine-grained information from lower layers (such as spatial details) may be lost. The deeper the network, the more likely gradient propagation will decay or explode (especially in very deep networks), making training difficult.

[0021] In core intelligent driving scenarios such as vehicle camera recognition of pedestrians, vehicles, and road signs, this invention improves the stability and robustness of information through optimized network design. Specifically, after receiving "feedback prediction / semantic information" from higher layers, a layer of the CNN compares its current features with its own and calculates errors, adaptively adjusting the layer (activation or parameters) accordingly. This improves representational robustness and detail preservation. This design effectively addresses complex interferences in intelligent driving, such as blurred vision in rainy weather, insufficient lighting at night, and mixed traffic of multiple vehicles and pedestrians. It reduces missed pedestrian detection, misidentification of vehicles, and misinterpretation of signs, while accurately preserving key details, providing more reliable perceptual basis for decision-making and further enhancing driving safety.

[0022] To ensure the plan is clearly described, the following will combine... Figure 1 The overall technical solution of this invention is described in detail. This embodiment takes pedestrian recognition in intelligent driving as an example, but this method is also applicable to the recognition of other targets such as vehicles and traffic signs. Specifically, it may include the following steps: 1. Network initialization and data preparation.

[0023] First, a multi-layered convolutional neural network (backbone) is constructed, such as using a ResNet or VGG architecture. This network consists of sequentially connected convolutional layers, activation function layers (such as ReLU), and pooling layers. Then, raw road images captured by an onboard camera are used as input data.

[0024] 2. Select the feedback level pair and perform forward propagation.

[0025] As attached Figure 1As shown, in this embodiment, we choose to implement a feedback self-learning mechanism on two adjacent layer pairs of the network, i.e., layer pairs ( , +1) and hierarchy pairs ( +1, +2). This multi-layered feedback structure can form a more refined closed-loop correction system.

[0026] Forward propagation to hierarchy : Input image after the first The network processing before and after the layer (usually including convolution → activation → pooling) yields the first feature map output by that layer. This feature map has a high spatial resolution, preserving rich spatial details such as edges and textures.

[0027] Forward propagation to hierarchy +1: First feature map Continue to spread forward, after the first... The +1 layer processing (convolution → activation → pooling) yields the second feature map. At this point, the second feature map Spatial resolution compared to It has been reduced, but contains more advanced semantic information.

[0028] Forward propagation to hierarchy +2: Second feature map Continue to spread forward, after the first... The +2 layer processing yields the feature map. Its spatial resolution is lower, and its semantic information is the most abstract.

[0029] 3. Feedback prediction process (see attached document) Figure 1 In G (Module) The core of this step is to use high-level semantic information to predict and enhance low-level features.

[0030] For hierarchy pairs ( , +1): High-level features Input to feedback prediction channel The predictor first upsamples the data through an upsampling layer (such as transposed convolution or bilinear interpolation). Spatial dimensions and number of channels Consistent with the previous approach, a learnable network module for detail enhancement (e.g., an "upsampling + feature fusion" collaborative structure, an "upsampling + attention mechanism," etc.) is used to fuse the upsampled features with detailed features from shallower skip connections in the network, followed by fine-tuning through convolutional layers. Finally, the feedback prediction channel outputs a feature that is consistent with the previous approach in terms of spatial size and number of channels. Fully aligned, and with richer and more accurate predicted feature maps. .

[0031] For hierarchy pairs ( +1, +2): Similarly, the features Input to feedback prediction channel Through its internal upsampling and detail enhancement modules, the output is similar to... Aligned Predictive Features .

[0032] 4. Error Calculation and Modulation Process (see attached document) Figure 1 In H (Module) This step achieves adaptive optimization of the layer by calculating the error and modulating the features.

[0033] At the level Calculate the first feature map With predicted feature map The difference between them generates an error signal. = - This error signal reflects the deviation between high-level semantic predictions and the actual features at the lower level.

[0034] Error signal Input to modulator ( There are typically three modulation methods, which will be described in detail in the following examples. In this example, the second modulation method (feature map-level modulation) will be used as an example. This modulator is a lightweight convolutional subnetwork that will... As input, after nonlinear transformation, the output is a... Feature correction matrix of the same shape .

[0035] Next, feature modulation is performed: the correction matrix is ​​added to the original feature map to obtain the modulated feature map. = + This operation is equivalent to... Targeted enhancements and repairs were performed on areas where there were discrepancies or lost details.

[0036] At the level +1: The process is exactly the same. Calculation = - and through the modulator ( Generate modulated features .

[0037] in, Figure 1 In It refers to three modulation methods. The meaning is a broad concept and can be understood as a modulator, without referring to any specific modulator.

[0038] 5. Re-propagate and output.

[0039] Using the modulated feature map Replace the original feature map Then, from the first The layer begins to re-execute the forward propagation computation, i.e., using the optimized version. To calculate the first The outputs of layer +1 and subsequent layers are used. The recalculated features are propagated and eventually pass through a fully connected layer and a classifier to output the recognition results of targets such as pedestrians and vehicles in the image. This recognition result is then sent to the decision control module of the intelligent driving system to implement subsequent functions such as vehicle warning, planning, or control.

[0040] 6. Training Process During end-to-end training of the entire network, the feedback prediction channel G and modulator H The parameters will be optimized along with the main network using the gradient descent algorithm. The loss function typically employs a combination of cross-entropy loss and feature reconstruction loss to simultaneously ensure classification accuracy and feature quality.

[0041] Through the above combination Figure 1 As can be seen from the detailed description, the present invention introduces a feedback self-learning loop consisting of a feedback prediction channel and a modulator at multiple levels, enabling the network to dynamically correct and enhance the expression of low-level features using high-level semantic information, thereby significantly improving the robustness and accuracy of visual perception in complex driving environments.

[0042] In the above Figure 1 Based on the overall architecture shown, one of the core innovations of this invention lies in the modulator. H Flexible design and implementation. For example... Figure 2 As shown, modulator ( For error signals The following three different mechanisms can be used to handle this, in order to adapt to different computing resource constraints and performance requirements.

[0043] After the aforementioned steps, the first [item] has been obtained. Original feature map of the layer And the error signal obtained through feedback prediction and error calculation. The modulator's goal is to utilize... Generate a modulated signal, for Optimize.

[0044] Modulation Mechanism 1: Channel-level Modulation This mechanism achieves fast, lightweight adjustments by globally scaling and biasing each channel of the feature map.

[0045] (1) Error information compression: compression of error signals (Its shape is [C,H,W], where C is the number of channels, and H and W are the height and width.) Perform a global average pooling operation along the spatial dimension to compress it into a C-dimensional vector. This vector serves as a channel-level global error descriptor, summarizing the average error level of each channel.

[0046] (2) Generating modulation parameters: Input this C-dimensional error descriptor into a small parameter generation network consisting of two fully connected layers. The network outputs two C-dimensional vectors: channel scaling coefficients. and channel bias coefficient .

[0047] (3) Application modulation: The generated and Acting on the original feature map To correct The calculation process is as follows: .

[0048] in This indicates channel-by-channel multiplication. This operation is equivalent to adaptively enhancing or suppressing the characteristic responses of different channels based on the error.

[0049] This method has extremely low computational overhead, making it ideal for edge devices with limited computing power. It can quickly correct intensity deviations across the entire feature channel, achieving coarse-grained and highly efficient adjustment of feature distribution.

[0050] Modulation Mechanism 2: Feature-map-level Modulation This mechanism aims to refine the spatial details of the feature map at the pixel level.

[0051] (1) Error feature extraction: extracting error signals Input a lightweight convolutional subnetwork. This subnetwork typically consists of 1-2 convolutional layers, and uses activation functions such as ReLU to introduce nonlinearity.

[0052] (2) Generating the correction matrix: The convolutional sub-network above finally outputs a matrix that is consistent with the original feature map. Feature correction matrices with identical spatial dimensions and number of channels Each element of this matrix represents the amount of fine adjustment required for the corresponding spatial location and channel.

[0053] (3) Applying modulation: The generated correction matrix Compared with the original feature map By directly adding element by element, the modulated feature map is obtained: = +

[0054] This method preserves spatial information, enabling the enhancement or repair of details in specific regions of the feature map, such as strengthening blurred pedestrian edges. It has a moderate computational cost and provides excellent performance in scenarios requiring fine-grained correction.

[0055] Modulation Mechanism 3: Kernel-level Modulation This mechanism is the most fundamental optimization method. It directly adjusts the core parameters of the network—the convolution kernel weights—to achieve long-term, fundamental improvements in feature extraction capabilities.

[0056] (1) Error information compression and encoding: First, the error signal is compressed and encoded. Adaptive pooling is used to reduce the dimensionality and flatten it into a one-dimensional vector. This vector is then encoded through several fully connected layers to capture the complex mapping between the error and the adjustment of the convolutional kernel weights.

[0057] (2) Generating convolutional kernel adjustment: The encoded vector passes through a final fully connected layer to adjust the error signal. Generated through simple calculations This refers to the "kernel adjustment," which has the same shape as the original kernel.

[0058] (3) Update parameters and recalculate: Add this adjustment to the original convolution kernel (or apply it to the original convolution kernel in other ways) to obtain a new and better convolution kernel.

[0059] This is the key difference: it doesn't generate a new feature map, but rather updates the network parameters. Subsequently, the updated convolutional kernels must be used to recursively apply the changes to the first feature map. The original input of the layer is re-convolved to compute a completely new, modulated feature map. .

[0060] This method directly optimizes the feature extractor itself, resulting in the most lasting and fundamental effects, significantly improving the network's inherent capabilities. However, it incurs the highest computational and memory costs, typically requiring a training environment with ample computing power, or for offline optimization of key network layers.

[0061] The modulator of this invention provides a variety of solutions, ranging from lightweight to weight-saving, and from temporary correction to permanent optimization. In practical applications, the most suitable modulation mechanism can be flexibly configured for different layers in the network according to the real-time requirements and accuracy needs of different modules of the intelligent driving system (such as forward collision warning and traffic sign recognition), thereby achieving the best balance between performance and efficiency.

[0062] After a detailed description of the overall technical solution and three modulation methods of the embodiments of the present invention, the following will provide a detailed description of a convolutional neural network optimization method based on autonomous learning provided by the embodiments of the present invention.

[0063] like Figure 3 As shown in the figure, an embodiment of the present invention provides a convolutional neural network optimization method based on autonomous learning, which may include the following steps: S310 acquires raw image data containing pedestrians, vehicles, or traffic signs captured by an onboard camera.

[0064] In this step, the system captures real-time images of the road environment using image sensors deployed on the vehicle (such as front-facing cameras and surround-view cameras). This raw image data forms the basic input to the intelligent driving perception system, and its content typically includes complex scene information, such as roads, pedestrians, other vehicles, traffic lights and signs, etc.

[0065] S320 inputs the original image data into the convolutional neural network for forward propagation. The network contains multiple sequentially connected layers, in which the spatial resolution of the feature maps decreases layer by layer as the layer deepens.

[0066] The raw image obtained in step S310 is input into a predefined base convolutional neural network (such as VGG, ResNet, or other backbone networks). This network consists of multiple hierarchical modules connected sequentially, each typically containing convolution, non-linear activation (such as ReLU), and pooling operations. As the data propagates forward through the network layer by layer, the spatial dimensions (height and width) of the feature maps are gradually reduced through pooling or stride convolution, while the number of feature channels increases, and the semantic abstraction level continuously improves.

[0067] S330, during the forward propagation process, records feature maps extracted from adjacent layers. These feature maps include those from the previous layer with higher spatial resolution. The first feature map, and from the immediately following layer Then, the next level with lower spatial resolution +1 is the second feature map.

[0068] During the forward propagation of the network, the system selects one or more adjacent hierarchical pairs ( , +1), and cache the feature maps from the outputs of these two levels. Specifically: First feature map From the hierarchy Output. This layer is usually located in the lower to middle layers of the network. Its output feature map has a high spatial resolution and preserves rich spatial details such as pedestrian outlines, vehicle edges, and textures.

[0069] Second feature map From the immediate adjacent, deeper levels +1 output. Due to further pooling or convolutional downsampling, its spatial resolution is lower than... However, it contains higher-level and more abstract semantic information (such as "part of a vehicle" or "humanoid").

[0070] As one implementation of this invention, the previous layer with higher spatial resolution... The first layer in a convolutional neural network whose output feature map has a spatial resolution lower than that of the input image. This definition provides an objective, clear, and efficient starting point for the core feedback mechanism. The determination method will be explained in detail below, taking into account the network structure.

[0071] 1. Network structure analysis.

[0072] Consider a typical convolutional neural network architecture, whose initial part is usually as follows: Input image → Convolutional layer 1 (Conv1) → Activation function (ReLU) → Pooling layer (Pool1) → Convolutional layer 2 (Conv2) Assume the spatial resolution of the input image is [H, W] (e.g., [224, 224]). Convolutional layer 1 (Conv1) typically uses a small stride (e.g., stride=1) and padding (padding=1), and its output feature map size usually remains unchanged or changes very little, with the spatial resolution still considered to be at the [H, W] level. Pooling layer 1 (Pool1) is a typical spatial downsampling operation. Taking max pooling with a stride of 2 as an example, it halves the size of the input feature map in both the width and height dimensions. Therefore, the spatial resolution of its output feature map is explicitly reduced to [H / 2, W / 2].

[0073] 2. Hierarchy The determination.

[0074] According to the above definition, the layer whose spatial resolution of the first output feature map is lower than that of the input image is the layer immediately following the first spatial downsampling operation. In this example, pooling layer 1 (Pool1) is the first to perform spatial downsampling, reducing the feature map resolution from [H,W] to [H / 2,W / 2]. Therefore, the subsequent convolutional layer 2 (Conv2) is the previous layer as defined in this implementation. The layers following convolutional layer 2 (such as the immediately following pooling layer 2 or convolutional layer 3) naturally become the next layer. +1.

[0075] Choosing this level as the feedback starting point has the following significant beneficial effects: 1. It achieves the optimal balance between detail and semantics. The feature maps at this level (such as Conv2) have just undergone their first downsampling and still retain very rich spatial detail information, far exceeding that of features in deeper layers of the network. At the same time, they have already undergone a layer of convolution and non-linear activation, possessing higher-level semantic information than the original pixels. Introducing feedback here allows higher-level information to most effectively act on these "semi-processed" features that still retain detail, achieving accurate correction.

[0076] 2. It provides an objective and unambiguous execution standard. This definition avoids vague expressions such as "lower and middle layers." For any given CNN architecture, those skilled in the art can uniquely and accurately determine the layer by simply analyzing the first change in feature map size during the forward propagation. The location of the patent ensures the repeatability of the solution and the stability of the patent.

[0077] 3. Ensures high efficiency of the feedback mechanism. Introducing feedback from the beginning of the network allows for early correction of feature biases, preventing errors from propagating and amplifying in deeper layers. Simultaneously, since the resolution of this layer has not yet undergone multiple reductions, the calculation of feedback prediction channels is more efficient. G and modulatorH The cost is relatively controllable, achieving an optimal balance between performance and overhead.

[0078] This embodiment uses hierarchical levels. Objectively defined as the level where the resolution first decreases in the network, it provides a significant and easy-to-implement starting point for the entire feedback self-learning system, and is one of the key design features that enable the invention to operate efficiently.

[0079] S340, the second feature map is input to the feedback prediction channel. The feedback prediction channel contains an upsampling layer and a learnable network module for detail enhancement. Through the transformation of the feedback prediction channel, the output is similar to the first feature map in terms of spatial size and number of channels. Figure 1 A consistent predictive feature map that is enhanced in spatial detail.

[0080] This step is crucial for introducing feedback information. It involves using the second feature map from higher levels. Input to a specially designed feedback prediction channel The predictor is not a simple upsampler; its structure includes: 1. Upsampling layer. Used to... Enlarging the spatial dimensions to make it consistent with the first feature map The dimensions are initially aligned, and the number of channels is matched. The number of channels is the same.

[0081] 2. A learnable network module for detail enhancement. This module refines the coarse features after upsampling. It can be implemented using an "upsampling + feature fusion" structure (i.e., fusing details from skip connections in shallower layers of the network) or an "upsampling + attention mechanism" structure (i.e., adaptively enhancing details in key regions). Finally, the predictor outputs a predicted feature map. It is not only with The dimensions of the passageways are completely consistent, and the spatial details have been reconstructed and enhanced under semantic guidance.

[0082] S350: Calculate the difference between the first feature map and the predicted feature map to generate an error signal.

[0083] The ideal prediction features generated by the feedback prediction channel The first feature actually extracted from the network Compare them. Calculate the difference between the two (e.g., = - or = - ), generate an error signal This signal quantifies the deviation between the current network's low-level feature extraction results and high-level semantic expectations, directly reflecting the loss of details or feature distortion caused by image quality degradation or limitations of the model itself.

[0084] S360, the error signal is input to a modulator, which generates a modulation signal based on the error signal, adaptively adjusts the first feature map, and outputs the modulated feature map.

[0085] Error signal It is fed into a modulator ( The modulator's function is to convert the error signal into specific feature correction instructions. Depending on the application scenario's requirements for computational efficiency and correction granularity, one of three mechanisms can be employed.

[0086] Mechanism 1, channel-level modulation, includes: generating a set of channel scaling coefficients and channel bias coefficients based on the error signal, and scaling and biasing each channel of the first feature map.

[0087] Specifically, according to Generate a set of channel-level scaling and bias parameters for... Perform overall calibration.

[0088] Mechanism 2, feature map-level modulation, includes: generating a feature correction matrix with the same shape as the first feature map based on the error signal, and adding the matrix to the first feature map.

[0089] Specifically, according to Generate a fine-grained correction matrix ,and Add them point by point to repair spatial details.

[0090] Mechanism 3, convolution kernel-level modulation, includes: generating a convolution kernel adjustment amount based on the error signal, used to adjust the... The weight parameters of the convolution kernel are adjusted, and the first feature map is recalculated using the adjusted convolution kernel.

[0091] Specifically, according to Dynamic adjustment of the first The convolutional kernel weights of the layer are adjusted, and the features are recalculated from the root. This is done by analyzing the first feature map. After performing the above adaptive adjustments, the modulator outputs an optimized, modulated feature map. .

[0092] In one implementation, dynamically selecting the implementation method of the modulator based on the computing power of the deployment device may include the following steps: When computational efficiency is a priority, the channel-level modulation with the lightest computational load is selected; When fine-tuning of the spatial details of the feature map is required, the feature map-level modulation with a computational load higher than channel-level modulation but lower than convolution kernel-level modulation is selected. When long-term optimization of core network parameters is required and computational resources allow, the convolutional kernel-level modulation with the heaviest computational load should be selected.

[0093] In this implementation, firstly, a modulation strategy is formulated based on the real-time requirements, accuracy requirements, and computing resources of the deployed hardware for different functional modules of the intelligent driving system.

[0094] For embedded platforms with extremely limited computing resources (such as microprocessors used for basic alarm functions), computational efficiency is prioritized, and the system defaults to channel-level modulation with the lightest computational load. This approach generates modulation parameters through global pooling and lightweight fully connected layers, achieving rapid calibration of characteristic channels with minimal computational overhead, ensuring stable system operation under resource constraints.

[0095] For in-vehicle intelligent computing units with moderate computing power (such as domain controllers used for panoramic perception), their tasks require fine-tuning the spatial details of feature maps to improve the accuracy of localization and contouring. In this case, the system chooses feature map-level modulation, which has a higher computational load than channel-level modulation but significantly lower than convolutional kernel-level modulation. This approach generates a spatial detail correction matrix through a lightweight convolutional sub-network, enabling pixel-level enhancement of specific regions of the feature map and effectively handling partially occluded or blurred targets.

[0096] For cloud servers with sufficient computing power or during offline model training, the goal is not only to process the current data but also to optimize the core network parameters over the long term to improve the model's inherent capabilities. In this case, when computing resources permit, the system selects the most computationally demanding convolutional kernel-level modulation. This approach optimizes feature extraction capabilities from the root by dynamically adjusting the convolutional kernel weights, thereby achieving continuous evolution of model performance. It is particularly suitable for scene-specific optimization of pre-trained models.

[0097] During actual operation, the system has a built-in resource scheduler. This scheduler evaluates available computing resources and current task requirements in real time or according to preset configurations.

[0098] When the system initiates a low-latency, high-throughput real-time sensing task, the resource scheduler identifies its need to "prioritize computational efficiency" and assigns channel-level modulation to the relevant network layers.

[0099] When the system processes critical scenarios (such as navigating complex intersections) and needs to generate high-precision target segmentation maps, the scheduler will identify the need for "fine-grained spatial detail correction" and enable feature map-level modulation for relevant layers even if the current computing power is not at its peak, to ensure output quality.

[0100] When the vehicle is parked or the network connection is good, the system can upload some data to the cloud. The cloud server uses its abundant computing resources to perform incremental training on the received difficult examples using convolutional kernel-level modulation, achieving "long-term optimization" of the network's core parameters. The optimized model parameters are then sent back to the vehicle, completing the iterative capability development of the entire system.

[0101] This implementation, through the aforementioned dynamic selection mechanism, enables the invention to flexibly adapt to various hardware platforms, from low-end to high-end, achieving "on-demand allocation" under different resource constraints. It ensures basic functionality and real-time performance in resource-constrained environments while also maximizing model performance under appropriate conditions, achieving a high degree of balance between computational efficiency and recognition accuracy. This significantly enhances the practical value and deployment flexibility of the recognition method in industrial practice.

[0102] S370, replace the original first feature map with the modulated feature map, and re-perform the self-feature map. Layered forward propagation calculation.

[0103] The higher quality obtained using step S360 Replace the original Then, the network from the first The layer begins, and the forward propagation is re-executed. This means that subsequent layers ( +1, +2, ...) will be used for secondary calculations based on this corrected and enhanced feature to obtain more accurate high-level semantic features.

[0104] The S380 uses the final network output recognition results for decision-making and control in the intelligent driving system.

[0105] After the aforementioned processing, which includes feedback self-learning, the accuracy and robustness of the network's final classification or detection results (such as "pedestrian 10 meters ahead" or "vehicle on the left") are significantly improved. These results are transmitted to the decision-making and control center of the intelligent driving system, serving as the basis for key decisions such as vehicle path planning, collision warning, and adaptive cruise control, thereby effectively improving driving safety in complex real-world road conditions.

[0106] This invention constructs a multi-level, closed-loop feedback correction system by introducing an inter-layer feedback self-learning mechanism consisting of a feedback prediction channel and a modulator into the network. This mechanism utilizes high-level abstract semantic information as a guiding signal, refines the reconstruction of low-level features through a learnable feedback prediction channel, and dynamically corrects deviations in the low-level feature representation using a flexible modulator. This innovative design effectively overcomes the problems of feature distortion, detail loss, and semantic bias faced by traditional feedforward convolutional neural networks in complex scenarios, significantly improving the model's feature robustness and detail preservation capabilities under various interference environments such as sudden changes in illumination, motion blur, and rain / snow. This method not only effectively reduces missed and false detections of key targets such as pedestrians and vehicles but also improves the stability of inference while maintaining high accuracy. Ultimately, it provides more reliable and accurate recognition results for the perception module of intelligent driving systems, laying a solid technical foundation for vehicle decision-making and control, and significantly improving driving safety.

[0107] exist Figure 3 Based on the illustrated embodiment, as one implementation of the present invention, the learnable network module for detail enhancement is a feature fusion module; The feedback prediction channel generates a prediction feature map through the following steps a1 to a3: Step a1: Upsample the second feature map to initially align its spatial size and number of channels with the first feature map.

[0108] Will come from high level +1 Second Feature Map Input upsampling layer. Because... Having undergone preceding pooling or strided convolution operations, its spatial size is smaller than that of the first feature map. This step restores its spatial dimensions to a similar level using upsampling operations such as bilinear interpolation or transposed convolution. Same scale, and make its number of channels the same as With the same number of channels, preliminary alignment of spatial dimensions is achieved, laying the foundation for subsequent feature fusion.

[0109] Step a2, compare the upsampled features with those from the convolutional neural network. The detailed features that are connected by shallower layers are then fused together.

[0110] The upsampled features are compared with those obtained from the network through the skip connection mechanism. The original detailed features from shallower layers are fused. These shallower features (e.g., those from the first layer) -1 or The -2 layer has higher spatial resolution and retains rich details such as edges and textures, but its semantic level is lower. This step uses channel concatenation or element-wise addition to complementarily fuse the upsampling results of high-level features (rich in semantic information) with shallow features (rich in detail information), effectively solving the problem of detail blurring caused by simple upsampling.

[0111] Step a3: Process the fused features through at least one convolutional layer to output a predicted feature map.

[0112] The fused features are input into a refined module consisting of at least one convolutional layer. The functions of this convolutional layer are: ① to reduce the dimensionality and integrate the information of the concatenated features; ② to enhance the model's expressive power through a non-linear activation function; and ③ to adaptively learn how to optimally combine the fused semantic and detailed information, correcting potential semantic biases and structural irregularities. Finally, this module outputs a feature with good spatial size, number of channels, and... The predicted feature maps are completely identical and have been enhanced in both spatial detail and semantic accuracy. .

[0113] This implementation effectively utilizes the diverse features within the network by constructing a feedback prediction path of "upsampling-feature fusion-convolutional refinement," achieving accurate feedback and detailed reconstruction of high-level semantic information to low-level features, and significantly improving the quality of feature representation.

[0114] exist Figure 3 Based on the illustrated embodiment, as one implementation of the present invention, the learnable network module for detail enhancement is an attention mechanism module; The feedback prediction channel generates a prediction feature map through the following steps b1 to b3: Step b1: Upsample the second feature map to initially align its spatial size and number of channels with the first feature map.

[0115] Will come from high level +1 Second Feature Map The input upsampling layer is used to restore its spatial dimensions to match those of the first feature map through operations such as bilinear interpolation or transposed convolution. At the same scale, complete the basic alignment of spatial dimensions and make its channel number the same as... The number of channels is the same.

[0116] Step b2: Input the upsampled features into the attention mechanism module to generate feature weights for enhancing key details.

[0117] The upsampled features are input into an attention mechanism module. This module adaptively learns and generates a feature weight map by analyzing the spatial and channel information of the feature map. Specifically, for spatial attention, the module evaluates the importance of each spatial location in the feature map and generates a spatial weight matrix to highlight key regions relevant to target recognition (such as pedestrian edges, vehicle outlines, etc.) while suppressing background interference. For channel attention, the module evaluates the importance of different feature channels and generates channel weight vectors to strengthen feature channels that contribute more to the current recognition task.

[0118] Step b3: Use feature weights to weight the upsampled features and output the predicted feature map.

[0119] The spatial weight matrix and channel weight vector generated in step b2 are applied to the upsampled features. First, the channel weights are used to scale different feature channels, and then the spatial weights are used to emphasize different spatial locations, achieving targeted enhancement of key details. This dual attention weighting mechanism enables the network to automatically focus on the feature components most relevant to the recognition task, ultimately outputting a predicted feature map that significantly enhances details while preserving semantic information. .

[0120] This implementation introduces an attention mechanism, enabling the feedback prediction channel to adaptively focus on key details. This effectively solves the problems of structural irregularities and blurred details in complex scenarios, providing a higher quality foundation for subsequent error calculation and feature modulation.

[0121] exist Figure 3 Based on the illustrated embodiment, as one implementation of the present invention, the convolutional neural network is configured to include multiple feedback self-learning structures; Each feedback self-learning structure is based on a hierarchical pair ( , +1) Establish; this level includes the previous level with higher spatial resolution. and immediately following the hierarchy Then, the next level with lower spatial resolution +1; Multiple feedback self-learning structures together constitute a multi-layered feedback self-learning system.

[0122] Specifically, first, a deep convolutional neural network is constructed as the basic architecture. Within this network, multiple non-contiguous layer pairs are selected to establish a feedback self-learning structure. For example, a structure based on layer pairs (…) can be established simultaneously. , The first feedback structure and hierarchical pair () , The second feedback structure, in which, > This ensures that the two feedback structures do not overlap in hierarchy.

[0123] The execution of a multi-layered feedback self-learning system includes the following steps: Step c1, forward propagation and feature recording.

[0124] The input image data undergoes normal forward propagation in the network. When it propagates to the level of the first feedback structure... At that time, the system records the first feature map output by that layer. ; Continue to spread to higher levels At that time, record the second feature map. Similarly, when propagating to the hierarchy of the second feedback structure... and At that time, the corresponding feature maps are recorded respectively. and .

[0125] Step c2, Parallel Feedback Prediction The feedback prediction channels in each feedback structure operate in parallel: First Feedback Prediction Channel G ( → ) Received feature map Through upsampling and detail enhancement modules, the output is compared to Aligned Predicted Feature Map .

[0126] Second Feedback Prediction Channel G ( → ) Received feature map Output and Aligned Predicted Feature Map .

[0127] Step c3, parallel error calculation and modulation.

[0128] Each feedback structure independently calculates the error and performs characteristic modulation: Calculate the first error signal = - The modulated feature map is generated by the first modulator. .

[0129] Calculate the second error signal = - The modulated feature map is generated by the second modulator. .

[0130] Step c4, sequential recalculation mechanism.

[0131] Because the feedback structures are built at different depths, the system employs a sequential recalculation strategy: First, use the modulated feature map Replace the original feature map From the hierarchy Begin recalculating forward; when the recalculation reaches the level... When using the feature map obtained at this time Replace the original feature map Continue to complete the calculations for the remaining levels.

[0132] This implementation achieves the following beneficial effects by constructing a multi-layered feedback self-learning system: 1. Progressive feature optimization: The bottom-level feedback structure mainly corrects spatial details and local features, while the middle-level feedback structure performs calibration at a higher semantic level, forming a progressive optimization process from details to semantics.

[0133] 2. Error propagation suppression: By setting feedback correction mechanisms at multiple key points in the network, the inter-layer propagation of feature bias is effectively suppressed, significantly improving the feature representation quality of the entire network.

[0134] 3. Improved training stability: The multi-layer feedback structure provides more short-circuit paths for gradient propagation, alleviating the gradient vanishing problem in deep networks and making the training process more stable and efficient.

[0135] 4. Enhanced anti-interference capability: Different levels of feedback structure have different robustness to different types of input disturbances, and multi-layer cooperation ensures the stable performance of the system in complex environments.

[0136] This multi-layered feedback self-learning system design enables the network to perform fine-grained calibration of feature representations from multiple levels of abstraction, demonstrating excellent performance and robustness in complex visual perception tasks in intelligent driving.

[0137] Secondly, embodiments of the present invention provide a convolutional neural network optimization device based on autonomous learning, the device comprising: The image data acquisition module is used to acquire raw image data containing pedestrians, vehicles, or traffic signs captured by the vehicle-mounted camera. An image data input module is used to input the original image data into a convolutional neural network for forward propagation. The network contains multiple sequentially connected layers, wherein the spatial resolution of the feature map decreases layer by layer as the layer deepens. The feature map recording module is used to record feature maps extracted from adjacent layers during the forward propagation process, wherein the feature maps extracted from adjacent layers include those from the previous layer with higher spatial resolution. The first feature map, and from the immediately following layer Then, the next level with lower spatial resolution +1 second feature map; A predictive feature map generation module is used to input the second feature map into a feedback prediction channel. The feedback prediction channel includes an upsampling layer and a learnable network module for detail enhancement. Through the transformation of the feedback prediction channel, the output is similar in spatial size and number of channels to the first feature map. Figure 1 A consistent predictive feature map that is enhanced in spatial detail; An error signal calculation module is used to calculate the difference between the first feature map and the predicted feature map, and generate an error signal; The modulated feature map generation module is used to input the error signal to a modulator, which generates a modulation signal based on the error signal, adaptively adjusts the first feature map, and outputs the modulated feature map. The feature map adjustment module is used to replace the original first feature map with the modulated feature map and re-perform the self-adjustment. Layered forward propagation calculation; The recognition result output module is used to apply the final recognition results output by the network to the decision control of the intelligent driving system.

[0138] Thirdly, embodiments of the present invention provide an electronic device, including: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method described in the first aspect.

[0139] Fourthly, according to an embodiment of the present invention, a computer-readable storage medium is provided thereon storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0140] This invention constructs a multi-level, closed-loop feedback correction system by introducing an inter-layer feedback self-learning mechanism consisting of a feedback prediction channel and a modulator into the network. This mechanism utilizes high-level abstract semantic information as a guiding signal, refines the reconstruction of low-level features through a learnable feedback prediction channel, and dynamically corrects deviations in the low-level feature representation using a flexible modulator. This innovative design effectively overcomes the problems of feature distortion, detail loss, and semantic bias faced by traditional feedforward convolutional neural networks in complex scenarios, significantly improving the model's feature robustness and detail preservation capabilities under various interference environments such as sudden changes in illumination, motion blur, and rain / snow. This method not only effectively reduces missed and false detections of key targets such as pedestrians and vehicles but also improves the stability of inference while maintaining high accuracy. Ultimately, it provides more reliable and accurate recognition results for the perception module of intelligent driving systems, laying a solid technical foundation for vehicle decision-making and control, and significantly improving driving safety.

Claims

1. A convolutional neural network optimization method based on autonomous learning, characterized in that, The method includes: Acquire raw image data containing pedestrians, vehicles, or traffic signs captured by vehicle-mounted cameras; The original image data is input into a convolutional neural network for forward propagation. The network contains multiple sequentially connected layers, wherein the spatial resolution of the feature map decreases with each layer as the depth increases. During the forward propagation process, feature maps extracted from adjacent layers are recorded, wherein the feature maps extracted from adjacent layers include those from the previous layer with higher spatial resolution. The first feature map, and from the immediately following layer Then, the next level with lower spatial resolution +1 second feature map; The second feature map is input into the feedback prediction channel, which includes an upsampling layer and a learnable network module for detail enhancement. Through the transformation of the feedback prediction channel, a predicted feature map that is consistent with the first feature map in terms of spatial size and number of channels and is enhanced in terms of spatial details is output. Calculate the difference between the first feature map and the predicted feature map to generate an error signal; The error signal is input to a modulator, which generates a modulation signal based on the error signal, adaptively adjusts the first feature map, and outputs a modulated feature map. The original first feature map is replaced with the modulated feature map, and the process is repeated from the first feature map. Layered forward propagation calculation; The final network output recognition results are used for decision-making and control in the intelligent driving system.

2. The method according to claim 1, characterized in that, The learnable network module used for detail enhancement is a feature fusion module; The feedback prediction channel generates the prediction feature map through the following steps: The second feature map is upsampled to initially align its spatial size and number of channels with the first feature map. The upsampled features are compared with those from the convolutional neural network. The detailed features from the shallower hierarchical jumps are then fused together; The fused features are processed through at least one convolutional layer to output the predicted feature map.

3. The method according to claim 1, characterized in that, The learnable network module used for detail enhancement is an attention mechanism module; The feedback prediction channel generates the prediction feature map through the following steps: The second feature map is upsampled to initially align its spatial size and number of channels with the first feature map. The upsampled features are input into the attention mechanism module to generate feature weights for enhancing key details; The upsampled features are weighted using the feature weights to output the predicted feature map.

4. The method according to claim 1, characterized in that, The modulator generates a modulation signal based on the error signal and adaptively adjusts the first feature map. Specifically, this is achieved as follows: Channel-level modulation includes: generating a set of channel scaling coefficients and channel bias coefficients based on the error signal, and scaling and biasing each channel of the first feature map; or, Feature map-level modulation includes: generating a feature correction matrix with the same shape as the first feature map based on the error signal, and adding the matrix to the first feature map; or, Convolutional kernel-level modulation includes: generating a convolutional kernel adjustment amount based on the error signal, used to adjust the kernel level... The weight parameters of the convolution kernel are adjusted, and the first feature map is recalculated using the adjusted convolution kernel.

5. The method according to claim 4, characterized in that, Based on the computing power of the deployed equipment, the implementation method of the modulator is dynamically selected, including: When computational efficiency is a priority, the channel-level modulation with the lightest computational load is selected; When fine-tuning of the spatial details of the feature map is required, the feature map-level modulation with a computational load higher than channel-level modulation but lower than convolution kernel-level modulation is selected. When long-term optimization of core network parameters is required and computational resources allow, the convolutional kernel-level modulation with the heaviest computational load should be selected.

6. The method according to claim 1, characterized in that, The convolutional neural network is configured to include multiple feedback self-learning structures; Each of the aforementioned feedback self-learning structures is based on a hierarchical pair defined in the method of claim 1. , +1) Establish; Multiple feedback self-learning structures together constitute a multi-layered feedback self-learning system.

7. The method according to claim 1, characterized in that, The previous level with higher spatial resolution , is the layer in the convolutional neural network where the spatial resolution of the first output feature map is lower than the resolution of the input image.

8. A convolutional neural network optimization device based on autonomous learning, characterized in that, The device includes: The image data acquisition module is used to acquire raw image data containing pedestrians, vehicles, or traffic signs captured by the vehicle-mounted camera. An image data input module is used to input the original image data into a convolutional neural network for forward propagation. The network contains multiple sequentially connected layers, wherein the spatial resolution of the feature map decreases layer by layer as the layer deepens. The feature map recording module is used to record feature maps extracted from adjacent layers during the forward propagation process, wherein the feature maps extracted from adjacent layers include those from the previous layer with higher spatial resolution. The first feature map, and from the immediately following layer Then, the next level with lower spatial resolution +1 second feature map; A predictive feature map generation module is used to input the second feature map into a feedback prediction channel. The feedback prediction channel includes an upsampling layer and a learnable network module for detail enhancement. Through the transformation of the feedback prediction channel, a predictive feature map that is consistent with the first feature map in terms of spatial size and number of channels and is enhanced in terms of spatial details is output. An error signal calculation module is used to calculate the difference between the first feature map and the predicted feature map, and generate an error signal; The modulated feature map generation module is used to input the error signal to a modulator, which generates a modulation signal based on the error signal, adaptively adjusts the first feature map, and outputs the modulated feature map. The feature map adjustment module is used to replace the original first feature map with the modulated feature map and re-perform the self-adjustment. Layered forward propagation calculation; The recognition result output module is used to apply the final recognition results output by the network to the decision control of the intelligent driving system.

9. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images