Adaptive Image Processing Method and System Based on Nulling Neural Network

Abstract

The present invention relates to the technical field of data processing, and specifically relates to an adaptive image processing method based on a nullifying neural network. Spatiotemporally synchronized data of a target to be measured is obtained through a multi-modal sensor array. The spatiotemporally synchronized data includes visible light-infrared dual-band environmental images, point cloud data, and millimeter-wave radar sensor data. The environmental images are preprocessed, and the preprocessed environmental images are input into the nullifying neural network to extract image features. The nullifying neural network can dynamically adjust the connection weights and activation thresholds of neurons according to the complexity of the environmental images. The image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data are fused. According to the fused feature data, a pre-trained multi-layer classifier is used to identify the scene type of the target to be measured, and a response strategy is adaptively selected according to the identified scene type. The present invention improves the accuracy of image processing in the field of autonomous driving.

Description

Technical Field

[0001] The present invention belongs to the technical field of image data processing, and particularly relates to an adaptive image processing method and system based on a nulling neural network. Background Art

[0002] As a major innovation in the transportation field in recent years, autonomous driving technology has many potential advantages such as improving traffic safety, alleviating traffic congestion, and reducing energy consumption, and has become a research hotspot in the global technology and automotive industries. In the early image processing of autonomous driving, traditional image processing methods mainly relied on handcrafted features and machine learning algorithms. For example, features of an image were obtained through methods such as edge detection and color feature extraction, and then machine learning algorithms such as support vector machines (SVMs) and decision trees were used for classification and recognition. These methods could solve some simple image processing problems to a certain extent, but had obvious limitations. First, the extraction of handcrafted features required a large amount of domain knowledge and experience and was difficult to adapt to complex and changing road environments. Different scenarios might require different feature extraction methods, which made the scalability and generality of the system poor. Second, the performance of traditional machine learning algorithms depended on the quality of the features. When image data was affected by factors such as noise and illumination changes, the stability and reliability of the features would be greatly affected, resulting in a decrease in recognition accuracy. In addition, traditional methods had low computational efficiency when dealing with large-scale image data and were difficult to meet the real-time requirements of autonomous driving systems.

[0003] With the development of deep learning technology, neural networks have achieved great success in the field of image processing and have gradually been applied to autonomous driving systems. Neural networks have powerful feature learning capabilities and can automatically learn effective feature representations from a large amount of image data without manual feature design. For example: An improved method for autonomous driving image classification based on a convolutional neural network, the specific steps being: (1) collecting real-time surrounding environment information and performing image classification on it; (2) constructing a neural network and importing the image into the neural network for feature extraction; (3) performing optimized stitching processing on each group of image data to generate video data; (4) analyzing the video data and giving a risk warning; Another example: An autonomous driving information recognition method based on a sparse neural network, including constructing a sparse neural network model for an autonomous driving environment perception image, performing adversarial sample training in autonomous driving on the sparse neural network model through a cost function constructed by a sparse constraint term and a sparse regularization term; recognizing the autonomous driving environment perception image through the trained sparse neural network to obtain detection targets in the image to achieve autonomous driving information recognition.

[0004] Although significant progress has been made in neural network-based image processing for autonomous driving, there are still some deficiencies. For example, current neural network models need to improve their robustness when facing complex environments and adversarial samples. In rapidly changing or highly complex time-varying scenarios, the features of images change greatly, leading to a decrease in the accuracy of neural network image processing. Summary of the Invention

[0005] The present invention provides an adaptive image processing method and system based on a nullifying neural network, aiming to solve the problem that the low accuracy of existing autonomous driving image processing leads to incorrect decisions.

[0006] First aspect: The present invention provides an adaptive image processing method based on a nullifying neural network, and the method includes the following steps:

[0007] Step S1: Obtain spatio-temporal synchronous data of a target to be measured through a multi-modal sensor array, where the spatio-temporal synchronous data includes visible light-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, and multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit;

[0008] Step S2: Preprocess the environmental image, and input the preprocessed environmental image into a nullifying neural network to extract image features; the nullifying neural network can dynamically adjust the connection weights and activation thresholds of neurons according to the complexity of the environmental image;

[0009] Step S3: Fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data;

[0010] Step S4: According to the fused feature data, use a pre-trained classifier to identify the scene type of the target to be measured; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree;

[0011] Step S5: Adaptively select a coping strategy according to the identified scene type.

[0012] Further, the preprocessing of the environmental image in step S2, inputting the preprocessed environmental image into a nullifying neural network to extract image features specifically is:

[0013] Step S201: Perform light correction, image sharpening, noise reduction, normalization, and grayscale processing on the environmental image to obtain a preprocessed environmental image;

[0014] Step S202: Calculate the image complexity according to the information entropy, texture complexity, or edge density of the preprocessed environmental image;

[0015] Step S203: Classify the image complexity level according to predefined rules;

[0016] Step S204: Dynamically adjust the connection weights and activation thresholds of the neurons in the nullifying neural network according to the image complexity level;

[0017] Step S205: Input the preprocessed environmental image into the nullifying neural network with adjusted parameters for image feature extraction.

[0018] Furthermore, the specific calculation formulas for dynamically adjusting the connection weights and activation thresholds of the neurons in the nullifying neural network according to the image complexity level are as follows:

[0019] , where is the original connection weight, is the adjustment coefficient corresponding to the complexity level, is the adjusted connection weight;

[0020] Among them, is the original activation threshold, is the offset corresponding to the complexity level, is the adjusted activation threshold.

[0021] Furthermore, the specific method for fusing the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data in step S3 is any one of the weighted average method, Kalman filtering method, and Bayesian estimation method.

[0022] Furthermore, before fusing the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, it also includes: denoising and filtering the lidar three-dimensional point cloud data and millimeter-wave radar sensor data.

[0023] Furthermore, the pre-trained classifier in step S4 can also be a voting-based ensemble classifier composed of a decision tree, SVM, and naive Bayes classifier.

[0024] Furthermore, use online learning and / or incremental learning techniques to update the parameters of the pre-trained classifier.

[0025] Furthermore, the specific method for adaptively selecting a coping strategy according to the recognized scene type in step S5 is: query the pre-established decision information table according to the recognized scene type to determine the coping strategy; the coping strategies include slow down, stop moving forward, turn on the warning light, and drive according to the construction indication signs.

[0026] Furthermore, according to the new scenario data and decision feedback accumulated during the actual operation process, the decision information table is updated regularly or in real time.

[0027] Second aspect: The present invention provides an adaptive image processing system based on a nullifying neural network, including an acquisition module, a feature extraction module, a nullifying neural network module, a data fusion module, a scene recognition module, and an adaptive control module; the acquisition module is used to obtain spatio-temporal synchronization data of the target to be measured, and the spatio-temporal synchronization data includes visible light-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, where multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit;

[0028] The feature extraction module is used to preprocess the environmental image and input the preprocessed environmental image into the nullifying neural network to extract image features;

[0029] The nullifying neural network module is used to dynamically adjust the connection weights and activation thresholds of the neurons in the nullifying neural network according to the image complexity;

[0030] The data fusion module is used to fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree;

[0031] The scene recognition module is used to identify the scene type of the target to be measured by using a pre-trained classifier according to the fused feature data;

[0032] The adaptive control module is used to adaptively select coping strategies according to the identified scene type.

[0033] Compared with the prior art, the beneficial effects of the present invention are:

[0034] (1) Multi-sensor information fusion to improve the accuracy of image recognition: In order to improve the accuracy and adaptability of image recognition in complex environments, data acquisition and information fusion are carried out through multi-modal sensor spatio-temporal alignment technology and dual-band imaging design. Through the complementary information of multi-sensors, the surrounding environment is more comprehensively perceived, reducing the uncertainty of single image information under harsh conditions and laying a solid foundation for accurately identifying the scene type.

[0035] (2) Improve the accuracy and adaptability of image feature extraction: The nullifying neural network can dynamically adjust the neuron connection weights and activation thresholds according to the complexity of the environmental image. When facing simple scene images, it reduces unnecessary consumption of computing resources and quickly and accurately extracts key features; when encountering complex scenes, it can automatically enhance the ability to extract complex features. Compared with traditional neural networks with fixed parameters, it greatly improves the accuracy and adaptability of image feature extraction for images of different complexities, providing more reliable feature data for subsequent scene recognition.

[0036] (3) Improve the accuracy and stability of scene recognition: Adopt a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree to achieve fast detection, semantic understanding, and dynamic decision-making, improving the accuracy and adaptability of scene recognition, reducing misjudgments, and providing strong guarantee for the autonomous driving system to make correct decisions.

[0037] (4) Implement more reasonable and intelligent response strategies: Query the pre-established decision information table according to the recognized scene type to determine response strategies, covering various strategies such as slow deceleration, stop moving forward, turn on warning lights, and drive according to construction signs, and can regularly or real-time update the decision information table according to new scene data and decision feedback accumulated during actual operation, which enables the autonomous driving system to quickly make reasonable and intelligent responses when facing various scenes. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for description in the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0039] Figure 1 It is a flowchart of the adaptive image processing method based on the nullifying neural network of the present invention;

[0040] Figure 2 It is a block diagram of the adaptive image processing system based on the nullifying neural network of the present invention. Detailed Embodiments

[0041] In order to make the purpose, technical solutions and advantages of the present invention more clear, the following further details the present invention in conjunction with the drawings and embodiments. Obviously, the specific embodiments described here are only used to explain the present invention, which are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0042] Example 1

[0043] As Figure 1 shown, the embodiment of the present invention relates to an adaptive image processing method based on a nullifying neural network, and the method includes the following steps:

[0044] Step S1: Obtain spatio-temporally synchronized data of the target to be measured through a multi-modal sensor array. The spatio-temporally synchronized data includes visible light-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, wherein multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit;

[0045] Specifically, professional image acquisition devices are used, such as visible light cameras and infrared cameras equipped with large-size image sensors and high-magnification optical zoom lenses, to ensure that the subtle features of the environment where the target to be measured is located can be clearly captured, and high-quality environmental images are obtained. At the same time, a high-precision lidar sensor is used to quickly generate three-dimensional point cloud data with extremely high density, accurately outlining the three-dimensional shape of the target and its surrounding environment. The millimeter-wave radar sensor is used to utilize the characteristics of the millimeter-wave frequency band to efficiently detect the motion parameters of the target, such as speed, acceleration, etc., and comprehensively collect the spatio-temporally synchronized data of the target to be measured, including environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data. Compared with single-mode image acquisition, the present invention has the following advantages through multi-modal sensor spatio-temporal alignment technology (assisted by an inertial measurement unit) and dual-band imaging design: First, in terms of multi-modal spatio-temporal alignment, single-mode image acquisition does not consider the spatio-temporal synchronization problem of different sensors, resulting in errors in data fusion. The present invention uses spatio-temporal alignment technology (assisted by an inertial measurement unit) for timestamp synchronization and spatial coordinate unification to ensure that the data is aligned in time and space, improving the accuracy of subsequent fusion. Second, the dual-band imaging design uses both visible light and infrared images at the same time. Visible light has rich details when the light is sufficient, while infrared can capture thermal radiation information in low light or bad weather. Single-mode image acquisition only uses one of them and cannot adapt to complex environments. The combination of dual bands can be complementary, enhancing the robustness of environmental perception. Third, multi-source data complementarity. The present invention combines image, point cloud, and radar data. Multi-modal data can describe the target from different dimensions. For example, lidar provides an accurate three-dimensional structure, millimeter-wave radar has strong speed measurement and penetration, and visible light / infrared provides texture and thermal features. This multi-source data can improve the comprehensiveness of scene understanding, especially in complex or dynamic environments.

[0046] Step S2: Preprocess the environmental image, and input the preprocessed environmental image into a nullifying neural network to extract image features; the nullifying neural network can dynamically adjust the connection weights and activation thresholds of neurons according to the complexity of the environmental image;

[0047] Since autonomous driving not only requires object recognition but also an understanding of their motion trends, interrelationships, etc. When the nullifying neural network extracts image features, it can do so from multiple dimensions. Through the forward propagation of multiple layers of neurons, from low-level edge and texture features to high-level semantic features, it comprehensively analyzes the environmental image. Moreover, the nullifying neural network can dynamically adjust the neuron connection weights and activation thresholds. For example, in a complex environment such as a city street with heavy traffic, pedestrians shuttling, and numerous traffic signs, the nullifying neural network can quickly analyze the image, enhance the weights of neurons related to feature extraction of key elements such as pedestrians, vehicles, traffic lights, etc., weaken the weights of irrelevant background information, accurately perceive the dynamically changing driving environment, and provide timely and accurate data support for autonomous driving decisions.

[0048] Specifically, the preprocessing of the environmental image in step S2 and the input of the preprocessed environmental image into the nullifying neural network to extract image features are specifically as follows:

[0049] Step S201: Perform illumination correction, image sharpening, denoising, normalization, and grayscale processing on the environmental image to obtain the preprocessed environmental image;

[0050] Step S202: Calculate the image complexity according to the information entropy or texture complexity or edge density of the preprocessed environmental image; In the present invention, taking information entropy as an example, the calculation process of image complexity is introduced in detail. First, convert the color image to a grayscale image. Since information entropy mainly focuses on the distribution of image grayscale values, after converting the color image to a grayscale image, it is more convenient for subsequent calculations. Assume the size of the image is M×N. After converting to a grayscale image, each pixel point (i,j) (i = 1,⋯,M; j = 1,⋯,N) in the image has a grayscale value g(i,j), and the range of grayscale values is usually from 0 (black) to 255 (white). Then, count all the grayscale values in the image. Create an array histogram with a length of 256 to store the number of occurrences of each grayscale value. Traverse each pixel point in the image. For the grayscale value g(i,j) of each pixel point, increment the value of histogram[g(i,j)] by 1. Finally, calculate the probability of occurrence of each grayscale value. The total number of pixels in the image is M×N. For the grayscale value k (k = 0,⋯,255), the probability p(k) of its occurrence is histogram[k] / (M×N). Finally, calculate the information entropy of the image using the following formula . The larger the information entropy, the more uniform the distribution of grayscale values in the image, the greater the amount of information contained in the image, usually meaning the image is more complex; the smaller the information entropy, the more concentrated the distribution of image grayscale values, and the relatively simpler the image.

[0051] Step S203: Classify the image complexity according to predefined rules;

[0052] According to the calculated image complexity, the images are divided into three levels: simple, medium, and complex. For example, when the image information entropy is greater than a certain threshold, it is determined as a complex image; when the information entropy is less than another lower threshold, it is determined as a simple image.

[0053] Step S204: Dynamically adjust the connection weights and activation thresholds of the neurons in the annihilation neural network according to the image complexity level;

[0054] Specifically, according to the image complexity level, the annihilation neural network dynamically adjusts the connection weights and activation thresholds of the neurons. For complex images, increase the connection weights between neurons so that the network can better capture the subtle features and complex relationships in the image. At the same time, adjust the activation threshold to make the neurons more easily activated to enhance the response to complex features. For simple images, appropriately weaken the connection strength and reduce the activation sensitivity of the neurons to avoid overlearning of simple features by the network. For example, in the case of complex images, increase the connection weights of the neurons by 10% - 20% and reduce the activation threshold by 5% - 10%; in the case of simple images, reduce the connection weights by 10% - 15% and increase the activation threshold by 5% - 8%. Through this dynamic parameter adjustment mechanism, the annihilation neural network can adaptively optimize its own performance to handle input images of different complexities. This adaptability greatly enhances the processing ability of images of different complexities, improves the efficiency and quality of feature extraction, provides more representative and reliable image features for subsequent data fusion and scene type recognition, and further improves the performance and accuracy of the entire adaptive image processing method.

[0055] Furthermore, the specific calculation formulas for the connection weights and activation thresholds of the neurons in the annihilation neural network are:

[0056] , where is the original connection weight, is the adjustment coefficient corresponding to the complexity level, is the adjusted connection weight;

[0057] where is the original activation threshold, is the offset corresponding to the complexity level, is the adjusted activation threshold.

[0058] Step S205: Input the preprocessed environmental image into the annihilation neural network after parameter adjustment for image feature extraction.

[0059] When the pre - processed environmental image enters the nullifying neural network, the data starts from the input layer and propagates forward layer by layer according to the connection weights and activation functions predefined in the nullifying neural network. Inside each neuron, the input signal first undergoes a weighted summation operation, that is, multiplying each input signal by the corresponding connection weight and then accumulating them. This process is like weighted integration of information from different sources, and the magnitude of the connection weight determines the influence degree of each input signal on the neuron output. Then, the result of the weighted summation is compared with the activation threshold. If the result is greater than the activation threshold, the neuron will be activated and then undergoes a non - linear transformation through the activation function. Commonly used activation functions include ReLU (Rectified Linear Unit), Sigmoid, and Tanh. Among them, the ReLU function has the significant advantage of simple calculation, which can greatly improve the operation efficiency and perform well in alleviating the vanishing gradient problem. Therefore, the activation function of this invention is the ReLU function. Through the continuous forward propagation of multiple layers of neurons, the originally complex image data is gradually abstracted into features at different levels, such as from low - level features like edges and textures to high - level semantic features.

[0060] Step S3: Fuse the image features, lidar three - dimensional point cloud data, and millimeter - wave radar sensor data.

[0061] To improve the adaptability to complex environments, the image acquisition device is fused with other sensors (such as lidar, millimeter - wave radar, etc.). Through the complementary information of multiple sensors, the surrounding environment can be perceived more comprehensively, reducing the uncertainty of single - image information under harsh conditions and laying a solid foundation for accurately identifying the scene type.

[0062] Specifically, any one of the data fusion methods, such as the weighted average method, Kalman filtering method, and Bayesian estimation method, is used to fuse the image features extracted from the environmental image, the three - dimensional point cloud data obtained by the lidar, and the sensor data of the millimeter - wave radar. To improve the data quality and reliability, before fusing the data, first, the lidar three - dimensional point cloud data and the millimeter - wave radar sensor data are denoised and filtered. Then, different types of data (image features, lidar three - dimensional point cloud data, and millimeter - wave radar sensor data) are normalized so that they can be effectively fused at the same scale. For example, the image features and the point cloud data are spliced according to certain rules, combined with the motion features in the millimeter - wave radar data, to form fusion feature data containing multi - dimensional information, giving full play to the advantages of various types of data and providing more comprehensive information for subsequent analysis.

[0063] Step S4: Based on the fused feature data, use a pre-trained classifier to identify the scene type of the target to be measured; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree;

[0064] To improve the accuracy and stability of scene recognition, the present invention adopts a hierarchical classifier, which includes: the first layer uses a lightweight convolutional neural network for fast target detection and preliminary scene classification; the second layer uses a semantic understanding module based on a graph neural network to construct a scene semantic map and mine the relationships and context information between targets; the third layer uses a meta-reinforcement learning strategy to generate a dynamic decision tree and adaptively adjust the classification strategy according to the scene risk level; the hierarchical classifier adopted by the present invention has the following advantages: Considering context information: The graph neural network semantic understanding module in the hierarchical classifier can construct a scene semantic map, mine the relationships and context information between targets, and have a deeper understanding of complex scenes. Dynamic decision-making ability: The meta-reinforcement learning strategy of the hierarchical classifier can generate a dynamic decision tree and adaptively adjust the classification strategy according to the scene risk level, and can better handle scenes with different degrees of complexity and risk levels. Multi-scale feature utilization: The lightweight convolutional neural network of the hierarchical classifier can perform fast target detection and preliminary scene classification, realize multi-scale feature extraction and utilization, and can effectively process image data with different resolutions and scales. However, the complexity of this hierarchical classifier is high, the training difficulty is large, and the design of the hierarchical classifier may rely on more prior knowledge. Therefore, the present invention can also adopt a voting-based ensemble classifier composed of a decision tree, an SVM, and a naive Bayes classifier. Through the voting mechanism, each classifier cooperates with each other, which not only avoids the limitations of a single classifier when facing complex and diverse data, but also circumvents the potential problems brought by the complex structure, difficult training, and excessive dependence on prior knowledge of the hierarchical classifier. In practical applications, the fused feature data can be effectively classified at a relatively low cost and in a more convenient way, so as to accurately identify the scene type of the target to be measured. Further, the classifier parameters are updated in real time through online learning and / or incremental learning techniques, so that it can adapt to new scene data in a timely manner, significantly improve the accuracy of scene recognition, and reduce misjudgment. In complex and ever-changing actual driving scenarios, it maintains stable and reliable recognition performance, providing a strong guarantee for the correct decision-making of the autonomous driving system.

[0065] Step S5: Select a coping strategy adaptively according to the recognized scene type.

[0066] Specifically, step S5 selects a coping strategy adaptively according to the recognized scene type as follows: query a pre-established decision information table according to the recognized scene type, so as to determine the coping strategy; the coping strategies include slow down, stop advancing, turn on the warning light, and drive according to the construction sign.

[0067] The creation process of the decision information table is as follows: First, clarify the scenario type in combination with the actual business requirements and usage environment. For example, in the autonomous driving scenario, the scenario types include "traffic congestion ahead", "encountering a construction area", "pedestrians crossing the road", etc. Then, for each scenario type, determine the corresponding coping strategies. For example, when the scenario type is "traffic congestion ahead", the coping strategy can be "slow down gradually". If the scenario type is "encountering a construction area", the strategy can be "drive according to the construction signs". Finally, use different data structures to store the decision information table. The common storage methods are dictionaries (in Python), database tables, etc.

[0068] During the actual operation process, new scenario data and decision feedback are continuously collected. The new scenario data refers to the scenario types that have not appeared before, and the decision feedback is the effective coping strategies adopted for these new scenarios. For example, during autonomous driving, a new road condition (new scenario data) is encountered. After system processing and manual intervention, a suitable coping strategy (decision feedback) is determined. In order to improve the ability to quickly and accurately find the coping strategy corresponding to the scenario type, the present invention updates the decision information table regularly or in real time according to the new scenario data and decision feedback accumulated during the actual operation process.

[0069] Furthermore, the step S5 of adaptively selecting the coping strategy according to the identified scenario type can also be: According to the identified scenario type, use a reinforcement learning model to generate the optimal coping strategy in real time. Among them, the reinforcement learning model takes the scenario type as the input, and continuously tries different actions and adjusts / optimizes the coping strategy according to the feedback. For example, in the autonomous driving scenario, according to the identified traffic congestion scenario type, the reinforcement learning model can dynamically adjust the vehicle's driving speed, following distance, and lane-changing strategy to achieve the optimal driving efficiency and safety.

[0070] The present invention inputs the preprocessed environmental image into a zeroing neural network that can dynamically adjust the connection weights and activation thresholds of neurons to extract image features. Then, it fuses the image features with various sensor data. Based on the fused feature data, it uses a trained classifier to identify the scene type. Finally, it adaptively selects a coping strategy according to the identified scene type. Among them, using the zeroing neural network for feature extraction greatly improves the accuracy and adaptability of image feature extraction for images with different complexities. Fusing multiple sensor data reduces the uncertainty of single-image information under harsh conditions. Using a multi-level classifier or a voting-based ensemble classifier composed of a decision tree, SVM, and naive Bayes classifier avoids the limitations of a single classifier, enabling the classifier to more accurately identify the scene type. Based on the accurate scene type, it adaptively selects a suitable coping strategy. This series of processes reflects the adaptive characteristics from data acquisition, data processing to scene judgment and then to decision-making selection, ultimately meeting the requirements for image processing accuracy in the field of autonomous driving.

[0071] Embodiment 2

[0072] As Figure 2 shown, the embodiment of the present invention relates to an adaptive image processing system based on a zeroing neural network, including an acquisition module, a feature extraction module, a zeroing neural network module, a data fusion module, a scene recognition module, and an adaptive control module. The acquisition module is used to obtain the spatio-temporal synchronous data of the target to be measured. The spatio-temporal synchronous data includes visible-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data. Among them, multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit.

[0073] The feature extraction module is used to preprocess the environmental image and input the preprocessed environmental image into the zeroing neural network to extract image features.

[0074] The zeroing neural network module is used to dynamically adjust the connection weights and activation thresholds of the neurons of the zeroing neural network according to the image complexity.

[0075] The data fusion module is used to fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data.

[0076] The scene recognition module is used to identify the scene type of the target to be measured according to the fused feature data by using a pre-trained classifier. The classifier is a hierarchical classifier that fuses a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree.

[0077] The adaptive control module is used to adaptively select a coping strategy according to the identified scene type.

[0078] The present invention uses technologies such as annihilation neural networks and data fusion technology for image processing to improve the accuracy of image processing, so that autonomous vehicles can correctly select coping strategies.

[0079] The above embodiments only express the preferred implementation modes of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims

1. An adaptive image processing method based on a nulling neural network, characterized in that It includes the following steps: Step S1: Obtain spatio-temporally synchronized data of the target to be measured through a multi-modal sensor array. The spatio-temporally synchronized data includes visible-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, where multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit; Step S2: Preprocess the environmental image and input the preprocessed environmental image into a nullifying neural network to extract image features; The nullifying neural network can dynamically adjust the connection weights and activation thresholds of neurons according to the complexity of the environmental image; Specifically, in step S201: Perform illumination correction, image sharpening, denoising, normalization, and grayscale processing on the environmental image to obtain the preprocessed environmental image; Step S202: Calculate the image complexity according to the information entropy, texture complexity, or edge density of the preprocessed environmental image; Step S203: Classify the image complexity according to predefined rules; Step S204: Dynamically adjust the connection weights and activation thresholds of the neurons in the nullifying neural network according to the image complexity level; Step S205: Input the preprocessed environmental image into the nullifying neural network after parameter adjustment to extract image features; Step S3: Fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data; Step S4: According to the fused feature data, use a pre-trained classifier to identify the scene type of the target to be measured; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree; Step S5: Adaptively select a coping strategy according to the identified scene type.

2. The adaptive image processing method based on the annihilation neural network according to claim 1, wherein, The specific calculation formula for dynamically adjusting the connection weights and activation thresholds of the neurons in the nullifying neural network according to the image complexity level is: , where is the original connection weight, is the adjustment coefficient corresponding to the complexity level, is the adjusted connection weight; Among them, is the original activation threshold, is the offset corresponding to the complexity level, is the adjusted activation threshold.

3. The adaptive image processing method based on the annihilation neural network according to claim 1, wherein, The specific method for step S3 to fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data is any one of the weighted average method, the Kalman filtering method, and the Bayesian estimation method.

4. The adaptive image processing method based on the annihilation neural network according to claim 3, wherein Before fusing the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, it also includes: performing denoising and filtering processing on the lidar three-dimensional point cloud data and millimeter-wave radar sensor data.

5. The adaptive image processing method based on the annihilation neural network according to claim 1, wherein The pre-trained classifier in step S4 can also be: a voting-based integrated classifier composed of a decision tree, an SVM, and a naive Bayes classifier.

6. The adaptive image processing method based on the annihilation neural network according to claim 5, wherein, Adopt online learning and / or incremental learning techniques to update the parameters of the pre-trained classifier.

7. The adaptive image processing method based on the annihilation neural network according to claim 1, characterized in that, The specific method for step S5 to adaptively select a coping strategy according to the identified scene type is: query the pre-established decision information table according to the identified scene type to determine the coping strategy; the coping strategies include slow deceleration, stop advancing, turn on the warning light, and drive according to the construction indication signs.

8. The adaptive image processing method based on the annihilation neural network according to claim 7, wherein Regularly or real-time update the decision information table according to the new scene data and decision feedback accumulated during the actual operation process.

9. An adaptive image processing system based on a nullifying neural network, comprising an acquisition module, a feature extraction module, a nullifying neural network module, a data fusion module, a scene recognition module, and an adaptive control module, characterized in that, The acquisition module is used to obtain the spatio-temporal synchronization data of the target to be measured. The spatio-temporal synchronization data includes visible-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, where multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit; The feature extraction module is used to preprocess the environmental image and input the preprocessed environmental image into a nullifying neural network to extract image features; The nullifying neural network module is used to dynamically adjust the connection weights and activation thresholds of the neurons in the nullifying neural network according to the image complexity. Specifically, calculate the image complexity based on the information entropy or texture complexity or edge density of the preprocessed environmental image; divide the image complexity into levels according to predefined rules; dynamically adjust the connection weights and activation thresholds of the neurons in the nullifying neural network according to the image complexity level; The data fusion module is used to fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data; The scene recognition module is used to identify the scene type of the target to be measured according to the fused feature data by using a pre-trained classifier. The classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree; The adaptive control module is used to adaptively select coping strategies according to the identified scene type.

Images