Self-adaptive image processing method and system based on zero neural network

Abstract

The invention relates to the technical field of data processing, in particular to an adaptive image processing method based on a zero neural network, and the method comprises the steps: obtaining the time-space synchronization data of a to-be-detected target through a multi-modal sensor array, and enabling the time-space synchronization data to comprise a visible light-infrared dual-band environment image, point cloud data and millimeter wave radar sensor data; preprocessing the environment image, and inputting the preprocessed environment image into a zero neural network to extract image features; the zero neural network can dynamically adjust the connection weight and the activation threshold of neurons according to the complexity of an environment image; and the image features, the laser radar three-dimensional point cloud data and the millimeter wave radar sensor data are fused, a scene type of a to-be-detected target is identified by using a pre-trained multilayer classifier according to the fused feature data, and a coping strategy is adaptively selected according to the identified scene type. The accuracy of image processing in the field of automatic driving is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image data processing, and particularly relates to an adaptive image processing method and system based on a nulling neural network. Background Art

[0002] As a major innovation in the transportation field in recent years, autonomous driving technology has many potential advantages such as improving traffic safety, alleviating traffic congestion, and reducing energy consumption, and has become a research hotspot in the global technology and automotive industries. In the early image processing for autonomous driving, traditional image processing methods mainly relied on handcrafted features and machine learning algorithms. For example, features of an image were obtained through methods such as edge detection and color feature extraction, and then machine learning algorithms such as support vector machines (SVMs) and decision trees were used for classification and recognition. These methods could solve some simple image processing problems to a certain extent, but had obvious limitations. First, the extraction of handcrafted features required a large amount of domain knowledge and experience and was difficult to adapt to complex and changing road environments. Different scenarios might require different feature extraction methods, which made the scalability and generality of the system poor. Second, the performance of traditional machine learning algorithms depended on the quality of the features. When image data was affected by factors such as noise and illumination changes, the stability and reliability of the features would be greatly affected, resulting in a decrease in recognition accuracy. In addition, traditional methods had low computational efficiency when dealing with large-scale image data and were difficult to meet the real-time requirements of autonomous driving systems.

[0003] With the development of deep learning technology, neural networks have achieved great success in the field of image processing and have gradually been applied to autonomous driving systems. Neural networks have powerful feature learning capabilities and can automatically learn effective feature representations from a large amount of image data without the need for handcrafted feature design. For example: An improved method for autonomous driving image classification based on a convolutional neural network, the specific steps being: (1) collecting real-time surrounding environment information and performing image classification on it; (2) constructing a neural network and importing the image into the neural network for feature extraction; (3) performing optimized stitching processing on each group of image data to generate video data; (4) analyzing the video data and giving a risk warning; or for another example: An autonomous driving information recognition method based on a sparse neural network, including constructing a sparse neural network model for an autonomous driving environment perception image, performing adversarial sample training for autonomous driving on the sparse neural network model through a cost function constructed by a sparse constraint term and a sparse regularization term; and recognizing the autonomous driving environment perception image through the trained sparse neural network to obtain detection targets in the image to achieve autonomous driving information recognition.

[0004] Although significant progress has been made in neural network-based image processing for autonomous driving, there are still some deficiencies. For example, current neural network models have room for improvement in terms of robustness when faced with complex environments and adversarial samples. In rapidly changing or highly complex time-varying scenarios, the features of images change significantly, resulting in a decrease in the accuracy of neural network-based image processing. Summary of the Invention

[0005] The present invention provides an adaptive image processing method and system based on a nullifying neural network, aiming to solve the problem that the low accuracy of existing autonomous driving image processing leads to incorrect decisions.

[0006] First aspect: The present invention provides an adaptive image processing method based on a nullifying neural network, and the method includes the following steps: Step S1: Obtain spatio-temporal synchronous data of a target to be measured through a multi-modal sensor array, where the spatio-temporal synchronous data includes visible-light-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, and multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit; Step S2: Preprocess the environmental image, and input the preprocessed environmental image into a nullifying neural network to extract image features; the nullifying neural network can dynamically adjust the connection weights and activation thresholds of neurons according to the complexity of the environmental image; Step S3: Fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data; Step S4: According to the fused feature data, use a pre-trained classifier to identify the scene type of the target to be measured; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree; Step S5: Adaptively select a coping strategy according to the identified scene type.

[0007] Further, in step S2, preprocessing the environmental image and inputting the preprocessed environmental image into a nullifying neural network to extract image features specifically includes: Step S201: Perform light correction, image sharpening, noise reduction, normalization, and grayscale processing on the environmental image to obtain a preprocessed environmental image; Step S202: Calculate the image complexity according to the information entropy, texture complexity, or edge density of the preprocessed environmental image; Step S203: Divide the image complexity into levels according to predefined rules; Step S204: Dynamically adjust the connection weights and activation thresholds of the neurons of the nullifying neural network according to the image complexity level; Step S205: Input the preprocessed environmental image into the nullifying neural network with adjusted parameters for image feature extraction.

[0008] Further, the specific calculation formulas for dynamically adjusting the connection weights and activation thresholds of the neurons in the nullifying neural network according to the image complexity level are as follows: , where is the original connection weight, is the adjustment coefficient corresponding to the complexity level, is the adjusted connection weight; Among them, is the original activation threshold, is the offset corresponding to the complexity level, is the adjusted activation threshold.

[0009] Further, the specific method for fusing the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data in step S3 is any one of the weighted average method, Kalman filtering method, and Bayesian estimation method.

[0010] Further, before fusing the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, it also includes: denoising and filtering the lidar three-dimensional point cloud data and millimeter-wave radar sensor data.

[0011] Further, the pre-trained classifier in step S4 can also be: a voting-based ensemble classifier composed of a decision tree, SVM, and naive Bayes classifier.

[0012] Further, use online learning and / or incremental learning techniques to update the parameters of the pre-trained classifier.

[0013] Further, the specific method for adaptively selecting a coping strategy according to the recognized scene type in step S5 is: query the pre-established decision information table according to the recognized scene type to determine the coping strategy; the coping strategies include slow down, stop advancing, turn on the warning light, and drive according to the construction indication sign.

[0014] Further, update the decision information table regularly or in real time according to the new scene data and decision feedback accumulated during the actual operation process.

[0015] Second aspect: The present invention provides an adaptive image processing system based on a nullifying neural network, including an acquisition module, a feature extraction module, a nullifying neural network module, a data fusion module, a scene recognition module, and an adaptive control module; the acquisition module is used to obtain spatio-temporal synchronization data of a target to be measured, and the spatio-temporal synchronization data includes visible light-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data, wherein multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit; The feature extraction module is used to preprocess the environmental image and input the preprocessed environmental image into the nullifying neural network to extract image features; The nullifying neural network module is used to dynamically adjust the connection weights and activation thresholds of the neurons of the nullifying neural network according to the image complexity; The data fusion module is used to fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree; The scene recognition module is used to identify the scene type of the target to be measured by using a pre-trained classifier according to the fused feature data; The adaptive control module is used to adaptively select a coping strategy according to the identified scene type.

[0016] Compared with the prior art, the beneficial effects of the present invention are: (1) Multi-sensor information fusion to improve the accuracy of image recognition: In order to improve the accuracy and adaptability of image recognition in complex environments, data acquisition and information fusion are carried out through multi-modal sensor spatio-temporal alignment technology and dual-band imaging design. Through the complementary information of multi-sensors, the surrounding environment is more comprehensively perceived, reducing the uncertainty of single-image information under harsh conditions and laying a solid foundation for accurately identifying the scene type.

[0017] (2) Improve the accuracy and adaptability of image feature extraction: The nullifying neural network can dynamically adjust the neuron connection weights and activation thresholds according to the complexity of the environmental image. When facing simple scene images, it reduces unnecessary consumption of computing resources and quickly and accurately extracts key features; when encountering complex scenes, it can automatically enhance the ability to extract complex features. Compared with traditional neural networks with fixed parameters, it greatly improves the accuracy and adaptability of image feature extraction for images with different complexities, providing more reliable feature data for subsequent scene recognition. (3) Improve the accuracy and stability of scene recognition: A hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree is adopted to achieve rapid detection, semantic understanding, and dynamic decision-making, improving the accuracy and adaptability of scene recognition, reducing misjudgments, and providing strong guarantee for the correct decision-making of the autonomous driving system. (4) Implement more reasonable and intelligent response strategies: Query the pre-established decision information table according to the recognized scene type to determine the response strategies, covering various strategies such as slow deceleration, stop advancing, turn on warning lights, and drive according to construction signs, and can update the decision information table regularly or in real time according to the new scene data and decision feedback accumulated during actual operation, which enables the autonomous driving system to quickly make reasonable and intelligent responses when facing various scenes. Description of the Drawings In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0018] Figure 1 It is a flowchart of the adaptive image processing method based on the annihilation neural network of the present invention; Figure 2 It is a block diagram of the adaptive image processing system based on the annihilation neural network of the present invention. Detailed Embodiments

[0019] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following further details the present invention in conjunction with the drawings and embodiments. Obviously, the specific embodiments described here are only used to explain the present invention, which are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0020] Embodiment 1 As Figure 1 shown, the embodiment of the present invention relates to an adaptive image processing method based on the annihilation neural network, and the method includes the following steps: Step S1: Obtain the spatio-temporal synchronous data of the target to be measured through a multi-modal sensor array, and the spatio-temporal synchronous data includes visible light-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter wave radar sensor data, wherein the multi-sensor spatio-temporal alignment is realized through an inertial measurement unit; Specifically, professional-level image capture devices are used, such as visible light cameras and infrared cameras equipped with large-size image sensors and high-magnification optical zoom lenses, to ensure that the subtle features of the environment where the target to be measured is located can be clearly captured and high-quality environmental images can be obtained. At the same time, high-precision lidar sensors are used to quickly generate extremely dense three-dimensional point cloud data, accurately outlining the three-dimensional shape of the target and its surrounding environment. Millimeter-wave radar sensors are used to utilize the characteristics of the millimeter-wave frequency band to efficiently detect the motion parameters of the target, such as speed, acceleration, etc., and collect spatio-temporally synchronized data of the target to be measured in all directions, including environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data. Compared with single-mode image acquisition, the present invention has the following advantages through multi-modal sensor spatio-temporal alignment technology (assisted by an inertial measurement unit) and dual-band imaging design: First, in terms of multi-modal spatio-temporal alignment, single-mode image acquisition does not consider the spatio-temporal synchronization problem of different sensors, resulting in errors in data fusion. The present invention uses spatio-temporal alignment technology (assisted by an inertial measurement unit) for timestamp synchronization and spatial coordinate unification to ensure that the data is aligned in time and space, improving the accuracy of subsequent fusion. Second, the dual-band imaging design uses visible light and infrared images simultaneously. Visible light has rich details in sufficient light, while infrared can capture thermal radiation information in low light or bad weather. Single-mode image acquisition only uses one of them and cannot adapt to complex environments. The combination of dual bands can be complementary, enhancing the robustness of environmental perception. Third, multi-source data complementarity. The present invention combines image, point cloud, and radar data. Multi-modal data can describe the target from different dimensions. For example, lidar provides an accurate three-dimensional structure, millimeter-wave radar has strong speed measurement and penetration, and visible light / infrared provides texture and thermal features. This multi-source data can improve the comprehensiveness of scene understanding, especially performing better in complex or dynamic environments.

[0021] Step S2: Preprocess the environmental image and input the preprocessed environmental image into a nullifying neural network to extract image features; the nullifying neural network can dynamically adjust the connection weights and activation thresholds of neurons according to the complexity of the environmental image; Since autonomous driving not only requires identifying objects but also understanding their motion trends, mutual relationships, etc. When the nullifying neural network extracts image features, it can do so from multiple dimensions. Through the forward propagation of multiple layers of neurons, from low-level edge and texture features to high-level semantic features, the environmental image is comprehensively analyzed. Moreover, the nullifying neural network can dynamically adjust the connection weights and activation thresholds of neurons. For example, in an environment with high complexity such as a city street with heavy traffic, pedestrians shuttling, and numerous traffic signs, the nullifying neural network can quickly analyze the image, enhance the weights of neurons related to the extraction of features of key elements such as pedestrians, vehicles, and traffic lights, weaken the weights of irrelevant background information, and accurately perceive the dynamically changing driving environment, providing timely and accurate data support for autonomous driving decisions.

[0022] Specifically, in step S2, the environmental image is preprocessed, and the preprocessed environmental image is input into the nullification neural network to extract image features, specifically as follows: Step S201: Perform illumination correction, image sharpening, noise reduction, normalization, and grayscale processing on the environmental image to obtain the preprocessed environmental image; Step S202: Calculate the image complexity according to the information entropy or texture complexity or edge density of the preprocessed environmental image; In the present invention, taking the information entropy as an example, the calculation process of the image complexity is introduced in detail. First, convert the color image into a grayscale image. Since the information entropy mainly focuses on the distribution of the grayscale values of the image, after converting the color image into a grayscale image, it is more convenient for subsequent calculations. Assume that the size of the image is M×N. After converting it into a grayscale image, each pixel point (i,j) (i = 1,⋯,M; j = 1,⋯,N) in the image has a grayscale value g(i,j), and the range of the grayscale value is usually from 0 (black) to 255 (white). Then, count all the grayscale values in the image. Create an array histogram with a length of 256 to store the number of times each grayscale value appears. Traverse each pixel point in the image. For the grayscale value g(i,j) of each pixel point, increment the value of histogram[g(i,j)] by 1. Finally, calculate the probability of each grayscale value appearing. The total number of pixels in the image is M×N. For the grayscale value k (k = 0,⋯,255), the probability p(k) of its appearance is histogram[k] / (M×N). Finally, use the following formula to calculate the information entropy of the image . The larger the information entropy, the more uniform the distribution of the grayscale values in the image, the greater the amount of information contained in the image, and usually it means the image is more complex; the smaller the information entropy, the more concentrated the distribution of the grayscale values in the image, and the image is relatively simple.

[0023] Step S203: Classify the image complexity into levels according to predefined rules; According to the calculated image complexity, the image is divided into three levels: simple, medium, and complex. For example, when the image information entropy is greater than a certain threshold, it is determined as a complex image; when the information entropy is less than another lower threshold, it is determined as a simple image.

[0024] Step S204: Dynamically adjust the connection weights and activation thresholds of the neurons in the nullification neural network according to the image complexity level; Specifically, according to the image complexity level, the nullifying neural network dynamically adjusts the connection weights and activation thresholds of neurons. For complex images, the connection weights between neurons are increased so that the network can better capture the subtle features and complex relationships in the images. At the same time, the activation threshold is adjusted so that neurons are more easily activated to enhance the response to complex features. For simple images, the connection strength is appropriately weakened and the activation sensitivity of neurons is reduced to avoid overlearning of simple features by the network. For example, in the case of complex images, the connection weights of neurons are increased by 10% - 20%, and the activation threshold is reduced by 5% - 10%; in the case of simple images, the connection weights are reduced by 10% - 15%, and the activation threshold is increased by 5% - 8%. Through this dynamic parameter adjustment mechanism, the nullifying neural network can adaptively optimize its own performance to handle input images of different complexities. This adaptive ability greatly enhances the processing ability of images of different complexities, improves the efficiency and quality of feature extraction, provides more representative and reliable image features for subsequent data fusion and scene type recognition, and further enhances the performance and accuracy of the entire adaptive image processing method.

[0025] Furthermore, the specific calculation formulas for the connection weights and activation thresholds of the neurons in the nullifying neural network are as follows: , where is the original connection weight, is the adjustment coefficient corresponding to the complexity level, is the adjusted connection weight; where is the original activation threshold, is the offset corresponding to the complexity level, is the adjusted activation threshold.

[0026] Step S205: Input the preprocessed environmental image into the nullifying neural network after parameter adjustment for image feature extraction.

[0027] When the preprocessed environmental image enters the nullifying neural network, the data starts from the input layer and propagates forward layer by layer according to the connection weights and activation functions predefined in the nullifying neural network. Inside each neuron, the input signal first undergoes a weighted summation operation, that is, each input signal is multiplied by the corresponding connection weight and then accumulated. This process is like weighted integration of information from different sources, and the magnitude of the connection weight determines the influence degree of each input signal on the neuron output. Then, the result of the weighted summation is compared with the activation threshold. If the result is greater than the activation threshold, the neuron will be activated and then undergo a non - linear transformation through the activation function. Commonly used activation functions include ReLU (Rectified Linear Unit), Sigmoid, and Tanh. Among them, the ReLU function has the significant advantage of simple calculation, which can greatly improve the operation efficiency and performs well in alleviating the vanishing gradient problem. Therefore, the activation function of this invention is the ReLU function. Through the continuous forward propagation of multiple layers of neurons, the originally complex image data is gradually abstracted into features at different levels, such as from low - level features like edges and textures to high - level semantic features.

[0028] Step S3: Fuse the image features, lidar three - dimensional point cloud data, and millimeter - wave radar sensor data. To improve the adaptability to complex environments, the image acquisition device is fused with other sensors (such as lidar, millimeter - wave radar, etc.). Through the complementary information of multiple sensors, the surrounding environment can be perceived more comprehensively, reducing the uncertainty of single - image information under harsh conditions and laying a solid foundation for accurate scene type recognition.

[0029] Specifically, any one of the data fusion methods, such as the weighted average method, Kalman filtering method, and Bayesian estimation method, is used to fuse the image features extracted from the environmental image, the three - dimensional point cloud data obtained by the lidar, and the sensor data of the millimeter - wave radar. To improve the data quality and reliability, before fusing the data, first, the lidar three - dimensional point cloud data and the millimeter - wave radar sensor data are denoised and filtered. Then, different types of data (image features, lidar three - dimensional point cloud data, and millimeter - wave radar sensor data) are normalized so that they can be effectively fused at the same scale. For example, the image features and the point cloud data are spliced according to certain rules, combined with the motion features in the millimeter - wave radar data, to form fusion feature data containing multi - dimensional information, giving full play to the advantages of various types of data and providing more comprehensive information for subsequent analysis.

[0030] Step S4: Based on the fused feature data, use a pre-trained classifier to identify the scene type of the target to be measured; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree; To improve the accuracy and stability of scene recognition, the present invention adopts a hierarchical classifier, which includes: the first layer uses a lightweight convolutional neural network for fast target detection and preliminary scene classification; the second layer uses a semantic understanding module based on a graph neural network to construct a scene semantic graph and mine the relationships and context information between targets; the third layer uses a meta-reinforcement learning strategy to generate a dynamic decision tree and adaptively adjust the classification strategy according to the scene risk level; the hierarchical classifier adopted by the present invention has the following advantages: Considering context information: The graph neural network semantic understanding module in the hierarchical classifier can construct a scene semantic graph, mine the relationships and context information between targets, and have a deeper understanding of complex scenes. Dynamic decision-making ability: The meta-reinforcement learning strategy of the hierarchical classifier can generate a dynamic decision tree and adaptively adjust the classification strategy according to the scene risk level, and can better handle scenes with different complexities and risk levels. Multi-scale feature utilization: The lightweight convolutional neural network of the hierarchical classifier can perform fast target detection and preliminary scene classification, realize multi-scale feature extraction and utilization, and can effectively process image data with different resolutions and scales. However, the hierarchical classifier has high complexity, difficult training, and the design of the hierarchical classifier may rely on more prior knowledge. Therefore, the present invention can also adopt a voting-based integrated classifier composed of a decision tree, an SVM, and a naive Bayes classifier. Through the voting mechanism, each classifier cooperates with each other, avoiding the limitations of a single classifier when facing complex and diverse data, and also avoiding the potential problems brought by the complex structure, difficult training, and excessive dependence on prior knowledge of the hierarchical classifier. In practical applications, it is possible to effectively classify the fused feature data at a relatively low cost and in a more convenient way, so as to accurately identify the scene type of the target to be measured. Further, the classifier parameters are updated in real time through online learning and / or incremental learning techniques, enabling it to adapt to new scene data in a timely manner, significantly improving the accuracy of scene recognition and reducing misjudgment. In complex and ever-changing actual driving scenarios, it maintains stable and reliable recognition performance, providing a strong guarantee for the correct decision-making of the autonomous driving system. Step S5: Select a coping strategy adaptively according to the identified scene type.

[0031] Specifically, step S5 selecting a coping strategy adaptively according to the identified scene type is specifically: querying a pre-established decision information table according to the identified scene type to determine the coping strategy; the coping strategies include slow down, stop advancing, turn on the warning light, and drive according to the construction indication sign.

[0032] The creation process of the decision information table is as follows: First, clarify the scenario types in combination with the actual business requirements and usage environment. For example, in the autonomous driving scenario, the scenario types include "congestion ahead", "encountering a construction area", "pedestrians crossing the road", etc. Then, for each scenario type, determine the corresponding coping strategies. For example, when the scenario type is "congestion ahead", the coping strategy can be "slow down". If the scenario type is "encountering a construction area", the strategy can be "drive according to the construction signs". Finally, use different data structures to store the decision information table. Common storage methods include dictionaries (in Python), database tables, etc.

[0033] During the actual operation process, new scenario data and decision feedback are continuously collected. The new scenario data refers to the scenario types that have not appeared before, and the decision feedback is the effective coping strategies adopted for these new scenarios. For example, during autonomous driving, a new road condition (new scenario data) is encountered. After system processing and manual intervention, a suitable coping strategy (decision feedback) is determined. In order to quickly and accurately find the coping strategy corresponding to the scenario type, the present invention updates the decision information table regularly or in real time according to the new scenario data and decision feedback accumulated during the actual operation process.

[0034] Furthermore, step S5 adaptively selecting the coping strategy according to the identified scenario type can also be: According to the identified scenario type, use the reinforcement learning model to generate the optimal coping strategy in real time. Among them, the reinforcement learning model takes the scenario type as the input, and continuously tries different actions and adjusts / optimizes the coping strategy according to the feedback. For example, in the autonomous driving scenario, according to the identified traffic congestion scenario type, the reinforcement learning model can dynamically adjust the vehicle's driving speed, following distance, and lane-changing strategy to achieve the optimal driving efficiency and safety.

[0035] The present invention inputs the preprocessed environmental image into a nullifying neural network that can dynamically adjust the connection weights and activation thresholds of neurons to extract image features. Then, it fuses the image features with various sensor data. Based on the fused feature data, it uses a trained classifier to identify the scene type. Finally, it adaptively selects a coping strategy according to the identified scene type. Among them, using the nullifying neural network for feature extraction greatly improves the accuracy and adaptability of extracting image features of different complexities. Fusing multiple sensor data reduces the uncertainty of single-image information under harsh conditions. Using a multi-level classifier or a voting-based ensemble classifier composed of a decision tree, SVM, and naive Bayes classifier avoids the limitations of a single classifier, enabling the classifier to more accurately identify the scene type. Selecting an appropriate coping strategy adaptively based on the accurate scene type. This series of processes reflects the adaptive characteristics from data acquisition, data processing to scene judgment and then to decision-making selection, ultimately meeting the requirements for image processing accuracy in the field of autonomous driving.

[0036] Embodiment 2 As Figure 2 shown, an embodiment of the present invention relates to an adaptive image processing system based on a nullifying neural network, including an acquisition module, a feature extraction module, a nullifying neural network module, a data fusion module, a scene recognition module, and an adaptive control module. The acquisition module is used to obtain spatio-temporally synchronized data of a target to be measured. The spatio-temporally synchronized data includes visible-infrared dual-band environmental images, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data. Among them, multi-sensor spatio-temporal alignment is achieved through an inertial measurement unit; The feature extraction module is used to preprocess the environmental image and input the preprocessed environmental image into the nullifying neural network to extract image features; The nullifying neural network module is used to dynamically adjust the connection weights and activation thresholds of the neurons of the nullifying neural network according to the image complexity; The data fusion module is used to fuse the image features, lidar three-dimensional point cloud data, and millimeter-wave radar sensor data; The scene recognition module is used to identify the scene type of the target to be measured according to the fused feature data by using a pre-trained classifier; the classifier is a hierarchical classifier that fuses a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree; The adaptive control module is used to adaptively select a coping strategy according to the identified scene type.

[0037] The present invention uses technologies such as nullifying neural networks and data fusion technologies for image processing to improve the accuracy of image processing, so that autonomous driving vehicles can correctly select coping strategies.

[0038] The above embodiments only represent the preferred implementation modes of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the invention patent shall be subject to the appended claims.

Claims

1. An adaptive image processing method based on an annihilation neural network, characterized in that: The following steps are involved: Step S1: acquiring spatiotemporal synchronization data of the target to be measured through a multimodal sensor array, wherein the spatiotemporal synchronization data includes a visible light-infrared dual-band environment image, a laser radar three-dimensional point cloud data, and a millimeter wave radar sensor data, wherein the multi-sensor spatiotemporal alignment is achieved through an inertial measurement unit; Step S2: preprocessing the environment image, and inputting the preprocessed environment image into a zeroing neural network to extract image features; The zeroing neural network can dynamically adjust the connection weights and activation thresholds of neurons according to the complexity of the environment image; Step S3: fusing the image features, the laser radar three-dimensional point cloud data and the millimeter wave radar sensor data; Step S4: According to the fused feature data, a pre-trained classifier is used to identify the scene type of the target to be tested; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree; Step S5: Adaptively select a response strategy based on the identified scenario type.

2. The adaptive image processing method based on annihilation neural network according to claim 1, characterized in that: In step S2, the environment image is preprocessed and the preprocessed environment image is input into a zeroing neural network to extract image features, specifically: Step S201: performing illumination correction, image sharpening, denoising, normalization and grayscale processing on the environment image to obtain a pre-processed environment image; Step S202: Calculating image complexity according to information entropy or texture complexity or edge density of the preprocessed environment image; Step S203: classifying the image complexity according to predefined rules; Step S204: dynamically adjusting the connection weights and activation thresholds of the neurons of the zeroing neural network according to the image complexity level; Step S205: input the preprocessed environment image into the parameter-adjusted zeroing neural network to extract image features.

3. The adaptive image processing method based on annihilation neural network according to claim 2, characterized in that: The specific calculation formula for dynamically adjusting the connection weights and activation thresholds of the zeroing neural network neurons according to the image complexity level is: ,in, is the original connection weight, is the adjustment coefficient corresponding to the complexity level, is the adjusted connection weight; in, is the original activation threshold, is the offset corresponding to the complexity level, is the adjusted activation threshold.

4. The adaptive image processing method based on annihilation neural network according to claim 1, characterized in that: The specific method of step S3 for fusing the image features, the laser radar three-dimensional point cloud data and the millimeter wave radar sensor data is: any one of the weighted average method, the Kalman filter method and the Bayesian estimation method.

5. The adaptive image processing method based on annihilation neural network according to claim 4, characterized in that: Before fusing the image features, the laser radar three-dimensional point cloud data and the millimeter wave radar sensor data, the method also includes: denoising and filtering the laser radar three-dimensional point cloud data and the millimeter wave radar sensor data.

6. The adaptive image processing method based on annihilation neural network according to claim 1, characterized in that: The pre-trained classifier in step S4 may also be a voting ensemble classifier composed of a decision tree, an SVM and a naive Bayes classifier.

7. The adaptive image processing method based on annihilation neural network according to claim 6, characterized in that: Online learning and / or incremental learning techniques are used to update the pre-trained classifier parameters.

8. The adaptive image processing method based on annihilation neural network according to claim 1, characterized in that: Step S5 adaptively selects a response strategy based on the identified scene type, specifically: querying a pre-established decision information table based on the identified scene type to determine a response strategy; the response strategy includes slowing down, stopping, turning on warning lights, and driving according to construction signs.

9. The adaptive image processing method based on annihilation neural network according to claim 8, characterized in that: The decision information table is updated regularly or in real time based on new scenario data and decision feedback accumulated during actual operation.

10. An adaptive image processing system based on a zeroing neural network, comprising an acquisition module, a feature extraction module, a zeroing neural network module, a data fusion module, a scene recognition module, and an adaptive control module, characterized in that: The acquisition module is used to obtain the time-space synchronization data of the target to be measured, and the time-space synchronization data includes visible light-infrared dual-band environmental images, laser radar three-dimensional point cloud data, and millimeter wave radar sensor data, wherein the multi-sensor time-space alignment is achieved through an inertial measurement unit; The feature extraction module is used to preprocess the environment image and input the preprocessed environment image into a zeroing neural network to extract image features; The zeroing neural network module is used to dynamically adjust the connection weights and activation thresholds of the zeroing neural network neurons according to the image complexity; A data fusion module, used to fuse the image features, the laser radar three-dimensional point cloud data and the millimeter wave radar sensor data; A scene recognition module is used to identify the scene type of the target to be tested using a pre-trained classifier based on the fused feature data; the classifier is a hierarchical classifier that integrates a lightweight convolutional network, a graph neural network semantic understanding module, and a meta-reinforcement learning dynamic decision tree; The adaptive control module is used to adaptively select a response strategy based on the identified scenario type.

Images