Optical sensor demisting method and system for automatic driving based on multi-agent optimization

Abstract

The invention discloses an optical sensor defogging method and system for automatic driving based on multi-agent optimization, and the method comprises the following steps: 1), collecting a fogged image through a vehicle-mounted optical sensor, and carrying out the image preprocessing, thereby obtaining an initial fogged image; 2) constructing a communication network based on multiple agents, and setting parameters of the communication network; and 3) optimizing the initial fogged image by using a communication network to obtain a defogged image, and outputting the defogged image. The system comprises a vehicle-mounted optical sensor, a high-performance concurrent processing unit and a vehicle-mounted computer, according to the invention, the image defogging task is distributed to a plurality of agent nodes for parallel processing, so that the real-time performance and the calculation efficiency of the system are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of in-vehicle optical sensor image defogging, and specifically to an optical sensor defogging method and system for autonomous driving based on multi-agent optimization. Background Art

[0002] With the rapid development of automotive intelligence, networking, and autonomous driving technologies, intelligent connected vehicles equipped with autonomous driving functions are in a new stage of transitioning from test demonstrations to commercial mass production applications. In the field of autonomous driving, vehicles rely on multiple image acquisition devices (such as cameras and lidar) to capture real-time environmental information around the vehicle, thereby supporting the vehicle's environmental perception, target recognition, and path planning. These devices can help the autonomous driving system perceive key information such as lane lines, pedestrians, obstacles, and traffic signals, providing accurate data support for the system. However, under foggy weather conditions, the performance of these image acquisition devices will decline significantly. Fog will cause image blurring and contrast reduction, directly affecting the visibility of objects and resulting in impaired imaging quality.

[0003] In this case, it is difficult for the autonomous driving system to capture clear and effective scene information, resulting in a significant decline in the accuracy of the environmental perception module. Perceptual errors or information loss may cause misjudgments in the system's driving decisions, such as being unable to recognize pedestrians, underestimating the distance to obstacles, or incorrectly detecting road boundaries, ultimately affecting the vehicle's dynamic planning and control performance. Seriously, the autonomous driving system may not be able to respond promptly and correctly to emergencies, thereby triggering traffic accidents and directly endangering the lives of passengers and the safety of other road users.

[0004] Therefore, it is imperative to study defogging methods based on in-vehicle optical sensors. Currently, the research on in-vehicle optical sensor defogging systems mainly falls into two directions: physical defogging and algorithmic defogging:

[0005] Physical defogging technology mainly reduces the interference of fog on imaging through hardware improvements, including installing an anti-fog coating, an infrared supplementary light, or using a polarization light filter in front of the camera. These methods can improve the imaging quality to a certain extent, but their effects are easily affected by factors such as installation angle, light conditions, and equipment aging, with limited applicability and difficulty in dealing with complex situations in dynamic environments.

[0006] Algorithmic defogging technology uses computer vision and image processing algorithms to post-process fogged images to restore clear images. In recent years, algorithms based on traditional methods and deep learning have been widely studied:

[0007] Early dehazing algorithms were mainly based on image restoration theory. By analyzing the physical imaging model of images (such as the dark channel prior method, histogram equalization, etc.), the original images affected by fog were restored. These methods are simple and efficient, but their effects are limited when dealing with complex foggy scenes.

[0008] With the rise of deep learning, image dehazing methods based on convolutional neural networks (CNNs) have developed rapidly. These methods can automatically learn the characteristics of fog through the training of a large amount of data of foggy images and clear images and achieve the dehazing function. For example, networks such as DehazeNet and AOD-Net have shown good performance in the dehazing task. In addition, the application of generative adversarial networks (GANs) and multi-modal fusion technologies has also enabled breakthroughs in the dehazing algorithm in terms of detail enhancement and environmental adaptability. However, these deep learning methods have high requirements for computing resources, complex processing procedures, and strong dependence on high-quality data sets, increasing the development and application costs.

[0009] Therefore, how to optimize computing resources and simplify the processing procedure on the premise of ensuring the dehazing effect has become an important research direction in this field. Summary of the Invention

[0010] The purpose of the present invention is to provide a multi-agent optimization-based defogging method for optical sensors used in autonomous driving, including the following steps:

[0011] 1) Use in-vehicle optical sensors to collect foggy images and perform image preprocessing to obtain initial foggy images;

[0012] 2) Construct a communication network based on multi-agents and set the communication network parameters;

[0013] 3) Use the communication network to optimize the initial foggy images to obtain defogged images and output them.

[0014] Furthermore, the step of performing image preprocessing includes normalization processing.

[0015] Furthermore, the communication network includes M agents, and each agent serves as a node;

[0016] The M agents communicate through a time-invariant undirected graph network G = {V, E}; V = {1, 2, 3,..., M} is the node set; is the edge set; (i, j) in the edge set E indicates that agent i can receive information from agent j.

[0017] Furthermore, the objective function minf(z) of the communication network is as follows:

[0018]

[0019] where f i (z) is the local objective function of agent i.

[0020] Furthermore, in the communication network, the local objective function f i (z) of agent i is as follows:

[0021] f i (z i ) = ||H i × z i - b i || 2 (2)

[0022] where H i is the fuzzy convolution kernel; z i is the observation value of agent i. b i is the error.

[0023] Furthermore, in step 2), the steps of setting the communication network parameters include:

[0024] 2.1) Initialize the tentative solution local gradient variable tracking variable

[0025] 2.2) Set the weight matrix A. A is a non - negative doubly stochastic matrix, and the dimension of A is the same as the dimension of the communication connection graph of the agents, generally an M - order square matrix, where M is the number of agents as mentioned above.

[0026] 2.3) Set the number of iterations as t max .

[0027] Furthermore, in step 3), the steps of optimizing the initial atomization image using the communication network include:

[0028] 3.1) Initialize the number of iterations t = 1;

[0029] 3.2) Update the weighted average of the tentative solution where is the decision variable of neighbor node j of agent i a ij is the weight; is the set of neighbor nodes of agent i;

[0030] 3.3) Update the weighted average of the local gradient variable where is the local gradient variable of agent j;

[0031] 3.4) Update the tentative solution of the decision variable That is:

[0032]

[0033] In the formula, l>0 is the regularization coefficient;

[0034] 3.5) Update the local gradient variable That is:

[0035]

[0036] 3.6) Update the tracking variable That is:

[0037]

[0038] 3.7) Determine whether t>t max holds. If not, let t=t + 1 and return to step 3.2). If so, use the decision variable as the defogged image optimized by agent i;

[0039] Furthermore, the weight a ij satisfies the following formula:

[0040]

[0041] A system applying the on-vehicle optical sensor defogging method includes an on-vehicle optical sensor, a high-performance concurrent processing unit, and an on-vehicle computer;

[0042] The on-vehicle optical sensor collects fogged images and transmits them to the high-performance concurrent processing unit;

[0043] The high-performance concurrent processing unit optimizes the initial fogged image using a communication network to obtain a defogged image and transmits it to the on-vehicle computer;

[0044] The high-performance concurrent processing unit includes an image acquisition module, an image preprocessing module, an agent network module, and an image output module;

[0045] The image acquisition module is used to receive fogged images from the on-vehicle optical sensor;

[0046] The image preprocessing module is used to perform normalization processing on the fogged image to obtain an initial fogged image;

[0047] The agent network module includes M agents and is used to optimize the initial fogged image to obtain a defogged image;

[0048] The image output module is used to transmit the defogged image to the on-vehicle computer;

[0049] The in-vehicle computer performs post-processing on the dehazed image.

[0050] Further, the agent network module includes a communication network construction unit, a variable initialization unit, a tentative solution weight average update unit, a local gradient variable weight average update unit, a decision variable tentative solution update unit, a local gradient variable update unit, and a tracking variable update unit;

[0051] The communication network construction unit is used to construct a communication connection matrix and define a doubly stochastic connected weight matrix;

[0052] The variable initialization unit is used to initialize the tentative solution Local gradient variable Tracking variable

[0053] The tentative solution weight average update unit is used to update the average value of the tentative solution weights

[0054] The local gradient variable weight average update unit is used to update the average value of the local gradient variable weights

[0055] The decision variable tentative solution update unit is used to update the tentative solution of the decision variable

[0056] The local gradient variable update unit is used to update the local gradient variable

[0057] The tracking variable update unit is used to update the tracking variable

[0058] The technical effects of the present invention are beyond doubt. By distributing the computing tasks to multiple nodes for parallel processing, the present invention significantly improves the real-time performance of the system. This architecture supports agents to cooperate in executing multiple tasks, including image contrast enhancement, color restoration, edge smoothing, noise reduction, and distortion compensation, etc., thereby greatly improving the efficiency and effect of image optimization.

[0059] By distributing the image dehazing task to multiple agent nodes for parallel processing, the present invention effectively improves the real-time performance and computing efficiency of the system. Each agent saves a foggy image to be optimized as a node, and realizes collaborative optimization by defining a local objective function. Agents exchange state information through an undirected connected communication network, and fuse neighbor information through a doubly stochastic weight matrix to gradually optimize the global decision vector in a distributed iterative manner. The algorithm includes steps such as image normalization preprocessing, local objective function definition, neighbor state and gradient tracking variable update, and decision variable iterative optimization.

[0060] The present invention significantly improves the imaging clarity of in-vehicle optical sensors in complex environments, providing accurate and high-quality data support for core tasks such as obstacle detection and path planning in autonomous driving systems. It also features high efficiency, real-time response, and strong robustness, ensuring stable performance under various environmental conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Attached Figure 1 is the step diagram of the whole invention

[0062] Attached Figure 2 is the specific flowchart of the algorithm

[0063] Attached Figure 3 is the framework diagram of this system

[0064] Attached Figure 4 is the hardware structure diagram of the system of the present invention

[0065] Attached Figure 5 are the original clear picture after normalization processing, the blurred picture after fogging processing, and the clear picture after fog removal and optimization using this algorithm in the embodiment of the present invention

[0066] Attached Figure 6 are the change situations of the local objective function (i.e., the loss function of each agent) and the global objective function (i.e., the global loss function) in the embodiment of the present invention

[0067] Attached Figure 7 is the change situation of the MSE value during iteration in the embodiment of the present invention

[0068] Attached Figure 8 is the change situation of the PSNR value during iteration in the embodiment of the present invention

[0069] Attached Figure 9 is the change situation of the SSIM value during iteration in the embodiment of the present invention.. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] The present invention will be further described below in conjunction with embodiments, but it should not be understood that the above-mentioned subject scope of the present invention is limited to the following embodiments. Without departing from the above-mentioned technical idea of the present invention, various substitutions and changes made according to common general technical knowledge and customary means in the art shall be included within the protection scope of the present invention.

[0071] Embodiment 1:

[0072] Refer to Figures 1 to 9 , an optical sensor defogging method for autonomous driving based on multi-agent optimization, comprising the following steps:

[0073] 1) Use on-vehicle optical sensors to collect atomization images and perform image preprocessing to obtain initial atomization images;

[0074] 2) Construct a communication network based on multi-agent and set communication network parameters;

[0075] 3) Use the communication network to optimize the initial atomization image to obtain a dehazed image and output it.

[0076] The steps of performing image preprocessing include normalization processing.

[0077] The communication network includes M agents, and each agent serves as a node;

[0078] The M agents communicate through a time-invariant undirected graph network G = {V, E}; V = {1, 2, 3,..., M} is the node set; is the edge set; (i, j) in the edge set E indicates that agent i can receive information from agent j.

[0079] The objective function minf(z) of the communication network is as follows:

[0080]

[0081] In the formula, f i (z) is the local objective function of agent i.

[0082] In the communication network, the local objective function f i (z) of agent i is as follows:

[0083] f i (z i ) = ||H i ×z i -b i || 2 (2)

[0084] Among them, H i is the fuzzy convolution kernel; z i is the observation value of agent i; b i is the error.

[0085] In step 2), the steps of setting the communication network parameters include:

[0086] 2.1) Initialize the tentative solution Local gradient variable Tracking variable

[0087] 2.2) Set the weight matrix A. A is a non - negative doubly stochastic matrix, and the dimension of A is the same as that of the communication connection graph of the agents, generally an M - order square matrix, where M is the number of the above - mentioned agents.

[0088] 2.3) Set the number of iterations as t max 。

[0089] In step 3), the steps of optimizing the initial fog - removed image using the communication network include:

[0090] 3.1) Initialize the number of iterations t = 1;

[0091] 3.2) Update the tentative solution The weighted average value of is the decision variable of neighbor node j of agent i a ij is the weight; is the set of neighbor nodes of agent i;

[0092] 3.3) Update the local gradient variable The weighted average value of is the local gradient variable of agent j;

[0093] 3.4) Update the tentative solution of the decision variable That is:

[0094]

[0095] In the formula, l>0 is the regularization coefficient;

[0096] 3.5) Update the local gradient variable That is:

[0097]

[0098] 3.6) Update the tracking variable That is:

[0099]

[0100] 3.7) Judge whether t>t max holds. If not, let t = t + 1 and return to step 3.2). If so, use the decision variable as the fog - removed image optimized by agent i;

[0101] The weight a ij satisfies the following formula:

[0102]

[0103] Example 2:

[0104] A system applying the vehicle-mounted optical sensor defogging method, comprising a vehicle-mounted optical sensor, a high-performance concurrent processing unit, and a vehicle-mounted computer;

[0105] The vehicle-mounted optical sensor collects fogged images and transmits them to the high-performance concurrent processing unit;

[0106] The high-performance concurrent processing unit optimizes the initial fogged image using a communication network to obtain a defogged image and transmits it to the vehicle-mounted computer;

[0107] The high-performance concurrent processing unit includes an image acquisition module, an image preprocessing module, an agent network module, and an image output module;

[0108] The image acquisition module is used to receive fogged images from the vehicle-mounted optical sensor;

[0109] The image preprocessing module is used to perform normalization processing on the fogged image to obtain an initial fogged image;

[0110] The agent network module includes M agents and is used to optimize the initial fogged image to obtain a defogged image;

[0111] The image output module is used to transmit the defogged image to the vehicle-mounted computer;

[0112] The vehicle-mounted computer performs post-processing on the defogged image (obstacle detection, path planning, or other autonomous driving-related applications).

[0113] The agent network module includes a communication network construction unit, a variable initialization unit, a tentative solution weight average update unit, a local gradient variable weight average update unit, a decision variable tentative solution update unit, a local gradient variable update unit, and a tracking variable update unit;

[0114] The communication network construction unit is used to construct a communication connection matrix and define a doubly stochastic connected weight matrix;

[0115] The variable initialization unit is used to initialize the tentative solution Local gradient variable Tracking variable

[0116] The tentative solution weight average update unit is used to update the tentative solution weight average

[0117] The local gradient variable weight average update unit is used to update the local gradient variable weight average

[0118] The decision variable tentative solution updating unit is used to update the tentative solution of the decision variable

[0119]

[0120] The local gradient variable updating unit is used to update the local gradient variable

[0121] The tracking variable updating unit is used to update the tracking variable

[0122] Embodiment 3:

[0123] An optical sensor defogging method for autonomous driving based on multi-agent optimization, comprising the following steps:

[0124] 1) Use the vehicle-mounted optical sensor to collect the fogged image and perform image preprocessing to obtain the initial fogged image;

[0125] 2) Construct a communication network based on multi-agents and set the communication network parameters;

[0126] 3) Use the communication network to optimize the initial fogged image to obtain the defogged image and output it.

[0127] Embodiment 4:

[0128] An optical sensor defogging method for autonomous driving based on multi-agent optimization, the technical content is the same as that of Embodiment 3. Further, the steps of performing image preprocessing include normalization processing.

[0129] Embodiment 5:

[0130] An optical sensor defogging method for autonomous driving based on multi-agent optimization, the technical content is the same as any one of Embodiments 3-4. Further, the communication network includes M agents, and each agent is used as a node;

[0131] The M agents communicate through a time-invariant undirected graph network G = {V, E}; V = {1, 2, 3,..., M} is the node set; is the edge set; (i, j) in the edge set E indicates that agent i can receive information from agent j.

[0132] Embodiment 6:

[0133] An optical sensor defogging method for autonomous driving based on multi-agent optimization, the technical content is the same as any one of Embodiments 3-5. Further, the objective function of the communication network is as follows:

[0134]

[0135] In the formula, f i (z) is the local objective function of agent i.

[0136] Example 7:

[0137] An optical sensor defogging method for autonomous driving based on multi-agent optimization, the technical content is the same as any one of Examples 3-6. Further, in the communication network, the local objective function f i (z) is as follows:

[0138] f i (z i ) = ||H i ×z i -b i || 2 (2)

[0139] Where H i is the fuzzy convolution kernel; z i is the observation value of agent i.

[0140] Example 8:

[0141] An optical sensor defogging method for autonomous driving based on multi-agent optimization, the technical content is the same as any one of Examples 3-7. Further, in step 2), the steps of setting the communication network parameters include:

[0142] 2.1) Initialize the tentative solution Local gradient variable Tracking variable

[0143] 2.2) Set the weight matrix A. A is a non-negative doubly stochastic matrix, and the dimension of A is the same as the dimension of the communication connection graph of the agents, generally an M-order square matrix, and M is the number of agents as described above.

[0144] 2.3) Set the number of iterations to t max .

[0145] Example 9:

[0146] An optical sensor defogging method for autonomous driving based on multi-agent optimization, the technical content is the same as any one of Examples 3-8. Further, in step 3), the steps of optimizing the initial fogged image using the communication network include:

[0147] 3.1) Initialize the number of iterations t = 1;

[0148] 3.2) Update the weight average value of the tentative solution The decision variable of neighbor node j of agent i a ij is the weight; is the set of neighbor nodes of agent i;

[0149] 3.3) Update the local gradient variable The weighted average of is the local gradient variable of agent j;

[0150] 3.4) Update the tentative solution of the decision variable That is:

[0151]

[0152] where l>0 is the regularization coefficient;

[0153] 3.5) Update the local gradient variable That is:

[0154]

[0155] 3.6) Update the tracking variable That is:

[0156]

[0157] 3.7) Determine whether t>t max holds. If not, let t=t + 1 and return to step 3.2). If so, use the decision variable as the defogged image optimized by agent i;

[0158] Example 10:

[0159] An optical sensor defogging method for autonomous driving based on multi-agent optimization, the technical content is the same as any one of Examples 3-9, and the weight a ij satisfies the following formula:

[0160]

[0161] Example 11:

[0162] A system applying the vehicle-mounted optical sensor defogging method, including a vehicle-mounted optical sensor, a high-performance concurrent processing unit, and a vehicle-mounted computer;

[0163] The vehicle-mounted optical sensor collects the fogged image and transmits it to the high-performance concurrent processing unit;

[0164] The high-performance concurrent processing unit optimizes the initial fogged image using the communication network to obtain a defogged image and transmits it to the vehicle-mounted computer;

[0165] The high-performance concurrent processing unit includes an image acquisition module, an image preprocessing module, an agent network module, and an image output module;

[0166] The image acquisition module is used to receive the foggy image from the vehicle-mounted optical sensor;

[0167] The image preprocessing module is used to perform normalization processing on the foggy image to obtain an initial foggy image;

[0168] The agent network module includes M agents and is used to optimize the initial foggy image to obtain a defogged image;

[0169] The image output module is used to transmit the defogged image to the vehicle-mounted computer;

[0170] The vehicle-mounted computer performs post-processing on the defogged image.

[0171] Example 12:

[0172] A system applying the vehicle-mounted optical sensor defogging method has the same technical content as Example 11. Further, the agent network module includes a communication network construction unit, a variable initialization unit, a tentative solution weight average update unit, a local gradient variable weight average update unit, a decision variable tentative solution update unit, a local gradient variable update unit, and a tracking variable update unit;

[0173] The communication network construction unit is used to construct a communication connection matrix and define a doubly stochastic connected weight matrix;

[0174] The variable initialization unit is used to initialize the tentative solution Local gradient variable Tracking variable

[0175] The tentative solution weight average update unit is used to update the tentative solution weight average

[0176] The local gradient variable weight average update unit is used to update the local gradient variable weight average

[0177] The decision variable tentative solution update unit is used to update the decision variable tentative solution

[0178]

[0179] The local gradient variable update unit is used to update the local gradient variable

[0180] The tracking variable update unit is used to update the tracking variable

[0181] Example 13:

[0182] An optical sensor defogging method for autonomous driving based on multi-agent optimization, comprising the following steps:

[0183] S1. Receive the fogged image collected by the vehicle-mounted optical sensor and perform image preprocessing;

[0184] S2. Build a communication network and initialize the variables;

[0185] S3. Optimize the image using the proposed algorithm;

[0186] S4. Output the optimized clear image.

[0187] The steps of optimizing the image using the proposed algorithm include:

[0188] Update the weighted average of the tentative solution by collecting the decision variables of the neighbor nodes and calculating the weighted average;

[0189] Update the weighted average of the local gradient variable by collecting the local gradients of the neighbor nodes and calculating the weighted average;

[0190] Update the tentative solution of the decision variable by solving an optimization problem including a local objective function, a linear term, and a quadratic term regularization term;

[0191] Update the local gradient variable, emphasizing the difference between the local solution of the agent and the weighted average of the neighbor solutions;

[0192] Update the tracking variable by integrating the gradient tracking information of the neighbors and the dynamic adjustment of the consistency variable.

[0193] The weight matrix is a doubly stochastic connected matrix, that is, the sum of the elements in each row and each column of the matrix is 1, ensuring the global propagation of information between agents.

[0194] Example 14:

[0195] A vehicle-mounted optical sensor defogging system based on multi-agent optimization, the system comprising:

[0196] An image acquisition module for acquiring the fogged image of the vehicle-mounted optical sensor;

[0197] An image preprocessing module for normalizing the acquired image;

[0198] An agent network module composed of M agents for optimizing the fogged image

[0199] An image output module, configured to output the optimized image of the agent as a clear picture after haze removal processing.

[0200] The agent network module includes:

[0201] A communication network construction unit, which constructs a communication connection matrix and defines a doubly stochastic connectivity weight matrix;

[0202] A variable initialization unit, which initializes a tentative solution Local gradient variable Tracking variable

[0203] A tentative solution weight average update unit, configured to collect the decision variables of neighbor nodes and calculate a weighted average;

[0204] A local gradient variable weight average update unit, configured to collect the local gradients of neighbor nodes and calculate a weighted average;

[0205] A decision variable tentative solution update unit, configured to solve an optimization problem including a local objective function, a linear term, and a quadratic term regularization term;

[0206] A local gradient variable update unit, configured to emphasize the difference between the local solution of the agent and the weighted average of the neighbor solutions;

[0207] A tracking variable update unit, configured to integrate the gradient tracking information of neighbors and the dynamic adjustment of consistency variables.

[0208] Example 15:

[0209] An optical sensor defogging system for autonomous driving based on multi-agent optimization, the hardware part of the system consists of an in-vehicle optical sensor, a high-performance concurrent processing unit, and an in-vehicle computer, as shown in the appendix Figure 4 shown. Among them, the in-vehicle optical sensor performs real-time continuous image acquisition, and the high-performance concurrent processing unit performs the calculation of the in-vehicle optical sensor defogging method proposed by the present invention and sends the image defogging result to the in-vehicle computer.

[0210] Example 16:

[0211] An optical sensor defogging method for autonomous driving based on multi-agent optimization, the content is as follows:

[0212] In this method, each agent is regarded as a node and holds an image to be optimized. Consider a system consisting of M agents that cooperate to solve the problem of image dehazing.

[0213] The agents need to reach an agreement on a common decision vector such that the sum of the local objective functions f i (z) of all agents is minimized while satisfying the respective local constraint sets of each agent The optimization objective of each agent is the local objective function f i (z), and the local objective functions and local constraint sets of each agent can be different.

[0214] This problem can be described as the mathematical problem P:

[0215] where only agent i knows the local objective function

[0216] The M agents communicate through a time-invariant undirected graph network, which ensures that there is at least one path between any two agents for information to propagate globally. This communication network is described by G = {V, E}, consisting of a node set V = {1, 2, 3, …, M} and an edge set The (i, j) in the edge set E means that agent i can receive information from agent j.

[0217] The graph G is undirected and connected, that is, if the edge (i, j) ∈ E then the edge (j, i) ∈ E, for all i = 1, 2, 3, …, M, (i, j) ∈ E.

[0218] The weight a ij of the communication graph G represents the degree of importance that agent i attaches to the information received from agent j. If there is no communication link between agent i and agent j, a ij = 0. The weight matrix is denoted as A, and A is a doubly stochastic matrix, that is, the sum of the elements in each row and each column of the matrix A is 1, and the mathematical form is:

[0219]

[0220] The specific update steps of the algorithm are as follows:

[0221] 1. After the system receives the foggy image input from the vehicle-mounted optical sensor, it first preprocesses the image to be optimized, normalizes the image, and compresses the pixel value range to between [0, 1]. Normalization can ensure that pixel values have a consistent ratio when performing calculations and comparisons, thereby improving the stability and efficiency of the algorithm.

[0222] 2. Save the image to be optimized at each agent, and define the local objective function f i (z) for each agent. This f i (z) can be defined according to the specific optimization objective and needs to satisfy that f i is a convex function. For example, a convex objective function based on the observation error:

[0223] f i (z i ) = ||H i × z i - b i || 2

[0224] where H i is the fuzzy convolution kernel, representing the impact of haze on the image. z i is the blurred image, i.e., the observation value of agent i, and z i is the tentative solution to the optimization result.

[0225] 3. Initialize variables. Initialize the tentative solution local gradient variable tracking variable

[0226] 4. Set the weight matrix A. A is a non - negative doubly stochastic matrix, and the dimension of A is the same as the dimension of the communication connection graph of the agents, generally an M - order square matrix, where M is the number of agents mentioned above.

[0227] 5. Denote the iteration number as t max , and in each iteration, make the following updates:

[0228] (1). Update the weighted average of the tentative solution by agent i collecting the decision variables of its neighbor nodes and calculating the weighted average, and using the neighbor information to update its own state to gradually reach a consensus solution.

[0229] (2). Update the weighted average of the local gradient variable by agent i collecting the local gradients of its neighbor nodes and calculating the weighted average, and using the neighbor information to obtain an approximation of the global gradient.

[0230]

[0231] (3). Update the tentative solution of the decision variable:

[0232] ​​By solving this formula, agent i, within its own constraint range z i ∈Z i Find the optimal z i , so that the function Minimize. Function g i (z i ) consists of three parts. The first part is the local objective function f i (z i ), describes the restoration goal of the image by agent i. The second part is the linear term This term is a weighted average of the local gradient variables and local gradient variables The third term is a quadratic regularization term that constrains z i Weighted average of nearby neighbor solutions By enforcing solutions close to neighbors, we gradually achieve consistency among agents. l>0 is the regularization coefficient, which is used to control the strength of this constraint.

[0233] (4) Update local gradient variables:

[0234] Emphasis on local solutions of intelligent agents Weighted average of neighbor solutions The difference between Adjust direction and gradually narrow this gap. Using the weighted average of local gradient variables right Corrections are made to ensure the dynamic consistency of global gradient information.

[0235] (5) Update tracking variables:

[0236] By integrating the gradient tracking information of neighbors and dynamically adjusting the consistency variables, the local gradient changes of each agent are reflected.

[0237] Determine whether the maximum number of iterations t has been reached max If it has not been reached, set t = t + 1 and continue updating; if the maximum number of iterations t has been reached max , the update stops.

[0238] After reaching the maximum number of iterations, the decision variable The value in it is the clear image after defogging processing. Subsequently, the system submits the optimized images of each agent for output and returns the processed clear images to the vehicle-mounted system for further downstream tasks, such as obstacle detection, path planning, or other autonomous driving-related applications.

[0239] Although each agent needs to store and update three variables in the algorithm: tentative solution, local gradient variable, and tracking variable, they actually only share two of these variables with neighboring nodes: the tentative solution and the tracking variable while the local gradient is always stored locally as a private variable. This approach is similar to only processing the local objective function and local constraint set locally, thus significantly reducing the demand for computing resources.

[0240] Embodiment 17:

[0241] An optical sensor defogging method for autonomous driving based on multi-agent optimization specifically includes the following parts:

[0242] S1. Receive the input image and perform image preprocessing; S2. Build a communication network and initialize the variables; S3. Optimize the image using the proposed algorithm; S4. Output the optimized clear image.

[0243] Specifically, in step S1, after the system receives the foggy image collected by the vehicle-mounted optical sensor, it first performs preprocessing on the image, that is, normalizes the input image.

[0244] In step S2, first build a communication network, using an undirected connected network G = {V, E} with M agents, where the node set V = {1, 2, 3,..., M} and the edge set consists. (i, j) in the edge set E means that agent i can receive information from agent j. V = {1, 2, 3,..., M} represents the set of agent nodes, represents the edge set; represents the weight matrix of the network. If a ij > 0, it means that the agents are connected. If a ij = 0, it means that the agents are not connected; this weight matrix also has the property of non-negative double stochastic, that is, A1 M =

[0245] 1, In addition, represents all the neighboring nodes of the node. Then, initialize the agent variables, set the tentative solution local gradient variable tracking variable Set a maximum number of iterations \(t\) again. max As the criterion for the algorithm to stop iterative update, the initial number of iterations \(t\) is set to 1. If the current number of iterations \(t < t\) max , then continue the algorithm; if the current number of iterations \(t\geq t\) max , then stop the iteration.

[0246] In step 3, use the proposed algorithm for iterative update. Specifically, first update the weighted average of the tentative solution , update the weighted average of the local gradient variable . Through , agent \(i\) collects the local gradient variables of neighbor nodes and calculates the weighted average to obtain an approximation of the global gradient using neighbor information; then update the tentative solution of the decision variable: Then update the local gradient variable: Finally, update the tracking variable:

[0247] Output the optimized image stored in the decision variable \(z\) i to the vehicle system.

[0248] Combined with Appendix Figure 3 , 4 , 5, in this embodiment, the specific steps of the method provided by the present invention are as follows:

[0249] (1). Define 5 agents, input a clear image with a size of 100×100, and process the original image through the Gaussian blur convolution kernel \(H\) to simulate the foggy image received by the vehicle-mounted optical sensor.

[0250] (2). Build a 5×5 doubly stochastic weight matrix \(A\), define the maximum number of iterations \(t\) max = 100000, initialize the current number of iterations \(t = 1\), and the hyperparameter \(l = 0.08\). Initialize as a 28×28 random matrix, and let

[0251] Set the loss function as: This function is the local objective function, where \(H\) is the Gaussian convolution kernel, \(G\) is the blurred image, \(\theta\) is the regularization parameter, set to 0.005, and \(M\) is the number of agents, which is 5 here. This problem can be expressed as:

[0252]

[0253] Update according to the algorithm steps and

[0254] Update the tentative solution of the decision variable Update the local gradient variable: Finally, update the tracking variable:

[0255] After reaching the maximum number of iterations, output the optimized image, as Figure 3 shown.

[0256] In this embodiment, PSNR, MSE, and SSIM are used to quantify the optimization results. MSE (Mean Squared Error) is the mean of the squares of the differences between two images, representing the average degree of pixel differences. The smaller the MSE, the more similar the two images are and the smaller the error. As can be seen from Figure 4 it, the MSE value in this embodiment converges to a relatively small value, in the interval (10 -2 , 10 -3 ), indicating that the optimization result is highly close to the reference image. PSNR (Peak Signal-to-Noise Ratio) is a measure based on MSE, used to describe the ratio of the peak of the image signal to the noise, expressed in logarithmic form, and is commonly used to measure the quality of reconstructed or compressed images. The larger the PSNR, the higher the image quality and the closer it is to the reference image. According to Figure 5 , the PSNR value of this embodiment converges to a relatively large value, further verifying the effectiveness of the optimization. SSIM (Structural Similarity Index Measure) measures the similarity between two images from three aspects: structure, brightness, and contrast, which is closer to the perceptual characteristics of the human eye. The range of the SSIM value is [0, 1], and 1 means that the two images are exactly the same. According to Figure 6 it, it can be seen that the SSIM value of this embodiment is close to 1. Through the comprehensive evaluation of the above three indicators, the optimization result of this embodiment shows high quality in terms of pixel error, signal quality, and structural similarity, and has a high degree of matching with the original clear image, verifying the effectiveness of the defogging system based on this multi-agent optimization method.

Claims

1. An optical sensor defogging method for autonomous driving based on multi-agent optimization, characterized in that: The following steps are involved: 1) Using the vehicle-mounted optical sensor to collect the fog image and perform image preprocessing to obtain the initial fog image; 2) Build a multi-agent based communication network and set the communication network parameters; 3) Use the communication network to optimize the initial fogged image, obtain the dehazed image, and output it.

2. According to claim 1, a multi-agent optimization-based optical sensor defogging method for autonomous driving is characterized in that: The step of image preprocessing includes normalization.

3. The optical sensor defogging method for autonomous driving based on multi-agent optimization according to claim 1, characterized in that: The communication network includes M intelligent agents, each of which serves as a node; M agents communicate through a time-invariant undirected graph network G = {V, E}; V = {1, 2, 3, ..., M} is a set of nodes; is an edge set; (i, j) in the edge set E indicates that agent i can receive information from agent j.

4. The optical sensor defogging method for autonomous driving based on multi-agent optimization according to claim 1, characterized in that: The objective function minf(z) of the communication network is as follows: In the formula, f i (z) is the local objective function of agent i.

5. The optical sensor defogging method for autonomous driving based on multi-agent optimization according to claim 4, characterized in that: In the communication network, the local objective function f of agent i is i (z) is as follows: f i (z i )=||H i ×z i -b i || 2 (2) Among them, H i is the blur convolution kernel; z i is the observation value of agent i; b i It's an error.

6. The optical sensor defogging method for autonomous driving based on multi-agent optimization according to claim 1, characterized in that: In step 2), the step of setting the communication network parameters includes: 2.1) Initialize the tentative solution Local gradient variables Tracking variables 2.2) Set the weight matrix A, A is a non-negative doubly random matrix, the dimension of A is consistent with the dimension of the communication connectivity graph of the agent, generally an M-order square matrix, where M is the number of agents mentioned above. 2.3) Set the number of iterations to t max .

7. The optical sensor defogging method for autonomous driving based on multi-agent optimization according to claim 1, characterized in that: In step 3), the step of optimizing the initial fogged image using the communication network includes: 3.1) Initialize the number of iterations t = 1; 3.2) Update tentative solution The weighted average is the decision variable of the neighbor node j of agent i a ij is the weight; is the set of neighbor nodes of agent i; 3.3) Update local gradient variables The weighted average is the local gradient variable of agent j; 3.4) Update the tentative solution of decision variables Right now: Where l>0 is the regularization coefficient; 3.5) Update local gradient variables Right now: 3.6) Update tracking variables Right now: 3.7) Determine if t>t jax Is it true? If not, set t = t + 1 and return to step 3.2). If so, use the decision variable Dehazed image as optimized by agent i.

8. The optical sensor defogging method for autonomous driving based on multi-agent optimization according to claim 7, characterized in that: Weight a ij Satisfy the following formula:

9. A system using the vehicle-mounted optical sensor defogging method according to any one of claims 1 to 8, characterized in that: Includes on-board optical sensors, high-performance concurrent processing units, and on-board computers; The vehicle-mounted optical sensor collects fog images and transmits them to a high-performance concurrent processing unit; The high-performance concurrent processing unit optimizes the initial fogged image using the communication network to obtain a defogged image and transmits it to the on-board computer; The high-performance concurrent processing unit includes an image acquisition module, an image preprocessing module, an agent network module and an image output module; The image acquisition module is used to receive the fogged image from the vehicle-mounted optical sensor; The image preprocessing module is used to perform normalization processing on the atomized image to obtain an initial atomized image; The agent network module includes M agents, which are used to optimize the initial fogged image to obtain a defogged image; The image output module is used to transmit the defogging image to the on-board computer; The on-board computer performs post-processing on the defogging image.

10. The system according to claim 9, characterized in that The agent network module includes a communication network building unit, a variable initialization unit, a tentative solution weight average updating unit, a local gradient variable weight average updating unit, a decision variable tentative solution updating unit, a local gradient variable updating unit, and a tracking variable updating unit; The communication network building unit is used to build a communication connection matrix and define a dual random connectivity weight matrix; The variable initialization unit is used to initialize the tentative solution Local gradient variables Tracking variables The tentative solution weight average updating unit is used to update the tentative solution weight average The local gradient variable weight average updating unit is used to update the local gradient variable weight average The decision variable tentative solution updating unit is used to update the decision variable tentative solution The local gradient variable updating unit is used to update the local gradient variable The tracking variable updating unit is used to update the tracking variable

Images