Rearview mirror image enhancement method and system

Abstract

The invention discloses a rearview mirror image enhancement method and system. The rearview mirror image enhancement method comprises the steps of obtaining a visible light image shot by a camera; inputting the visible light image into a pre-trained image processing model to obtain an enhanced image; wherein the image processing model at least comprises a simulation infrared network and an image fusion algorithm module, infrared simulation processing is carried out on the visible light image through the simulation infrared network to obtain a simulation infrared image, and the visible light image and the simulation infrared image are fused through the image fusion algorithm module to obtain an enhanced image; and displaying the enhanced image on a display screen of the vehicle machine. The visible light image is processed into the simulation infrared image through the image processing model, and then the simulation infrared image is fused with the visible light image, so that the purpose of enhancing the visible light image is achieved, the quality of the image displayed by the display screen can be effectively improved, a high-quality view environment is provided for a driver, and driving safety can be well achieved.

Description

Technical Field

[0001] This application relates to the field of automotive electronics, and particularly to a method and system for enhancing rearview mirror images. Background Art

[0002] CMS (Camera Monitor System, automotive electronic rearview mirror) is an important research direction in the current field of automotive electronics. Compared with traditional rearview mirrors, the CMS system can expand the driver's field of vision, reduce blind spots and visual dead angles, and its effect is more significant especially at night or under bad weather conditions, significantly improving driving safety.

[0003] Currently, the mainstream method for image enhancement based on CMS only provides the visible light image information captured by the camera for the driver. However, the single image information is limited, and more image information cannot be obtained for image enhancement, resulting in the inability to display a better-quality image on the in-vehicle display screen. Secondly, in the recently emerging CMS system, an infrared lens module and a visible light lens module are respectively used to capture visible light images and infrared images, and then the visible light images and infrared images are fused to improve the quality of the displayed images. However, the resolution of the infrared images captured by the camera is low, resulting in a low quality of the fused images, and further causing the inability to display a better-quality image on the in-vehicle display screen. Summary of the Invention

[0004] Based on the above deficiencies of the prior art, this application provides a method and system for enhancing rearview mirror images to solve the problem that a better-quality image cannot be displayed on the in-vehicle display screen.

[0005] To achieve the above object, this application provides the following technical solutions:

[0006] The first aspect of this application provides a method for enhancing rearview mirror images, including:

[0007] Obtain the visible light image captured by the camera;

[0008] Input the visible light image into a pre-trained image processing model to obtain an enhanced image; wherein, the image processing model at least includes a simulated infrared network and an image fusion algorithm module. The visible light image is subjected to infrared simulation processing through the simulated infrared network to obtain a simulated infrared image, and the visible light image and the simulated infrared image are fused through the image fusion algorithm module to obtain the enhanced image; the image processing model is pre-trained using sample visible light images.

[0009] Display the enhanced image on the in-vehicle display screen.

[0010] Optionally, in the above method for enhancing the rearview mirror image, the training method of the image processing model includes:

[0011] Obtain the sample visible light image captured by the camera;

[0012] Input the sample visible light image into the image processing model to obtain a sample enhanced image;

[0013] Determine whether the sample enhanced image meets the preset enhancement requirements;

[0014] If the sample enhanced image meets the preset enhancement requirements, determine the image processing model as the trained image processing model;

[0015] If the sample enhanced image does not meet the preset enhancement requirements, use the sample visible light image as the input of the loss function of the image processing model to obtain the sample enhanced image, and continuously adjust the parameters of the image processing model until the sample enhanced image meets the preset enhancement conditions, and confirm that the image processing model is successfully trained.

[0016] Optionally, in the above method for enhancing the rearview mirror image, the image processing model includes a simulation infrared network and an image fusion algorithm module. The step of inputting the sample visible light image into the image processing model to obtain a sample enhanced image includes:

[0017] Perform infrared simulation processing on the sample visible light image through the simulation infrared network to obtain a sample simulated infrared image;

[0018] Fuse the sample simulated infrared image and the sample visible light image through the image fusion algorithm module to obtain a sample enhanced image.

[0019] Optionally, in the above method for enhancing the rearview mirror image, the simulation infrared network includes an encoder, a converter, a decoder, and a discriminator. The step of performing infrared simulation processing on the sample visible light image through the simulation infrared network to obtain a sample simulated infrared image includes:

[0020] Perform downsampling processing on the sample visible light image through the encoder to obtain a feature map corresponding to the sample visible light image;

[0021] Perform convolution processing on the feature map through the converter to obtain infrared image features corresponding to the feature map;

[0022] Perform decoding processing on the infrared features through the decoder to obtain a decoded simulated infrared image;

[0023] Determine whether the decoded simulated infrared image meets the requirements of a real infrared image through the discriminator;

[0024] If the decoded simulated infrared image meets the requirements of a real infrared image, determine the decoded simulated infrared image as the sample simulated infrared image;

[0025] If the decoded simulated infrared image does not meet the requirements of a real infrared image, adjust the parameters of the simulated infrared network, and return to perform the downsampling process on the sample visible light image through the encoder to obtain the feature map corresponding to the sample visible light image until the decoded simulated infrared image meets the requirements of a real infrared image.

[0026] Optionally, in the above method for enhancing a rearview mirror image, the step of fusing the sample simulated infrared image and the sample visible light image through the image fusion algorithm module to obtain a sample enhanced image includes:

[0027] Construct an expression for an ideal fused image through the image fusion algorithm module;

[0028] Construct a target expression for the brightness of the visible light image based on the sample visible light image through the image fusion algorithm module;

[0029] Construct a linear regression equation through the image fusion algorithm module according to the target expression, the sample simulated infrared image, and the gain coefficient, and convert the linear regression equation into a matrix form to obtain a matrix equation;

[0030] Perform degradation processing on the ideal fused image, the sample visible light image, and the sample simulated infrared image through the image fusion algorithm module to obtain a degraded ideal fused image, a degraded sample visible light image, and a degraded sample simulated infrared image;

[0031] Rewrite the matrix equation according to the degraded ideal fused image, the degraded sample visible light image, and the degraded sample simulated infrared image through the image fusion algorithm module to obtain a target matrix equation;

[0032] Solve the target matrix equation through the image fusion algorithm module using the iterative weighted least squares method to obtain a target value, and solve the expression of the ideal fused image based on the target value to obtain the sample enhanced image.

[0033] Optionally, in the above method for enhancing a rearview mirror image, the step of solving the expression of the ideal fused image based on the target value to obtain the sample enhanced image includes:

[0034] Substitute the target value into the linear regression equation to obtain a linear regression expression;

[0035] Substitute the linear regression expression into the target expression of the visible light image brightness to obtain the visible light image brightness;

[0036] Substitute the visible light image brightness into the expression of the ideal fusion image for solution to obtain a sample enhanced image.

[0037] Optionally, in the above method for enhancing the rearview mirror image, the use of the iterative weighted least squares method to solve the target matrix equation to obtain a target value includes:

[0038] Calculate an initial estimate of the target matrix equation using the iterative weighted least squares method;

[0039] Calculate a residual based on the initial estimate, and calculate a weight matrix based on the residual to obtain a weighted solution;

[0040] Solve the target matrix equation according to the weighted solution to obtain a target value.

[0041] Optionally, in the above method for enhancing the rearview mirror image, solving the target matrix equation according to the weighted solution to obtain a target value includes:

[0042] Substitute the weighted solution into the target matrix equation to obtain a new residual;

[0043] Update the weight matrix according to the new residual to obtain a solution of weighted least squares;

[0044] Determine whether the solution of weighted least squares converges;

[0045] If the solution of weighted least squares does not converge, iterate the solution of weighted least squares and record the current iteration number;

[0046] Determine whether the current iteration number meets a preset iteration number;

[0047] If the current iteration number does not meet the preset iteration number, use the solution of weighted least squares as the initial estimate and return to execute calculating the residual based on the initial estimate until the solution of weighted least squares converges;

[0048] If the solution of weighted least squares converges, or the current iteration number meets the preset iteration number, substitute the solution of weighted least squares into the target matrix equation for solution to obtain a target value.

[0049] The second aspect of the present application provides an enhancement system for rearview mirror images, including:

[0050] An image acquisition unit for acquiring visible light images captured by a camera;

[0051] An image input unit for inputting the visible light image into a pre-trained image processing model to obtain an enhanced image; wherein, the image processing model at least includes a simulated infrared network and an image fusion algorithm module, performs infrared simulation processing on the visible light image through the simulated infrared network to obtain a simulated infrared image, and fuses the visible light image and the simulated infrared image through the image fusion algorithm module to obtain the enhanced image; the image processing model is pre-trained using sample visible light images.

[0052] A display unit for displaying the enhanced image on the in-vehicle display screen.

[0053] Optionally, in the above-mentioned enhancement system for rearview mirror images, it further includes:

[0054] An acquisition unit for acquiring sample visible light images captured by the camera;

[0055] An input unit for inputting the sample visible light image into the image processing model to obtain a sample enhanced image;

[0056] A requirement judgment unit for judging whether the sample enhanced image meets the preset enhancement requirements;

[0057] A model determination unit for, if the sample enhanced image meets the preset enhancement requirements, determining the image processing model as the trained image processing model;

[0058] An adjustment unit for, if the sample enhanced image does not meet the preset enhancement requirements, taking the sample visible light image as the input of the loss function of the image processing model to obtain the sample enhanced image, and continuously adjusting the parameters of the image processing model until the sample enhanced image meets the preset enhancement conditions, and confirming that the image processing model is successfully trained.

[0059] Optionally, in the above-mentioned enhancement system for rearview mirror images, the image processing model includes a simulated infrared network and an image fusion algorithm module, and the input unit includes:

[0060] An infrared processing unit for performing infrared simulation processing on the sample visible light image through the simulated infrared network to obtain a sample simulated infrared image;

[0061] A fusion unit for fusing the sample simulated infrared image and the sample visible light image through an image fusion algorithm module to obtain a sample enhanced image.

[0062] Optionally, in the above rearview mirror image enhancement system, the simulation infrared network includes an encoder, a converter, a decoder, and a discriminator. The infrared processing unit includes:

[0063] A downsampling unit for downsampling the sample visible light image through the encoder to obtain a feature map corresponding to the sample visible light image;

[0064] A convolution processing unit for convolving the feature map through the converter to obtain infrared image features corresponding to the feature map;

[0065] An upsampling unit for decoding the infrared features through the decoder to obtain a decoded simulated infrared image;

[0066] A judgment unit for judging whether the decoded simulated infrared image meets the requirements of a real infrared image through the discriminator;

[0067] An image determination unit for determining the decoded simulated infrared image as the sample simulated infrared image if the decoded simulated infrared image meets the requirements of a real infrared image;

[0068] An execution unit for adjusting the parameters of the simulation infrared network and returning to execute the downsampling of the sample visible light image through the encoder to obtain a feature map corresponding to the sample visible light image if the decoded simulated infrared image does not meet the requirements of a real infrared image until the decoded simulated infrared image meets the requirements of a real infrared image.

[0069] Optionally, in the above rearview mirror image enhancement system, the fusion unit includes:

[0070] A construction unit for constructing an expression of an ideal fusion image through the image fusion algorithm module;

[0071] An expression construction unit for constructing a target expression of the visible light image brightness based on the sample visible light image through the image fusion algorithm module;

[0072] A conversion unit for constructing a linear regression equation according to the target expression, the sample simulated infrared image, and a gain coefficient through the image fusion algorithm module, and converting the linear regression equation into a matrix form to obtain a matrix equation;

[0073] A degradation processing unit, configured to perform degradation processing on the ideal fusion image, the sample visible light image, and the sample simulated infrared image through the image fusion algorithm module to obtain a degraded ideal fusion image, a degraded sample visible light image, and a degraded sample simulated infrared image;

[0074] A rewriting unit, configured to rewrite the matrix equation according to the degraded ideal fusion image, the degraded sample visible light image, and the degraded sample simulated infrared image through the image fusion algorithm module to obtain a target matrix equation;

[0075] A first solving unit, configured to solve the target matrix equation by using the iterative weighted least squares method through the image fusion algorithm module to obtain a target value, and solve the expression of the ideal fusion image based on the target value to obtain a sample enhanced image.

[0076] Optionally, in the above-mentioned rearview mirror image enhancement system, the first solving unit includes:

[0077] A first substitution unit, configured to substitute the target value into the linear regression equation to obtain a linear regression expression;

[0078] A second substitution unit, configured to substitute the linear regression expression into the target expression of the visible light image brightness to obtain the visible light image brightness;

[0079] A second solving unit, configured to substitute the visible light image brightness into the expression of the ideal fusion image for solution to obtain a sample enhanced image.

[0080] Optionally, in the above-mentioned rearview mirror image enhancement system, the first solving unit includes:

[0081] An estimated value calculation unit, configured to calculate an initial estimated value of the target matrix equation by using the iterative weighted least squares method;

[0082] A residual calculation unit, configured to calculate a residual according to the initial estimated value, and calculate a weight matrix based on the residual to obtain a weighted solution;

[0083] A third solving unit, configured to solve the target matrix equation according to the weighted solution to obtain a target value.

[0084] Optionally, in the above-mentioned rearview mirror image enhancement system, the third solving unit includes:

[0085] A third substitution unit, configured to substitute the weighted solution into the target matrix equation to obtain a new residual;

[0086] An update unit, configured to update the weight matrix according to the new residual to obtain a solution of weighted least squares;

[0087] A convergence judgment unit, configured to judge whether the solution of weighted least squares converges;

[0088] An iteration unit, configured to, if the solution of weighted least squares does not converge, perform iteration on the solution of weighted least squares and record the current iteration count;

[0089] A count judgment unit, configured to judge whether the current iteration count meets a preset iteration count;

[0090] A first acting unit, configured to, if the current iteration count does not meet the preset iteration count, use the solution of weighted least squares as the initial estimate value and return to execute calculating the residual according to the initial estimate value until the solution of weighted least squares converges;

[0091] A fourth solving unit, configured to, if the solution of weighted least squares converges or the current iteration count meets the preset iteration count, substitute the solution of weighted least squares into the target matrix equation for solving to obtain a target value.

[0092] An enhancement method for a rearview mirror image provided by the present application. By acquiring a visible light image captured by a camera and then inputting the visible light image into a pre-trained image processing model to obtain an enhanced image. The image processing model at least includes a simulation infrared network and an image fusion algorithm module. The visible light image is subjected to infrared simulation processing through the simulation infrared network to obtain a simulated infrared image, and the visible light image and the simulated infrared image are fused through the image fusion algorithm module to obtain an enhanced image. The image processing model is pre-trained using sample visible light images. Finally, the enhanced image is displayed on the in-vehicle display screen. Thus, the visible light image is first processed into a simulated infrared image by the image processing model and then fused with the visible light image to achieve the purpose of enhancing the visible light image, which can effectively improve the quality of the image displayed on the display screen, and further provide a high-quality visual environment for the driver, and can better achieve driving safety. Description of the Drawings

[0093] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0094] Figure 1Schematic structural diagram of a rearview mirror image enhancement system provided by an embodiment of the present application;

[0095] Figure 2 Schematic flow diagram of a method for enhancing a rearview mirror image provided by an embodiment of the present application;

[0096] Figure 3 Schematic flow diagram of a method for training an image processing model provided by an embodiment of the present application;

[0097] Figure 4 Schematic flow diagram of a method for obtaining a sample enhanced image provided by an embodiment of the present application;

[0098] Figure 5 Schematic structural diagram of a simulation infrared network provided by an embodiment of the present application;

[0099] Figure 6 Schematic structural diagram of a generator network provided by an embodiment of the present application;

[0100] Figure 7 Schematic flow diagram of a method for obtaining a sample simulated infrared image provided by an embodiment of the present application;

[0101] Figure 8 Schematic structural diagram of a discriminator network provided by an embodiment of the present application;

[0102] Figure 9 Schematic flow diagram of a method for obtaining a sample enhanced image provided by an embodiment of the present application;

[0103] Figure 10 Schematic flow diagram of a method for calculating a target value provided by an embodiment of the present application;

[0104] Figure 11 Schematic flow diagram of a method for obtaining a sample enhanced image provided by an embodiment of the present application;

[0105] Figure 12 Schematic structural diagram of a rearview mirror image enhancement system provided by another embodiment of the present application. Detailed implementation manners

[0106] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0107] In this application, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0108] An embodiment of this application provides a method for enhancing a rearview mirror image, which is applied to an enhancement system, that is, a CMS image enhancement system, to solve the problem that the in-vehicle display screen cannot display images with good quality.

[0109] Therefore, Figure 1 As shown, an embodiment of this application provides a rearview mirror image enhancement system, including: a power supply, a visible light camera, an electronic control unit, an image processing unit, and a display.

[0110] Among them, the power supply is used to provide voltage and current for the CMS image enhancement system.

[0111] The visible light camera is used to capture visible light images and send them to the electronic control unit.

[0112] The electronic control unit is used to send the visible light images to the image processing unit for image processing, and to help manage various electronic devices of the vehicle to ensure that each system can work efficiently and safely in cooperation, improving the performance, comfort and safety of the vehicle.

[0113] The image processing unit is used to perform simulated infrared processing and fusion processing on the visible light images to achieve the purpose of enhancing the images, and send the enhanced images to the display.

[0114] The display is used to show the enhanced images to the driver to assist the driver in driving on the road.

[0115] Therefore, based on the above-provided rearview mirror image enhancement system, an embodiment of this application correspondingly provides a method for enhancing a rearview mirror image, as Figure 2 shown, specifically including the following steps:

[0116] S201. Obtain the visible light images captured by the camera.

[0117] Specifically, a visible light camera can be used to capture the vision images in front of, on the side of, or behind the vehicle, that is, visible light images. Then, a signal can be sent to the system through an electronic control unit so that the system can obtain the visible light images for subsequent processing.

[0118] S202. Input the visible light image into a pre-trained image processing model to obtain an enhanced image.

[0119] It should be noted that the image processing model can include a simulated infrared network and an image fusion algorithm module. The visible light image is processed by the simulated infrared network to obtain a simulated infrared image, and the visible light image and the simulated infrared image are fused by the image fusion algorithm module to obtain an enhanced image. The image processing model is pre-trained using sample visible light images.

[0120] It should also be noted that the essence of the image processing model is a multiple iterative linear regression model. Therefore, in the embodiments of the present application, the image processing model can be trained based on a neural network mechanism to obtain a training result.

[0121] Optionally, an embodiment of the present application provides a method for training an image processing model, as Figure 3 shown, including the following steps:

[0122] S301. Obtain sample visible light images captured by a camera.

[0123] It can be understood that in order for the visible light image to be enhanced by the image processing model to obtain an enhanced image, it is necessary to first train and optimize the image processing model so that the output image can achieve a high-quality enhancement effect. Therefore, in the embodiments of the present application, historical visible light images captured by a camera can be obtained as sample visible light images.

[0124] S302. Input the sample visible light image into the image processing model to obtain a sample enhanced image.

[0125] It should be noted that to construct an image processing model with a specific enhancement effect, it is necessary to build a simulated infrared network and an image fusion algorithm module to enhance the input sample visible light image, thereby training the image processing model to obtain a sample enhanced image.

[0126] So, optionally, in another embodiment of the present application, the image processing model includes a simulated infrared network and an image fusion algorithm module. Correspondingly, in the embodiments of the present application, a specific implementation manner of step S302, as Figure 4 shown, includes the following steps:

[0127] S401. Perform infrared simulation processing on the sample visible light image through the simulation infrared network to obtain the sample simulated infrared image.

[0128] It should be noted that during the training process of the simulation infrared network, the objective function used is:

[0129]

[0130] Among them, V is the total loss function, G is the generator, and D is the discriminator. represents the probability that the discriminator judges the real data as a true sample. represents the probability that the discriminator judges the sample generated by the generator as a false sample. The generative adversarial network optimizes and minimizes the loss function to achieve network optimization. During the training process, the Adam optimizer is used, appropriate learning rates and epochs are set, and the generator and discriminator are alternately trained, that is, first fix G and maximize D, and then fix D and minimize G to obtain the optimal solution.

[0131] It should also be noted that the simulation infrared network is a generative adversarial network (GAN), and its specific structure is as Figure 5 shown. It mainly includes a generator and a discriminator. Its principle takes the visible light image as input and generates a simulated infrared image through the generator. It should also be noted that to build the network structure of the generator, an encoder, a converter, and a decoder are required. So optionally, in another embodiment of the present application, as Figure 6 shown, the simulation infrared network may include an encoder, a converter, a decoder, and a discriminator. Correspondingly, in the embodiment of the present application, a specific implementation manner of step S401 is as Figure 7 shown and includes the following steps:

[0132] S701. Perform downsampling processing on the sample visible light image through the encoder to obtain the feature map corresponding to the sample visible light image.

[0133] Specifically, the encoder is composed of 3 convolutional layers, and each convolutional layer is composed of a convolutional kernel with a size of and a ReLU function. The stride of each convolutional kernel is 2 to achieve the downsampling operation. It should be noted that through the encoder, on the one hand, the size of the feature map can be reduced, reducing the network calculation amount, and on the other hand, the main feature information of the image can be retained, reducing the reconstruction error. Therefore, in the embodiment of the present application, the sample visible light image can be downsampled through the encoder to obtain the feature map corresponding to the sample visible light image, that is, the low-dimensional feature information of the image is included.

[0134] S702. Perform convolutional processing on the feature map through a converter to obtain the infrared image features corresponding to the feature map.

[0135] It can be understood that the converter is composed of N residual blocks to avoid the phenomenon of large training errors caused by too deep network layers. Moreover, convolutional processing is performed on the feature map through the converter to convert the information of the feature map into the information of infrared image features.

[0136] S703. Perform decoding processing on the infrared features through a decoder to obtain a decoded simulated infrared image.

[0137] Specifically, the decoder is composed of 3 deconvolutional layers corresponding to the encoder. Its purpose is to implement the upsampling process, retain the feature information while restoring the image size. Therefore, after obtaining the infrared image features, it is necessary to perform decoding processing on the infrared features through the decoder, that is, upsampling processing, so as to achieve the restoration of features and the generation of images, and thus a decoded simulated infrared image will be obtained.

[0138] S704. Determine whether the decoded simulated infrared image meets the requirements of a real infrared image through a discriminator.

[0139] It should be noted that the discriminator is a fully convolutional network composed of M convolutional layers. As Figure 8 shown, similar to the generator, each convolutional layer is composed of a convolutional kernel of size and a ReLU function, and the last layer is a Sigmoid function, that is, to implement the task of binary classification.

[0140] In addition, it should also be noted that the generator, through continuous training and optimization, makes the generated infrared simulated image as close as possible to the real high-resolution infrared image, so that the discriminator cannot distinguish between the generated infrared simulated image and the real infrared image. And through continuous training and optimization, the discriminator can learn the features of the real infrared image, improve the judgment ability, and accurately identify the real infrared image as much as possible. That is, through the mutual game between the generator and the discriminator, the purpose of a high-resolution simulated infrared image is achieved.

[0141] Therefore, after the decoded simulated infrared image is output by the generator, it is necessary to use the discriminator to judge the authenticity of the decoded simulated infrared image to obtain a high-resolution simulated infrared image. Therefore, the discriminator will judge whether the decoded simulated infrared image meets the requirements of a real infrared image. If the decoded simulated infrared image meets the requirements of a real infrared image, step S705 is executed. If the decoded simulated infrared image does not meet the requirements of a real infrared image, step S706 is executed.

[0142] S705. Determine the decoded simulated infrared image as the sample simulated infrared image.

[0143] Specifically, when the decoded simulated infrared image meets the requirements of the real infrared image, it indicates that the discriminator determines that the generated decoded simulated infrared image is true, that is, the decoded simulated infrared image is a high-resolution simulated infrared image. Then, at this time, the decoded simulated infrared image (i.e., the sample simulated infrared image) can be sent to the image fusion algorithm module for subsequent fusion processing.

[0144] S706. Adjust the parameters of the simulated infrared network.

[0145] It can be understood that when the decoded simulated infrared image does not meet the requirements of the real infrared image, it means that the discriminator determines that the generated decoded simulated infrared image is false, that is, it indicates that the decoded simulated infrared image generated by the generator fails to reach the high-resolution simulated infrared image. Therefore, continuous training and optimization are still needed. So, it is necessary to continuously adjust the parameters of the simulated infrared network and return to execute step S701 to continue training the generator and the discriminator until the decoded simulated infrared image meets the requirements of the real infrared image.

[0146] S402. Through the image fusion algorithm module, fuse the sample simulated infrared image and the sample visible light image to obtain a sample enhanced image.

[0147] It can be understood that the image fusion algorithm module is an image fusion algorithm. This algorithm can supplement the missing brightness information of the visible light image and then fuse it with the sample simulated infrared image, so as to achieve the image enhancement effect and obtain the sample enhanced image.

[0148] Optionally, in another embodiment of the present application, a specific implementation manner of step S402 is as Figure 9 shown, including the following steps:

[0149] S901. Through the image fusion algorithm module, construct an expression of an ideal fusion image.

[0150] Specifically, the expression for constructing the ideal fusion image is: .

[0151] Wherein, represents the ideal fusion image, represents the sample visible light image, represents the sample simulated infrared image, and the subscript represents the th channel of the image, represents the gain coefficient of each channel, represents the brightness of the visible light image.

[0152] S902. Based on the sample visible light images, construct the target expression of the visible light image brightness through the image fusion algorithm module.

[0153] It can be understood that from the expression of the above ideal fusion image, and are variables to be solved. Therefore, to solve F (the ideal fusion image), it is necessary to first solve I RGB . Therefore, I RGB (the visible light image brightness) can be constructed through the sample visible light images.

[0154] Specifically, the target expression of the visible light image brightness is:

[0155]

[0156] Among them, represents the visible light image of the th channel, represents the coefficient of the th channel.

[0157] S903. According to the target expression, the sample simulated infrared images and the gain coefficients, construct a linear regression equation through the image fusion algorithm module, and convert the linear regression equation into a matrix form to obtain a matrix equation.

[0158] Specifically, to construct a linear regression equation, it is necessary to transform the expression of the ideal fusion image, that is, is denoted as the brightness detail , and the gain coefficient is modeled as a multiple linear regression model about , , . That is, the expression of the linear regression equation will be obtained:

[0159]

[0160] Then, it is also necessary to convert the linear regression equation into a matrix form to obtain a matrix equation, so as to make the calculation and solution of the regression problem more efficient. Especially in the case of more features, matrix operations are often faster and more convenient than calculating each coefficient step by step.

[0161] Therefore, the expression of the matrix equation is:

[0162]

[0163]

[0164] Among them, , denote a column vector with the three-channel elements rearranged denote a column vector with elements rearranged, where T represents the transpose of a matrix, and it can be seen from the expression of the matrix equation that the unknown quantity to be solved in this fusion process is .

[0165] S904. Degrade the ideal fusion image, the sample visible light image, and the sample simulated infrared image through the image fusion algorithm module to obtain a degraded ideal fusion image, a degraded sample visible light image, and a degraded sample simulated infrared image.

[0166] It should be noted that since the ideal fusion image is unknown, the brightness details are unknown and the gain coefficient cannot be obtained through the matrix equation . Therefore, in the embodiments of the present application, the gain coefficient can be calculated by first degrading the ideal fusion image, the sample visible light image, and the sample simulated infrared image . The degradation process refers to the operation of subjecting the image to MTF low-pass filtering. Among them, the low-pass filter uses a Gaussian low-pass filter that matches the MTF of, and its amplitude-frequency characteristic is only determined by the Gaussian standard deviation, which is expressed in the frequency domain as:

[0167]

[0168] where is the Nyquist frequency represents the MTF value corresponding to the image at the Nyquist frequency.

[0169] In addition, according to the matrix equation, it can be determined that the MTF low-pass filter is used in the embodiments of the present application. Therefore only the low-frequency part is retained after degradation by this filter, which is the image information of RGB, that is the image information of and are respectively denoted as and after degradation.

[0170] S905. Rewrite the matrix equation according to the degraded ideal fusion image, the degraded sample visible light image, and the degraded sample simulated infrared image through the image fusion algorithm module to obtain a target matrix equation.

[0171] It can be understood that after obtaining the degraded ideal fusion image, the degraded sample visible light image, and the degraded sample simulated infrared image, the matrix equation can be rewritten to obtain a target matrix equation, so as to solve and obtain , that is, the rewritten expression of the target matrix equation is: .

[0172] Among them, the subscript represents the degradation operation, , , represents the column vector of the rearrangement of three-channel elements, represents the column vector of the rearrangement of elements.

[0173] S906. Through the image fusion algorithm module, use the iterative weighted least squares method to solve the target matrix equation to obtain the target value, and based on the target value, solve the expression of the ideal fusion image to obtain the sample enhanced image.

[0174] Specifically, after obtaining the target matrix equation, the can be obtained by the iterative weighted least squares method, that is, the target value. Then, after obtaining the target value, it is necessary to first substitute into the matrix equation, and then through the linear regression equation, the calculation formula of the visible light image brightness, and the expression of the ideal fusion image in sequence, the sample enhanced image after the fusion of the sample simulated infrared image and the sample visible light image can be obtained.

[0175] Optionally, in another embodiment of the present application, a specific implementation manner of using the iterative weighted least squares method by the image fusion algorithm module in step S906 to solve the target matrix equation to obtain the target value is as Figure 10 shown, including the following steps:

[0176] S1001. Use the iterative weighted least squares method to calculate the initial estimate of the target matrix equation.

[0177] Specifically, the expression of the initial estimate is:

[0178]

[0179] Among them, the superscript represents the transpose of the matrix. Substituting into the target matrix equation can obtain the initial estimate of the brightness details, that is, .

[0180] S1002. According to the initial estimate, calculate the residual, and based on the residual, calculate the weight matrix to obtain the weighted solution.

[0181] Specifically, after obtaining the initial estimate, the residual of the brightness details can be calculated by virtue of the initial estimate, that is, .

[0182] Next, according to the residual the weight matrix can be calculated . The weight matrix is a diagonal matrix, and its diagonal elements are usually selected as functions of the residual, such as the reciprocal of the residual.

[0183] Therefore, according to the solution of weighted least squares can be obtained, that is, the weighted solution, which is: .

[0184] S1003. Solve the target matrix equation according to the weighted solution to obtain the target value.

[0185] It can be understood that the weighted solution means giving different weights to different terms in the equation, so as to adjust the influence of certain data through the weights, making the solution more in line with the actual needs or more representative. Therefore, after obtaining the weighted solution, the target matrix equation can be solved through the weighted solution, making the result more representative.

[0186] Optionally, in another embodiment of the present application, a specific implementation manner of step S1003 includes the following steps:

[0187] Substitute the weighted solution into the target matrix equation to obtain a new residual.

[0188] It can be understood that after obtaining the weighted solution, it is necessary to substitute it into the target matrix equation to calculate a new round of residuals, so as to update the weight matrix again. Therefore, the expression of the new residual is: .

[0189] Update the weight matrix according to the new residual to obtain the solution of weighted least squares.

[0190] Specifically, after obtaining a new round of residuals, it is necessary to update the weight matrix again based on the new residual, so as to obtain the solution of weighted least squares, making the solution of weighted least squares reach the optimal.

[0191] Judge whether the solution of weighted least squares converges.

[0192] It should be noted that in order to know whether the weighted solution is the optimal solution, a threshold can be set to know whether the weighted solution converges. Therefore, if the weighted solution is greater than the preset threshold, it is determined that the solution of weighted least squares converges. Therefore, the solution of weighted least squares can be substituted into the target matrix equation for solution to obtain the target value.

[0193] If the weighted solution is less than a preset threshold, it is determined that the solution of the weighted least squares has not converged. In addition, to monitor the convergence process, by recording the number of iterations each time, it can be learned whether the weighted solution gradually approaches the convergence condition. Therefore, when the solution of the weighted least squares has not converged, it is necessary to iterate the solution of the weighted least squares and record the current number of iterations.

[0194] Determine whether the current number of iterations meets the preset number of iterations.

[0195] It should be noted that to ensure obtaining the optimal weighted value to achieve a high-quality enhanced image, it can be learned whether the weighted iterative weighted least squares method has given the best weighted value by determining whether the current number of iterations meets the preset number of iterations. Therefore, if the current number of iterations does not meet the preset number of iterations, it means that the weighted iterative weighted least squares method has not given the best weighted value. Therefore, it is necessary to use the solution of the weighted least squares as the initial estimate value and return to execute step S1002.

[0196] If the current number of iterations meets the preset number of iterations, it means that the weighted iterative weighted least squares method has given the best weighted value. Therefore, at this time, the solution of the weighted least squares can be substituted into the target matrix equation for solution to obtain the target value.

[0197] Optionally, in another embodiment of the present application, in step S906, a specific implementation manner of obtaining the sample enhanced image by solving the expression of the ideal fusion image based on the target value through the image fusion algorithm module is as Figure 11 shown, including the following steps:

[0198] S1101. Substitute the target value into the linear regression equation to obtain the linear regression expression.

[0199] Specifically, the linear regression expression is: .

[0200] S1102. Substitute the linear regression expression into the target expression of the visible light image brightness to obtain the visible light image brightness.

[0201] Specifically, . After substituting the linear regression expression into the target expression of the visible light image brightness, the visible light image brightness can be solved. .

[0202] S1103. Substitute the visible light image brightness into the expression of the ideal fusion image for solution to obtain the sample enhanced image.

[0203] It can be understood that the expression of the ideal fusion image is , and at this time and are both known values, so the ideal fused image, that is, the image after fusing the infrared image and the visible light image, can be solved and obtained.

[0204] S303. Determine whether the sample enhanced image meets the preset enhancement requirements.

[0205] It should be noted that in the image enhancement task, in order for the image processing model to output a high-quality enhanced image, in the embodiments of the present application, by setting the enhancement requirements, it is determined whether the image processing model reaches the expected training goal, that is, to determine whether the sample enhanced image meets the preset enhancement requirements. If the sample enhanced image meets the preset enhancement requirements, it means that the sample enhanced image has reached the expected enhancement effect, so step S304 is executed at this time. If the sample enhanced image does not meet the preset enhancement requirements, it means that the sample enhanced image has not reached the expected enhancement effect, so step S305 is executed at this time.

[0206] S304. Determine the image processing model as the trained image processing model.

[0207] S305. Use the sample visible light image as the input of the loss function of the image processing model to obtain the sample enhanced image, and continuously adjust the parameters of the image processing model until the sample enhanced image meets the preset enhancement conditions, and confirm that the image processing model is trained successfully.

[0208] Specifically, when the sample enhanced image does not meet the preset enhancement requirements, it is necessary to continue training and optimizing the image processing model, that is, it is necessary to use the sample visible light image as the input of the loss function of the image processing model to obtain the sample enhanced image, and then continuously adjust the parameters of the image processing model according to the sample enhanced image until the sample enhanced image meets the preset enhancement conditions, so as to determine that the image processing model is trained successfully, and then it can be applied to the actual situation.

[0209] S203. Display the enhanced image on the in-vehicle display screen.

[0210] An enhancement method for rearview mirror images provided by the present application obtains a visible light image captured by a camera, and then inputs the visible light image into a pre-trained image processing model to obtain an enhanced image. The image processing model at least includes a simulated infrared network and an image fusion algorithm module. The visible light image is subjected to infrared simulation processing by the simulated infrared network to obtain a simulated infrared image, and the visible light image and the simulated infrared image are fused by the image fusion algorithm module to obtain an enhanced image. The image processing model is pre-trained using sample visible light images, and finally the enhanced image is displayed on the in-vehicle display screen. Thus, by first processing the visible light image into a simulated infrared image by the image processing model and then fusing it with the visible light image, the purpose of enhancing the visible light image is achieved, the quality of the image displayed on the display screen can be effectively improved, and a high-quality visual environment can be provided for the driver, so that driving safety can be better realized.

[0211] Another embodiment of the present application provides an enhancement system for rearview mirror images, as Figure 12 shown, including the following units:

[0212] An image acquisition unit 1201 for acquiring a visible light image captured by a camera.

[0213] An image input unit 1202 for inputting the visible light image into a pre-trained image processing model to obtain an enhanced image. The image processing model at least includes a simulated infrared network and an image fusion algorithm module. The visible light image is subjected to infrared simulation processing by the simulated infrared network to obtain a simulated infrared image, and the visible light image and the simulated infrared image are fused by the image fusion algorithm module to obtain an enhanced image. The image processing model is pre-trained using sample visible light images.

[0214] A display unit 1203 for displaying the enhanced image on the in-vehicle display screen.

[0215] It should be noted that the specific working processes of the above units in the embodiments of the present application may be correspondingly referred to the steps S201 to S203 in the above method embodiments, and will not be elaborated here.

[0216] Optionally, in an enhancement system for rearview mirror images provided by another embodiment of the present application, it further includes:

[0217] An acquisition unit for acquiring a sample visible light image captured by a camera.

[0218] An input unit for inputting the sample visible light image into the image processing model to obtain a sample enhanced image.

[0219] A requirement judgment unit for judging whether the sample enhanced image meets a preset enhancement requirement.

[0220] A model determination unit, configured to determine the image processing model as a trained image processing model if the sample enhanced image meets the preset enhancement requirements.

[0221] An adjustment unit, configured to use the sample visible light image as the input of the loss function of the image processing model to obtain a sample enhanced image if the sample enhanced image does not meet the preset enhancement requirements, and continuously adjust the parameters of the image processing model until the sample enhanced image meets the preset enhancement conditions, and confirm that the image processing model is successfully trained.

[0222] Optionally, in an enhancement system for rearview mirror images provided by another embodiment of the present application, the image processing model includes a simulation infrared network and an image fusion algorithm module, and the input unit includes:

[0223] An infrared processing unit, configured to perform infrared simulation processing on the sample visible light image through the simulation infrared network to obtain a sample simulated infrared image.

[0224] A fusion unit, configured to fuse the sample simulated infrared image and the sample visible light image through the image fusion algorithm module to obtain a sample enhanced image.

[0225] Optionally, in an enhancement system for rearview mirror images provided by another embodiment of the present application, the simulation infrared network includes an encoder, a converter, a decoder, and a discriminator, and the infrared processing unit includes:

[0226] A downsampling unit, configured to perform downsampling processing on the sample visible light image through the encoder to obtain a feature map corresponding to the sample visible light image.

[0227] A convolution processing unit, configured to perform convolution processing on the feature map through the converter to obtain infrared image features corresponding to the feature map.

[0228] An upsampling unit, configured to perform decoding processing on the infrared features through the decoder to obtain a decoded simulated infrared image.

[0229] A judgment unit, configured to judge whether the decoded simulated infrared image meets the requirements of a real infrared image through the discriminator.

[0230] An image determination unit, configured to determine the decoded simulated infrared image as the sample simulated infrared image if the decoded simulated infrared image meets the requirements of a real infrared image.

[0231] An execution unit, configured to adjust the parameters of the simulation infrared network if the decoded simulated infrared image does not meet the requirements of a real infrared image, and return to execute downsampling processing on the sample visible light image through the encoder to obtain a feature map corresponding to the sample visible light image until the decoded simulated infrared image meets the requirements of a real infrared image.

[0232] Optionally, in an enhancement system for a rearview mirror image provided by another embodiment of the present application, the fusion unit includes:

[0233] A construction unit, configured to construct an expression of an ideal fusion image through an image fusion algorithm module.

[0234] An expression construction unit, configured to construct a target expression of the visible light image brightness based on a sample visible light image through an image fusion algorithm module.

[0235] A conversion unit, configured to construct a linear regression equation through an image fusion algorithm module according to the target expression, a sample simulated infrared image, and a gain coefficient, and convert the linear regression equation into a matrix form to obtain a matrix equation.

[0236] A degradation processing unit, configured to perform degradation processing on the ideal fusion image, the sample visible light image, and the sample simulated infrared image through an image fusion algorithm module to obtain a degraded ideal fusion image, a degraded sample visible light image, and a degraded sample simulated infrared image.

[0237] A rewriting unit, configured to rewrite the matrix equation according to the degraded ideal fusion image, the degraded sample visible light image, and the degraded sample simulated infrared image through an image fusion algorithm module to obtain a target matrix equation.

[0238] A first solving unit, configured to solve the target matrix equation by using the iterative weighted least squares method through an image fusion algorithm module to obtain a target value, and solve the expression of the ideal fusion image based on the target value to obtain a sample enhanced image.

[0239] In an enhancement system for a rearview mirror image provided by another embodiment of the present application, the first solving unit includes:

[0240] A first substitution unit, configured to substitute the target value into the linear regression equation to obtain a linear regression expression.

[0241] A second substitution unit, configured to substitute the linear regression expression into the target expression of the visible light image brightness to obtain the visible light image brightness.

[0242] A second solving unit, configured to substitute the visible light image brightness into the expression of the ideal fusion image for solving to obtain a sample enhanced image.

[0243] In an enhancement system for a rearview mirror image provided by another embodiment of the present application, the first solving unit includes:

[0244] An estimated value calculation unit, configured to calculate an initial estimated value of the target matrix equation by using the iterative weighted least squares method.

[0245] A residual calculation unit, configured to calculate a residual according to an initial estimate value, and calculate a weight matrix based on the residual to obtain a weighted solution.

[0246] A third solving unit, configured to solve a target matrix equation according to the weighted solution to obtain a target value.

[0247] In an enhanced system for a rearview mirror image provided by another embodiment of the present application, the third solving unit includes:

[0248] A third substitution unit, configured to substitute the weighted solution into the target matrix equation to obtain a new residual.

[0249] An updating unit, configured to update the weight matrix according to the new residual to obtain a solution of weighted least squares.

[0250] A convergence judgment unit, configured to judge whether the solution of weighted least squares converges.

[0251] An iteration unit, configured to, if the solution of weighted least squares does not converge, iterate the solution of weighted least squares and record the current iteration number.

[0252] A number judgment unit, configured to judge whether the current iteration number meets a preset iteration number.

[0253] A first acting unit, configured to, if the current iteration number does not meet the preset iteration number, use the solution of weighted least squares as the initial estimate value, and return to execute calculating the residual according to the initial estimate value until the solution of weighted least squares converges.

[0254] A fourth solving unit, configured to, if the solution of weighted least squares converges, or the current iteration number meets the preset iteration number, substitute the solution of weighted least squares into the target matrix equation for solving to obtain a target value.

[0255] It should be noted that the specific working processes of the respective units provided in the above embodiments of the present application may correspondingly refer to the corresponding steps in the above method embodiments, and will not be elaborated herein.

[0256] It should also be noted that an enhanced system for a rearview mirror image provided by an embodiment of the present application has the technical effects of any one of the above embodiments, and will not be elaborated herein.

[0257] Those skilled in the art may further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0258] The above description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to these embodiments shown herein, but rather to the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for enhancing a rearview mirror image, characterized in that: include: Obtain the visible light image captured by the camera; The visible light image is input into a pre-trained image processing model to obtain an enhanced image; wherein the image processing model at least includes a simulated infrared network and an image fusion algorithm module, the visible light image is subjected to infrared simulation processing by the simulated infrared network to obtain a simulated infrared image, and the visible light image and the simulated infrared image are fused by the image fusion algorithm module to obtain the enhanced image; the image processing model is pre-trained using a sample visible light image; The enhanced image is displayed on the vehicle display screen.

2. The method according to claim 1, characterized in that The training method of the image processing model comprises: Acquire a sample visible light image captured by the camera; Inputting the sample visible light image into an image processing model to obtain a sample enhanced image; Determining whether the sample enhanced image meets the preset enhancement requirements; If the sample enhanced image meets the preset enhancement requirement, determining the image processing model as a trained image processing model; If the sample enhanced image does not meet the preset enhancement requirements, the sample visible light image is used as the input of the loss function of the image processing model to obtain the sample enhanced image, and the parameters of the image processing model are continuously adjusted until the sample enhanced image meets the preset enhancement conditions, confirming that the image processing model training is successful.

3. The method according to claim 1, characterized in that The image processing model includes a simulated infrared network and an image fusion algorithm module. The sample visible light image is input into the image processing model to obtain a sample enhanced image, including: Performing infrared simulation processing on the sample visible light image through the simulated infrared network to obtain a sample simulated infrared image; The sample simulated infrared image and the sample visible light image are fused through an image fusion algorithm module to obtain a sample enhanced image.

4. The method according to claim 3, characterized in that The simulated infrared network includes an encoder, a converter, a decoder and a discriminator. The simulated infrared network is used to perform infrared simulation processing on the sample visible light image to obtain a sample simulated infrared image, including: Down-sampling the sample visible light image by the encoder to obtain a feature map corresponding to the sample visible light image; Performing convolution processing on the feature map through the converter to obtain infrared image features corresponding to the feature map; Decoding the infrared feature by the decoder to obtain a decoded simulated infrared image; Determining whether the decoded simulated infrared image meets the requirements of a real infrared image by the discriminator; If the decoded simulated infrared image meets the requirements of a real infrared image, determining the decoded simulated infrared image as a sample simulated infrared image; If the decoded simulated infrared image does not meet the requirements of a real infrared image, the parameters of the simulated infrared network are adjusted, and the process returns to executing the downsampling process of the sample visible light image through the encoder to obtain a feature map corresponding to the sample visible light image, until the decoded simulated infrared image meets the requirements of a real infrared image.

5. The method according to claim 3, characterized in that: The step of fusing the sample simulated infrared image and the sample visible light image through an image fusion algorithm module to obtain a sample enhanced image includes: Constructing an expression of an ideal fused image through the image fusion algorithm module; Constructing a target expression for the brightness of the visible light image based on the sample visible light image through the image fusion algorithm module; The image fusion algorithm module constructs a linear regression equation according to the target expression, the sample simulated infrared image and the gain coefficient, and converts the linear regression equation into a matrix form to obtain a matrix equation; The ideal fused image, the sample visible light image and the sample simulated infrared image are subjected to degradation processing by the image fusion algorithm module to obtain a degraded ideal fused image, a degraded sample visible light image and a degraded sample simulated infrared image; Rewriting the matrix equation according to the degraded ideal fusion image, the degraded sample visible light image and the degraded sample simulated infrared image through the image fusion algorithm module to obtain a target matrix equation; The image fusion algorithm module uses iterative weighted least squares method to solve the target matrix equation to obtain a target value, and based on the target value, the expression of the ideal fused image is solved to obtain a sample enhanced image.

6. The method according to claim 5, characterized in that The step of solving the expression of the ideal fused image based on the target value to obtain a sample enhanced image includes: Substituting the target value into the linear regression equation to obtain a linear regression expression; Substituting the linear regression expression into the target expression of the visible light image brightness to obtain the visible light image brightness; Substitute the brightness of the visible light image into the expression of the ideal fused image and solve it to obtain a sample enhanced image.

7. The method according to claim 5, characterized in that The method of solving the target matrix equation by using iterative weighted least squares method to obtain the target value includes: Calculating an initial estimate of the target matrix equation using an iterative weighted least squares method; Calculating a residual according to the initial estimated value, and calculating a weight matrix based on the residual to obtain a weighted solution; The target matrix equation is solved according to the weighted solution to obtain the target value.

8. The method according to claim 7, characterized in that Solving the target matrix equation according to the weighted solution to obtain the target value includes: Substituting the weighted solution into the target matrix equation to obtain a new residual; According to the new residual, the weight matrix is ​​updated to obtain a weighted least squares solution; Determining whether the weighted least squares solution converges; If the weighted least squares solution does not converge, iterate the weighted least squares solution and record the current number of iterations; Determine whether the current number of iterations meets a preset number of iterations; If the current number of iterations does not meet the preset number of iterations, the weighted least squares solution is used as the initial estimate, and the calculation of the residual according to the initial estimate is returned to be executed until the weighted least squares solution converges; If the solution of the weighted least squares converges, or the current number of iterations meets the preset number of iterations, the solution of the weighted least squares is substituted into the target matrix equation for solving to obtain the target value.

9. A rearview mirror image enhancement system, characterized in that: include: An image acquisition unit, used to acquire a visible light image taken by a camera; An image input unit is used to input the visible light image into a pre-trained image processing model to obtain an enhanced image; wherein the image processing model at least includes a simulated infrared network and an image fusion algorithm module, and the visible light image is subjected to infrared simulation processing by the simulated infrared network to obtain a simulated infrared image, and the visible light image and the simulated infrared image are fused by the image fusion algorithm module to obtain the enhanced image; the image processing model is pre-trained using a sample visible light image; A display unit, used to display the enhanced image on a vehicle display screen; Also includes: An acquisition unit, used for acquiring a sample visible light image taken by the camera; An input unit, used for inputting the sample visible light image into an image processing model to obtain a sample enhanced image; A requirement determination unit, used to determine whether the sample enhanced image meets a preset enhancement requirement; A model determination unit, configured to determine the image processing model as a trained image processing model if the sample enhanced image meets the preset enhancement requirement; An adjustment unit is used to use the sample visible light image as the input of the loss function of the image processing model to obtain the sample enhanced image if the sample enhanced image does not meet the preset enhancement requirements, and continuously adjust the parameters of the image processing model until the sample enhanced image meets the preset enhancement conditions, thereby confirming that the image processing model training is successful.

10. The system according to claim 9, characterized in that The image processing model includes a simulated infrared network and an image fusion algorithm module, and the input unit includes: An infrared processing unit, used for performing infrared simulation processing on the sample visible light image through the simulated infrared network to obtain a sample simulated infrared image; A fusion unit, used for fusing the sample simulated infrared image and the sample visible light image through an image fusion algorithm module to obtain a sample enhanced image; The simulated infrared network includes an encoder, a converter, a decoder and a discriminator, and the infrared processing unit includes: A downsampling unit, configured to perform downsampling processing on the sample visible light image through the encoder to obtain a feature map corresponding to the sample visible light image; A convolution processing unit, configured to perform convolution processing on the feature map through the converter to obtain infrared image features corresponding to the feature map; An up-sampling unit, used for decoding the infrared feature through the decoder to obtain a decoded simulated infrared image; A judging unit, used for judging whether the decoded simulated infrared image meets the requirements of a real infrared image through the discriminator; an image determination unit, configured to determine the decoded simulated infrared image as a sample simulated infrared image if the decoded simulated infrared image meets the requirements of a real infrared image; The execution unit is used to adjust the parameters of the simulated infrared network if the decoded simulated infrared image does not meet the requirements of the real infrared image, and return to execute the downsampling processing of the sample visible light image through the encoder to obtain the feature map corresponding to the sample visible light image until the decoded simulated infrared image meets the requirements of the real infrared image.

Images