Intelligent 3D reversing image enhancement system and method

Abstract

The present application relates to a kind of intelligent 3D reverse image enhancement system and method, the method therein includes the following steps: infrared structured light projector projects structured light to the area in the lower rear of vehicle, through infrared camera module synchronous acquisition infrared structured light image sequence, and the phase obtained by calculating the image collected is input into phase optimization network and is carried out image enhancement, three-dimensional reconstruction is carried out using enhanced phase information to obtain infrared structured light three-dimensional stereoscopic image;Through RGB color camera module, the long-distance background image information of the rear of vehicle is collected, and the rotation angle and translation vector of vehicle movement are obtained in real time by gyroscope;Two are input monocular depth estimation network, and the output RGB three-dimensional stereoscopic image;Finally, infrared structured light three-dimensional stereoscopic image and RGB three-dimensional stereoscopic image are depth fusion, obtain the three-dimensional stereoscopic image of complete depth and display in vehicle display, to overcome the deficiency of existing two-dimensional reverse image technology, improve the safety of reversing driving.

Description

Technical Field

[0001] This invention relates to the field of vehicle driving vision processing technology, and in particular to an intelligent 3D reversing image enhancement system and method. Background Technology

[0002] With the rapid development of industrial manufacturing and electronic science, more and more high-end in-vehicle equipment has been installed in civilian vehicles to improve driving safety and reduce driving difficulty. Among these, reversing is a scenario that drivers encounter almost every day. Because reversing differs from normal driving, drivers cannot directly observe the traffic conditions behind the vehicle during reversing. Relying solely on the rearview mirror to obtain information behind the vehicle creates a large number of blind spots, leading to failure to detect obstacles or inaccurate judgment of distances, which greatly increases safety hazards and driving difficulty.

[0003] To ensure reversing safety, radar technology is used in automotive driving. By installing several radar modules at the front and rear of the vehicle, an alarm is emitted to alert the driver when the radar senses the car approaching an obstacle. Radar technology is highly sensitive and provides timely feedback, and remains widely used. However, radar can only alert the driver through sound feedback and cannot visually display the blind spot behind the vehicle, making it difficult for the driver to determine the exact location of obstacles. In recent years, computer vision technology has been widely applied in daily life, including automotive reversing camera technology. By combining reversing cameras with radar technology, a comprehensive analysis and judgment of the situation behind the vehicle can be made from both visual and auditory dimensions. This solution has become a widely popular reversing assistance technology. However, existing reversing cameras are all simple two-dimensional images. In environments with monotonous backgrounds or low light, drivers may find it difficult to accurately judge distances from two-dimensional images, easily leading to misjudgments and oversights. Therefore, how to use high-quality three-dimensional images to replace traditional two-dimensional images to assist vehicle reversing is a very important research direction. Summary of the Invention

[0004] The present invention aims to overcome the shortcomings of the prior art and provide an intelligent 3D reversing image enhancement system.

[0005] The second objective of this invention is to provide an intelligent 3D reversing image enhancement method, which can overcome the shortcomings of existing two-dimensional reversing image technology and improve the safety of reversing driving.

[0006] The technical solution of this invention to solve the problems of the prior art is:

[0007] An intelligent 3D reversing image enhancement system includes an infrared structured light module, an RGB visible light module, and a data processing module. The infrared structured light module is installed at the rear of the vehicle and includes an infrared structured light projector and an infrared camera module. The infrared camera module is used to capture ground information behind the vehicle. The light emitted by the infrared structured light projector coincides with the field of view of the infrared camera module. The RGB visible light module is also installed at the rear of the vehicle and includes an RGB color camera module module and a gyroscope. The data processing module includes an image processor and an in-vehicle display. The image processor is responsible for intelligent image processing, and the in-vehicle display is used to display the generated image.

[0008] Preferably, the infrared structured light module is installed at a downward angle at the rear of the vehicle, and the RGB visible light module is installed above the rear windshield of the vehicle; the optical axis of the infrared camera module is parallel to the direction of the rear of the vehicle, and the optical axis of the infrared structured light projector forms an angle of approximately 30 degrees with the optical axis of the infrared camera module; the data processing module is located in the vehicle's cab, wherein the image processor is integrated into the vehicle's mainboard and is responsible for intelligent image processing throughout the entire system process; the vehicle display screen is located above the center console in the cab.

[0009] A method for enhancing intelligent 3D reversing images includes the following steps:

[0010] S1: The infrared structured light projector projects structured light into the area below and behind the vehicle, and the infrared camera module simultaneously acquires the infrared structured light image sequence.

[0011] S2: The phase of the image acquired by the infrared camera module is calculated. The calculated phase is input into the pre-trained phase optimization network for image enhancement. The enhanced phase information is used to perform three-dimensional reconstruction to obtain a small field of view close-range three-dimensional stereo image, i.e., an infrared structured light three-dimensional stereo image.

[0012] S3: Collect distant background image information behind the vehicle through the RGB color camera module, and obtain relevant data of vehicle motion in real time through the gyroscope. The relevant data includes rotation angle and translation vector. Then, input the collected image stream into the pre-trained monocular depth estimation network to output a large field of view distant three-dimensional stereo image with depth information, i.e., an RGB three-dimensional stereo image.

[0013] S4: The image processor performs deep fusion of the infrared structured light 3D image and the RGB 3D image to obtain a complete depth 3D image, which is then displayed on the vehicle display.

[0014] Preferably, in step S1, the structured light emitted by the infrared structured light projector is a sinusoidal grating phase-shifted image, represented as...

[0015] In (x,y)=a+bcos(2πfx+2nπ / N),

[0016] In the formula, a is the background light intensity; b is the structured light modulation degree, i.e., contrast; f is the frequency of the grating phase-shifted image; n is the grating phase-shifted image number, n = 1, 2, ..., N; where N is the number of phase-shifting steps, N is greater than or equal to 3; then the wrapping phase is calculated using the N-step phase-shifting method:

[0017]

[0018] Preferably, in step S2, the phase optimization network is learned and trained using computer-simulated images, wherein the computer-simulated images are modeled according to sinusoidal grating phase-shift images: sinusoidal grating phase-shift images with random Gaussian surfaces are generated as labels, and random noise, random phase shift, and reduced contrast are added to each label image to simulate low-quality structured light images in real outdoor scenes. The simulated network label images are represented as follows:

[0019] I lab (x,y)=A+B·cos[2πfx+2nπ / N+Φ(x,y)];

[0020] Where A is the natural light intensity; B is the structured light modulation degree; Φ(x,y) is a randomly distributed surface, i.e., the phase distribution in space; the simulated network input image is represented as:

[0021] I in (x,y)=A'(x,y)+B'(x,y)·cos[2πfx+2nπ / N+Φ(x,y)+ε(x,y)]+σ(x,y);

[0022] Where A'(x,y) is the non-uniformly distributed natural light intensity; B'(x,y) is the non-uniformly distributed structured light modulation degree, which is randomly set to a small interval to simulate low contrast in outdoor environments; ε(x,y) is a random phase shift to simulate phase error caused by light scattering; and σ(x,y) is random Gaussian white noise to simulate noise interference caused by complex environments.

[0023] Preferably, in step S2, the depth range of the 3D reconstruction of the infrared structured light module is divided into m depth regions R1, R2, ..., Rm, and network models are trained for each region. The data of each depth region R is trained separately to obtain m corresponding network models W1, W2, ..., Wm. Finally, the results of the m regions output from the m network models are integrated into a complete infrared structured light 3D stereoscopic image.

[0024] Preferably, in step S2, the phase optimization network is a Unet encoder-decoder structure, with the encoding module being downsampled and the decoding module being upsampled; both the encoding and decoding modules are feature pyramid structures; the encoding module has four layers, with the image dimensions of each layer reduced to half of the previous layer, and the first three layers each containing three deformable convolutional modules connected together; the fourth layer is the bottom layer, containing a deformable convolutional module, a residual module, an attention module, and another deformable convolutional module connected in sequence; the decoding module is structurally symmetrical to the encoding module and is the reverse process of the encoding module; the image dimensions of each layer in the decoding module are doubled until they are restored to the original image size; furthermore, corresponding layers of the encoding and decoding modules are directly transmitted via skip connections.

[0025] Preferably, in step S3, the RGB color camera module continuously acquires a video stream of two-dimensional images, and the gyroscope obtains the rotation angle and translation vector of the vehicle at each moment of movement; then the acquired data information is input into a monocular depth estimation network for depth estimation; wherein, the input data of the monocular depth estimation network is the current frame image, the difference image between the current frame image and the previous and next frames, and the motion data at the current moment obtained by the gyroscope.

[0026] Preferably, the dataset in the monocular depth estimation network uses a depth camera to acquire video streams and ground depth ground values ​​from the real environment. During training, the monocular depth estimation network is divided into a coarse estimation module, an edge estimation module, and a fine estimation module. The coarse estimation module extracts global features from the input current frame RGB image; the edge estimation module extracts edge features from the input difference image and motion vector; and the fine estimation module combines the outputs of the coarse estimation module and the edge estimation module to extract local features, ultimately outputting a higher-precision depth map. All modules adopt the Unet encoding and decoding structure. The encoding and decoding parts of the coarse estimation module and the fine estimation module each have 5 layers. In the encoding part, two dilated convolutional layers are connected in each layer to form a dilated spatial convolution pooling pyramid. The image is sampled in parallel with dilated convolutions at different sampling rates to obtain high-order context features of different proportions. In the decoding part, 3×3 convolutional layers are used in each layer for upsampling and recovery. The encoding and decoding parts of the edge estimation module each have 4 layers. In each layer, two 3×3 convolutional layers are connected one after the other. An attention module is added between the two bottom convolutional layers.

[0027] Preferably, in step S4, the image processor fuses the infrared 3D stereo image and the RGB 3D stereo image, wherein the RGB 3D stereo image is a large field of view, long-distance stereo information used to display the global background; the infrared 3D stereo image is a small field of view, short-distance stereo information used to highlight specific obstacles in the local area; the infrared 3D stereo image and the RGB 3D stereo image are fused in the same world coordinate system to obtain a complete 3D stereo image of the rear of the vehicle with a large field of view.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] 1. The intelligent 3D reversing image enhancement method of the present invention can realize a large field of view 3D stereoscopic reversing image. Compared with the traditional two-dimensional reversing image technology, the three-dimensional stereoscopic image can bring a more realistic visual experience, and the distance of obstacles can be more clearly distinguished from the depth image. In addition, the intelligent 3D reversing image enhancement method of the present invention can more intuitively display the real depth scene of the rear part of the vehicle, greatly improving the safety of the driver during the reversing process and avoiding potential safety hazards. It has a good improvement effect, especially for reversing scenarios with a single background or poor light at night.

[0030] 2. The intelligent 3D reversing image enhancement method of the present invention uses a deep learning intelligent algorithm to enhance and correct the phase information obtained by the traditional structured light 3D reconstruction algorithm. This can improve the quality of structured light ranging under the influence of outdoor interference light, so that the intelligent 3D reversing image enhancement method of the present invention can better adapt to various complex outdoor environments, improve the accuracy of 3D reconstruction, and ensure the reliability of 3D reversing images.

[0031] 3. In practical applications, high-precision ranging is required for near-distance obstacles, while the accuracy requirement is lower for distant backgrounds. The intelligent 3D reversing image enhancement method of this invention uses structured light 3D reconstruction to obtain a high-precision depth image of near-distance obstacles, and uses monocular depth estimation to obtain a global background depth image with a large field of view at a distance. By combining the characteristics of the two technologies, a full-field-of-view depth image can be obtained. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the intelligent 3D reversing image enhancement system of the present invention.

[0033] Figure 2 This is a schematic diagram of the intelligent 3D reversing image enhancement depth measurement method of the present invention.

[0034] Figure 3 This is a flowchart of the intelligent 3D reversing image enhancement method of the present invention.

[0035] Explanation of reference numerals in the attached diagram: 1. Infrared structured light module; 1_1. Infrared structured light projector; 1_2. Infrared camera module; 2. RGB visible light module; 2_1. RGB color camera module; 2_2. Gyroscope; 3. Data processing module. Detailed Implementation

[0036] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0037] See Figure 1 and Figure 2 The intelligent 3D reversing image enhancement system of the present invention includes an infrared structured light module 1, an RGB visible light module 2, and a data processing module 3. The infrared structured light module 1 includes an infrared structured light projector 1_1 and an infrared camera module 1_2; the RGB visible light module 2 includes an RGB color camera module 2_1 and a gyroscope 2_2; and the data processing module 3 includes an image processor and an in-vehicle display.

[0038] See Figure 1 and Figure 2 The infrared structured light module 1 is installed at a downward angle at the rear of the vehicle; the infrared camera module 1_2 should ensure that it captures ground information behind the vehicle; the light emitted by the infrared structured light projector 1_1 should coincide with the field of view of the infrared camera module 1_2; the optical axis of the infrared camera module 1_2 is parallel to the direction of the rear of the vehicle, and the optical axis of the infrared structured light projector 1_1 forms an angle of approximately 30 degrees with the optical axis of the infrared camera module 1_2 to ensure that the structured light illuminates from an oblique angle. The RGB visible light module 2 is installed above the rear windshield of the vehicle, using a short focal length lens or a fisheye camera. The data processing module 3 is located in the vehicle's cab, and its image processor is integrated into the vehicle's mainboard, responsible for intelligent image processing throughout the system; the vehicle display screen is located above the center console in the cab.

[0039] See Figure 3 The intelligent 3D reversing image enhancement method of the present invention includes the following steps:

[0040] S1, Infrared structured light projector 1_1 projects structured light into the area below and behind the vehicle, and infrared structured light image sequences are synchronously acquired by infrared camera module 1_2.

[0041] The structured light emitted by the infrared structured light projector 1_1 is a sinusoidal grating phase-shifted image, which can be represented as:

[0042] I n (x,y)=a+bcos(2πfx+2nπ / N),

[0043] In the formula: a is the background light intensity; b is the structured light modulation degree (contrast), both a and b are uniformly distributed values; f is the frequency of the grating phase-shift image; n is the grating phase-shift image number, n = 1, 2, ..., N; N is the phase-shift step number, and the value of N is greater than or equal to 3.

[0044] The wrapping phase is calculated using the N-step phase shift method:

[0045]

[0046] S2. The phase of the image acquired by the infrared camera module 1_2 is calculated, and the result is input into a pre-trained phase optimization network for image enhancement. The enhanced phase information is then used for 3D reconstruction to obtain a small field-of-view, near-range 3D stereo image. The phase optimization network (i.e., the structured light enhancement network) is learned and trained using computer-simulated images. The computer-simulated images are modeled according to sinusoidal grating phase-shift images: sinusoidal grating phase-shift images with random Gaussian surfaces are generated as labels. Random noise, random phase shifts, and reduced contrast are added to each label image to simulate low-quality structured light images in real outdoor scenes. The simulated network label images are represented as follows:

[0047] I lab (x,y)=A+B·cos[2πfx+2nπ / N+Φ(x,y)];

[0048] Where A is the natural light intensity; B is the structured light modulation degree; Φ(x,y) is a randomly distributed surface, i.e., the phase distribution in space; the simulated network input image is represented as:

[0049] I in (x,y)=A'(x,y)+B'(x,y)·cos[2πfx+2nπ / N+Φ(x,y)+ε(x,y)]+σ(x,y);

[0050] In the formula, A'(x,y) is the non-uniformly distributed natural light intensity; B'(x,y) is the non-uniformly distributed structured light modulation degree, which is randomly set to a small interval to simulate low contrast in outdoor environments; ε(x,y) is the random phase shift amount, used to simulate phase error caused by light scattering; σ(x,y) is random Gaussian white noise, used to simulate noise interference caused by complex environments.

[0051] Considering that structured light is significantly affected by ambient light outdoors, with weaker structured light intensity and lower image contrast at greater distances, a method is used to reduce the learning difficulty of the network. The depth range of the 3D reconstruction from the infrared structured light module 1 is divided into m depth regions R1, R2, ..., Rm for separate network model training. Each depth region R is trained individually, resulting in m corresponding network models W1, W2, ..., Wm. In practical applications, each network model W processes only the data within its corresponding depth region. Finally, the results from the m regions output by the m network models are integrated into a complete infrared structured light 3D image.

[0052] In this embodiment, the structured light enhancement network is a U-shaped encoder-decoder structure, with the encoding module being downsampled and the decoding module being upsampled. Both the encoding and decoding modules are feature pyramid structures. The encoding module has four layers, with the image dimensions of each layer reduced to half the size of the layer above. The first three layers each contain three deformable convolutional modules connected together. The fourth layer is the bottom layer, which contains a deformable convolutional module, a residual module, an attention module, and another deformable convolutional module connected in sequence. The decoding module is structurally symmetrical to the encoding module and is the reverse process of the encoding module. The image dimensions of each layer in the decoding module are doubled until the original image size is restored. Furthermore, corresponding layers in the encoding and decoding modules are directly transmitted via skip connections.

[0053] S3. Acquire distant background image information behind the vehicle using an RGB color camera module, and obtain relevant vehicle motion data in real time using a gyroscope. This motion data includes rotation angles and translation vectors. Then, input the acquired image stream into a pre-trained monocular depth estimation network, which outputs a large field-of-view, distant 3D stereo image with depth information.

[0054] Two-dimensional images acquired by the RGB color camera module are used to obtain motion vectors at each moment using a gyroscope. This information is then input into a monocular depth estimation network for depth estimation. The RGB color camera module continuously acquires video streams, and the relevant information of three adjacent frames and the gyroscope data are simultaneously input into the pre-trained monocular depth estimation network for feature extraction. This ensures that adjacent information can be obtained, which helps the network to predict the depth of similar objects more accurately.

[0055] In this embodiment, the monocular depth estimation network model is learned and trained using existing public datasets (such as NYU Depth, KITTI, etc.). The datasets obtain video streams and ground truth depth values ​​from real-world environments. During training, the current frame image is subtracted from the previous and next frame images to obtain two difference images at adjacent time points. Then, the motion vector of the gyroscope at the current time point is obtained. The input data of the monocular depth estimation network includes: the current frame image, the two difference images (i.e., the difference images between the current frame image and the previous and next frames), and the motion information of the gyroscope at the current time point. Multiple data inputs ensure richer detail information and more accurate edge depth information.

[0056] In addition, the monocular depth estimation network in this embodiment is divided into three modules: a coarse estimation module, an edge estimation module, and a fine estimation module. The coarse estimation module extracts global features from the input current frame RGB image; the edge estimation module extracts edge features from the difference image and motion vector; and the fine estimation module combines the outputs of the above two modules to extract local features, ultimately outputting a more accurate depth map. The coarse estimation module, edge estimation module, and fine estimation module all adopt the Unet encoding-decoding structure. The encoding and decoding parts of the coarse and fine estimation modules each have 5 layers. In each layer of the encoding part, two dilated convolutional layers are connected to form a dilated spatial convolution pooling pyramid, which samples the image in parallel with dilated convolutions at different sampling rates to obtain different proportions of high-order contextual features. In each layer of the decoding part, 3×3 convolutional layers are used for layer-by-layer upsampling and recovery. The encoding and decoding parts of the edge estimation module each have 4 layers, with two consecutive 3×3 convolutional layers in each layer. An attention module is added between the two bottom convolutional layers.

[0057] S4. The image processor performs deep fusion of the infrared structured light 3D image and the RGB 3D image to obtain a complete depth 3D image, which is then displayed on the vehicle's in-vehicle display. The RGB 3D image provides a wide field of view and long-range stereoscopic information, used to display the global background; the infrared 3D image provides a narrow field of view and short-range stereoscopic information, used to highlight specific obstacles. By fusing the two images in the same world coordinate system, a complete 3D image of the rear of the vehicle with a wide field of view can be obtained.

[0058] The above are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above content. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for enhancing intelligent 3D reversing images, characterized in that, An intelligent 3D reversing image enhancement system is adopted, which includes an infrared structured light module, an RGB visible light module, and a data processing module. The infrared structured light module is installed at the rear of the vehicle and includes an infrared structured light projector and an infrared camera module. The infrared camera module is used to capture ground information behind the vehicle. The light emitted by the infrared structured light projector coincides with the field of view of the infrared camera module. The RGB visible light module is installed at the rear of the vehicle and includes an RGB color camera module and a gyroscope. The data processing module includes an image processor and an in-vehicle display. The image processor is responsible for intelligent image processing, and the in-vehicle display is used to display the generated image. Includes the following steps: S1: The infrared structured light projector projects structured light into the area below and behind the vehicle, and the infrared camera module simultaneously acquires the infrared structured light image sequence. S2: The phase of the image acquired by the infrared camera module is calculated. The calculated phase is input into the pre-trained phase optimization network for image enhancement. The enhanced phase information is used to perform three-dimensional reconstruction to obtain a small field of view close-range three-dimensional stereo image, i.e., an infrared structured light three-dimensional stereo image. S3: Collect distant background image information behind the vehicle through the RGB color camera module, and obtain relevant motion data of the vehicle movement in real time through the gyroscope. The relevant motion data includes rotation angle and translation vector. Then, input the collected image stream into the pre-trained monocular depth estimation network to output a large field of view distant three-dimensional stereo image with depth information, i.e., an RGB three-dimensional stereo image. S4: The image processor performs deep fusion of the infrared structured light 3D image and the RGB 3D image to obtain a complete depth 3D image, which is then displayed on the vehicle display.

2. The intelligent 3D reversing image enhancement method according to claim 1, characterized in that, The infrared structured light module is installed at a downward angle at the rear of the vehicle, and the RGB visible light module is installed above the rear windshield of the vehicle; the optical axis of the infrared camera module is parallel to the direction of the rear of the vehicle, and the optical axis of the infrared structured light projector forms an angle of approximately 30 degrees with the optical axis of the infrared camera module; the data processing module is located in the vehicle's cab, where the image processor is integrated into the vehicle's mainboard and is responsible for intelligent image processing throughout the entire system process; the vehicle display is located above the center console in the cab.

3. The intelligent 3D reversing image enhancement method according to claim 1, characterized in that, In step S1, the structured light emitted by the infrared structured light projector is a sinusoidal grating phase-shifted image, represented as... In the formula, a is the background light intensity; b is the structured light modulation degree, i.e., contrast; f is the frequency of the grating phase-shifted image; n is the grating phase-shifted image number, n=1, 2, …, N; where N is the number of phase-shifting steps, N is greater than or equal to 3; then the wrapping phase is calculated using the N-step phase-shifting method: 。 4. The intelligent 3D reversing image enhancement method according to claim 3, characterized in that, In step S2, the phase optimization network is learned and trained using computer-simulated images, wherein the computer-simulated images are modeled according to sinusoidal grating phase-shift images: sinusoidal grating phase-shift images with random Gaussian surfaces are generated as labels, and random noise, random phase shift, and reduced contrast are added to each label image to simulate low-quality structured light images in real outdoor scenes. The simulated network label images are represented as follows: ； Where A is the natural light intensity; B is the structured light modulation degree; Φ(x, y) is a randomly distributed surface, i.e., the phase distribution in space; the simulated network input image is represented as: ； In the formula, A'(x, y) is the non-uniformly distributed natural light intensity; B'(x, y) is the non-uniformly distributed structured light modulation degree, which is randomly set to a small interval to simulate low contrast in outdoor environments; ɛ(x, y) is a random phase shift amount to simulate phase error caused by light scattering; σ(x, y) is random Gaussian white noise to simulate noise interference caused by complex environments.

5. The intelligent 3D reversing image enhancement method according to claim 4, characterized in that, In step S2, the depth range of the 3D reconstruction of the infrared structured light module is divided into m depth regions R1, R2, …, Rm, and network models are trained for each region. The data of each depth region R is trained separately to obtain m corresponding network models W1, W2, …, Wm. Finally, the results of the m regions output from the m network models are integrated into a complete infrared structured light 3D stereo image.

6. The intelligent 3D reversing image enhancement method according to claim 5, characterized in that, In step S2, the phase optimization network is a Unet encoder-decoder structure, with the encoding module being downsampled and the decoding module being upsampled. Both the encoding and decoding modules are feature pyramid structures. The encoding module has four layers, with the image dimensions of each layer reduced to half the size of the layer above. The first three layers each contain three deformable convolutional modules connected together. The fourth layer is the bottom layer, which contains a deformable convolutional module, a residual module, an attention module, and another deformable convolutional module connected in sequence. The decoding module is structurally symmetrical to the encoding module and is the reverse process of the encoding module. The image dimensions of each layer in the decoding module are doubled until they are restored to the original image size. Furthermore, corresponding layers in the encoding and decoding modules are directly transmitted via skip connections.

7. The intelligent 3D reversing image enhancement method according to claim 6, characterized in that, In step S3, the RGB color camera module continuously acquires a video stream of two-dimensional images, and the gyroscope obtains the rotation angle and translation vector of the vehicle at each moment of movement. Then, the acquired data information is input into a monocular depth estimation network for depth estimation. The input data of the monocular depth estimation network includes the current frame image, the difference images between the current frame image and the previous and next frames, and the motion data at the current moment obtained by the gyroscope.

8. The intelligent 3D reversing image enhancement method according to claim 7, characterized in that, The dataset in the monocular depth estimation network uses a depth camera to obtain video streams and ground depth values ​​of the real environment. During training, the monocular depth estimation network is divided into a coarse estimation module, an edge estimation module, and a fine estimation module. The coarse estimation module extracts global features from the input current frame RGB image; the edge estimation module extracts edge features from the input difference image and motion vector; and the fine estimation module combines the outputs of the coarse and edge estimation modules to extract local features, ultimately outputting a higher-precision depth map. All three modules employ a Unet encoding / decoding structure. The encoding and decoding parts of the coarse and fine estimation modules each have 5 layers. Each layer in the encoding part contains two connected dilated convolutional layers, forming a dilated spatial convolution pooling pyramid that samples the image in parallel with dilated convolutions at different sampling rates, obtaining varying proportions of high-order contextual features. Each layer in the decoding part uses 3×3 convolutional layers for layer-by-layer upsampling and recovery. The encoding and decoding parts of the edge estimation module each have 4 layers, with two consecutive 3×3 convolutional layers in each layer. An attention module is added between the two bottom convolutional layers.

9. The intelligent 3D reversing image enhancement method according to claim 8, characterized in that, In step S4, the image processor fuses the infrared 3D stereo image and the RGB 3D stereo image. The RGB 3D stereo image is a large field of view, long-distance stereo information used to display the global background; the infrared 3D stereo image is a small field of view, short-distance stereo information used to highlight specific obstacles. The infrared 3D stereo image and the RGB 3D stereo image are fused in the same world coordinate system to obtain a complete 3D stereo image of the rear of the vehicle with a large field of view.

Images