Image pair construction method and device, model training method and electronic equipment

Abstract

The invention relates to an image pair construction method and device, a model training method and electronic equipment, and the method comprises the steps: obtaining a first image and a second image which are formed after ambient light passes through a beam splitter in an imaging system, enabling the brightness of the first image to be smaller than that of the second image, enabling the imaging system to be a shading system with an opening, and enabling the brightness of the first image to be smaller than that of the second image to be smaller than that of the second image; the beam splitter acquires ambient light through the opening; and constructing an image pair according to the first image and the second image. Bright field imaging to a large target can be realized; under the action of light divided by the beam splitter, the brightness of the first image can be smaller than that of the second image, and the view field angles of the collected first image and second image can be the same, so that when the image pair is constructed according to the first image and the second image, the registration quality of the image pair can be improved, the registration error can be reduced, and the registration accuracy is improved. And meanwhile, an image pair formed by a weak light image and a normal light color image can be acquired.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technologies, and in particular, to an image pair construction method, apparatus, model training method, and electronic device. Background Art

[0002] An image pair mainly refers to two or more images having a correlation or similarity. In related technologies, image pairs are usually acquired by setting two image acquisition devices with parallel optical axes. However, when performing image registration through related technologies, the field of view angles of the acquired images are different, resulting in a relatively large error in the registered images. Summary of the Invention

[0003] The purpose of the present disclosure is to provide an image pair construction method, apparatus, model training method, and electronic device to solve the technical problems existing in related technologies.

[0004] To achieve the above purpose, in a first aspect, the present disclosure provides an image pair construction method, including: Obtaining a first image and a second image formed after environmental light passes through a beam splitter in an imaging system, where the brightness of the first image is less than that of the second image, the imaging system is a light-shielding system provided with an opening, and the beam splitter obtains environmental light through the opening; Constructing an image pair based on the first image and the second image.

[0005] Optionally, the first image is a RAW format image, and the second image is a color image; or, The first image is a color image, and the second image is an infrared image.

[0006] Optionally, the imaging system includes a first camera and a second camera. Obtaining a first image and a second image formed after environmental light passes through a beam splitter in the imaging system includes: Obtaining the first image formed after the environmental light passes through the beam splitter in the imaging system and enters the first camera, and the second image formed after the environmental light enters the second camera after passing through the beam splitter.

[0007] Optionally, the sensitivity of the first camera is greater than that of the second camera, or the sensitivity of the second camera is greater than that of the first camera.

[0008] Optionally, the imaging system further includes an attenuation sheet, and the attenuation sheet is disposed between the first camera and the beam splitter. Obtaining the first image formed after the environmental light passes through the beam splitter in the imaging system and enters the first camera includes: Obtain the first image formed when the ambient light enters the first camera after passing through the beam splitter and the attenuation sheet in the imaging system in sequence.

[0009] Optionally, the method further includes: Controlling the exposure signal generated by the first camera to be transmitted to the second camera so that the first camera and the second camera are exposed simultaneously; or, Controlling the exposure signal generated by the second camera to be transmitted to the first camera so that the first camera and the second camera are exposed simultaneously.

[0010] Optionally, constructing an image pair according to the first image and the second image includes: Converting the first image into a third image, where the third image is a color image; Aligning the spatial positions of the target pixel points in the second image with the spatial positions of the corresponding pixel points in the third image to obtain a fourth image; Constructing the image pair according to the fourth image and the first image.

[0011] Optionally, converting the first image into a third image includes: Performing mosaic processing, or mosaic processing and white balance processing on the first image to obtain a fifth image; Determining the third image according to the average brightness of the fifth image.

[0012] Optionally, determining the third image according to the average brightness of the fifth image includes: When the average brightness of the fifth image is greater than or equal to the preset brightness threshold, determining the fifth image as the third image.

[0013] Optionally, determining the third image according to the average brightness of the fifth image includes: When the average brightness of the fifth image is less than the preset brightness threshold, performing histogram equalization processing on the fifth image and determining the fifth image after histogram equalization processing as the third image.

[0014] Optionally, aligning the spatial positions of the target pixel points in the second image with the spatial positions of the corresponding pixel points in the third image to obtain a fourth image includes: Calculating the transformation relationship matrix between the first image and the second image by using the scale-invariant feature transform algorithm with the first image as the reference image; Align the spatial position of the target pixel points in the second image with the spatial position of the corresponding pixel points in the third image according to the conversion relationship matrix.

[0015] Optionally, constructing the image pair according to the fourth image and the first image includes: Crop the fourth image and the first image according to a preset corresponding position; Construct the cropped fourth image and the first image with the same preset corresponding position into an image pair.

[0016] In a second aspect, the present disclosure provides a model training method, including: Train the low-light enhancement model with the image pair constructed by any one of the image pair construction methods provided in the first aspect of the present disclosure.

[0017] In a third aspect, the present disclosure provides an image pair construction device, including an imaging system with a beam splitter. The imaging system is a light-shielding system with an opening. The beam splitter is arranged at the entrance of the opening of the imaging system, and ambient light is obtained through the opening; The imaging system is configured to obtain a first image and a second image formed by ambient light after passing through the beam splitter in the imaging system, and construct an image pair according to the first image and the second image, wherein the brightness of the first image is less than the brightness of the second image.

[0018] Optionally, the imaging system further includes a first camera and a second camera; The first camera is configured to obtain the transmitted light formed by ambient light after passing through the beam splitter to obtain the first image; The second camera is configured to obtain the reflected light formed by ambient light after passing through the beam splitter to obtain the second image.

[0019] Optionally, the image pair construction device further includes an attenuation sheet, and the attenuation sheet is arranged between the first camera and the beam splitter; The first camera is configured to obtain the light formed by ambient light after passing through the beam splitter and the attenuation sheet to obtain the first image.

[0020] Optionally, the imaging system further includes a plurality of light-shielding plates and a bracket, and each side of the bracket is provided with a groove; The plurality of light-shielding plates form a three-dimensional structure, and the edges of two adjacent light-shielding plates are connected through the grooves of the bracket to form a light-shielding system, and an opening is provided on one of the light-shielding plates to inject ambient light into the beam splitter.

[0021] Optionally, the size of the beam splitter is obtained according to the distance from the focus of the optical axis of the beam splitter to the center point of the outer surface of the second camera and the viewing angle range of the first camera and / or the second camera.

[0022] Optionally, the image pair construction device further includes a fixing structure for fixing the imaging system to the vehicle.

[0023] Optionally, the fixing structure includes an optical breadboard, a suction cup device, and an optical fixing member; The optical fixing member is used to fix the first camera, the second camera, and the beam splitter on the optical breadboard; The suction cup device is used to fix the optical breadboard to the vehicle.

[0024] Optionally, the optical fixing member includes a column and a sleeve. A screw is provided on the side of the sleeve for adjusting the height of the column; The beam splitter, the first camera, and the second camera are all connected to the corresponding sleeves through columns, and the sleeves connect the columns to the optical breadboard to fix the beam splitter, the first camera, and the second camera.

[0025] Optionally, the suction cup device includes a double-hinged suction cup and a rubber suction cup. One end of the double-hinged suction cup is connected to the optical breadboard, and the other end of the double-hinged suction cup is connected to the rubber suction cup; The rubber suction cup is used to fix the optical breadboard to the vehicle.

[0026] Fourthly, the present disclosure provides an electronic device, including: A memory storing a computer program thereon; A processor for executing the computer program in the memory to implement the steps of any of the methods provided in the first aspect or the second aspect of the present disclosure.

[0027] Through the above technical solutions, the ambient light obtained by the imaging system with an opening is transmitted to the beam splitter, and bright-field imaging of the maximum target can be achieved; after passing through the beam splitter, a first image and a second image are formed in the imaging system. Through the action of the light divided by the beam splitter, the brightness of the first image can be made less than that of the second image, and the viewing angles of the first image and the second image collected can be made the same. Furthermore, when constructing an image pair based on the first image and the second image, the registration quality of the image pair can be improved, the registration error can be reduced, and at the same time, an image pair composed of a low-light image and a normal-light color image can be collected.

[0028] Other features and advantages of the present disclosure will be described in detail in the following detailed description section. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide a further understanding of the present disclosure, and constitute a part of the specification. Together with the following detailed description, they are used to explain the present disclosure, but do not constitute a limitation to the present disclosure. In the accompanying drawings: Figure 1 is a schematic diagram of a device for constructing an image pair in the first related art.

[0030] Figure 2 is a schematic diagram of a device for acquiring an image pair in the second related art.

[0031] Figure 3 is a schematic flowchart of a process for acquiring an image pair in the third related art.

[0032] Figure 4 is a schematic diagram showing a method for constructing an image pair according to an exemplary embodiment of the present disclosure.

[0033] Figure 5 is a schematic flowchart showing a method for constructing an image pair according to an exemplary embodiment of the present disclosure.

[0034] Figure 6 is a schematic diagram showing a method for model training according to an exemplary embodiment of the present disclosure.

[0035] Figure 7 is a schematic diagram of a device for constructing an image pair according to an exemplary embodiment of the present disclosure.

[0036] Figure 8 is a schematic diagram showing the imaging of an imaging system according to an exemplary embodiment of the present disclosure.

[0037] Figure 9 is a schematic diagram of a light shielding plate and a bracket device according to an exemplary embodiment of the present disclosure.

[0038] Figure 10 is a front view of a fixing structure according to an exemplary embodiment of the present disclosure.

[0039] Figure 11 is a top view of a fixing structure according to an exemplary embodiment of the present disclosure.

[0040] Figure 12 is a schematic diagram of a system for constructing an image pair according to an exemplary embodiment of the present disclosure.

[0041] Figure 13 is a schematic diagram of a system for model training according to an exemplary embodiment of the present disclosure.

[0042] Figure 14 It is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners

[0043] The following will describe in detail the specific implementation manners of the present disclosure in conjunction with the accompanying drawings. It should be understood that the specific implementation manners described herein are only for explaining and illustrating the present disclosure, and are not used to limit the present disclosure.

[0044] Existing vehicles largely image the outside world through visible light cameras and achieve environmental perception during parking by image analysis. In low-light environments, it is difficult for in-vehicle cameras to clearly capture the targets in the environment, making it impossible for the vehicle to accurately perceive the surrounding environment, thus affecting the safety of parking. With the development of deep learning technology, deep learning models for low-light enhancement have achieved good results. However, almost all existing deep learning models perform low-light enhancement based on RGB (Red-Green-Blue) images and require the participation of a traditional ISP (Image Signal Processor), and both the efficiency and the effect are reduced due to the influence of the ISP. In theory, RAW images (RAW Image, digital negatives) contain more original information. If low-light enhancement is directly performed on RAW images, better enhancement effects can be achieved, and since it does not go through ISP processing, the efficiency of low-light enhancement can also be significantly improved. However, an end-to-end low-light enhancement model based on RAW images requires image pairs composed of low-light RAW images and color images to train the low-light enhancement model.

[0045] In the first related technology, as Figure 1 shown, this technology mainly fixes two image acquisition devices in parallel through jigs or mechanical devices so that the optical axes of the two image acquisition devices are parallel. When acquiring images, the two image acquisition devices are triggered to take synchronous shots, and the two images taken are pre-processed such as image registration to form an image pair dataset.

[0046] In the second related technology, as Figure 2 shown, a stereoscopic beam splitter is used to split the signal light returned from the target into two beams and respectively inject them into two image acquisition devices to achieve the acquisition of image pairs with overlapping fields of view, reducing the difficulty of image registration and greatly improving the accuracy of image registration.

[0047] In the third related technology, as Figure 3 shown, by means of algorithms to change the brightness of normal-brightness images and add noise, artificial synthesis of image pair data of low light and normal illumination can avoid the problems of image registration errors and different noise categories in constructing real image pairs.

[0048] However, the inventors found that when image registration is performed through the first related technology, the field of view angles of the two image acquisition devices do not completely overlap, resulting in a large error in image registration, and it is impossible to collect data on weak light images and normal light color image pairs; when image registration is performed through the second related technology, this technology is mainly used for dark field imaging of small targets and cannot be used for bright field imaging of large targets. At the same time, this technology can only collect weak light images and color image pairs by setting different exposure times for the two image acquisition devices, which will result in different noise categories for the two images and is not conducive to improving the performance of the model. When image registration is performed through the third related technology, the physical properties of weak light images cannot be truly simulated, weak light Raw images cannot be generated, and it is also difficult to produce a weak light data set that meets the requirements.

[0049] In view of this, the present disclosure provides an image pair construction method, apparatus, model training method, and electronic device to solve the technical problems existing in the related art.

[0050] As Figure 4 shown, Figure 4 is a schematic diagram showing an image pair construction method according to an exemplary embodiment of the present disclosure. Referring to Figure 4 , it includes: S401: Obtain a first image and a second image formed by environmental light after passing through a beam splitter in an imaging system, where the brightness of the first image is less than the brightness of the second image, the imaging system is a light-shielding system provided with an opening, and the beam splitter obtains environmental light through the opening; S402: Construct an image pair according to the first image and the second image.

[0051] Through the above technical solution, the environmental light obtained by the imaging system provided with an opening is transmitted to the beam splitter, enabling bright field imaging of large targets; after passing through the beam splitter, a first image and a second image are formed in the imaging system. Through the action of the light divided by the beam splitter, the brightness of the first image can be made less than the brightness of the second image, and the field of view angles of the collected first image and second image can be made the same. Furthermore, when constructing an image pair according to the first image and the second image, the registration quality of the image pair can be improved, the registration error can be reduced, and at the same time, it is possible to collect an image pair composed of a weak light image and a normal light color image.

[0052] To enable those skilled in the art to better understand the image pair construction method provided by the present disclosure, the above steps will be described in detail with examples below.

[0053] Exemplarily, the ambient light can be natural light. For example, it can be the light reflected when sunlight shines on an object. In this regard, the embodiments of the present disclosure do not make specific limitations. The beam splitter can be used to split an incident light beam into a reflected light and a transmitted light with a certain light intensity ratio. The imaging system can be a device that captures, processes, and records image information through optical, electronic, or other technical means. Among them, the imaging system can be stationary or in a moving state. In the embodiments of the present disclosure, the imaging system is provided with an opening, and the ambient light can enter the beam splitter of the imaging system through this opening. After passing through the beam splitter, the ambient light becomes transmitted ambient light and reflected ambient light, and then forms a first image and a second image in the imaging system. Among them, when the transmitted ambient light forms a first image in the imaging system, the reflected ambient light forms a second image in the imaging system, and when the reflected ambient light forms a first image in the imaging system, the transmitted ambient light forms a second image in the imaging system; and the brightness of the first image can be less than the brightness of the second image. By setting beam splitters with different ratios, the brightness of the first image can be made less than the brightness of the second image, or different exposure times can be set in the first camera and the second camera to make the brightness of the first image less than the brightness of the second image. In this regard, the embodiments of the present disclosure do not make specific limitations.

[0054] In the embodiments of the present disclosure, the beam splitter is arranged at the entrance of the opening of the imaging system, so that the ambient light can be obtained more conveniently. In this regard, the embodiments of the present disclosure do not make specific limitations.

[0055] By setting an opening in the imaging system to inject the ambient light into the beam splitter, maximum target bright-field imaging can be achieved; then, the ambient light is divided into transmitted ambient light and reflected ambient light by the beam splitter, and a first image and a second image are formed in the imaging system, so that the first image and the second image obtained under the same field of view angle can be realized, and thus the registration quality of the first image and the second image can be improved. And a dual-channel imaging method is realized, and the acquisition of an image pair composed of a low-light image and a normal-brightness image with high time synchronization of a moving target can be performed.

[0056] In a possible manner, the first image is a RAW format image, and the second image is a color image; or, The first image is a color image, and the second image is an infrared image.

[0057] It should be understood that the image obtained from the imaging system can be a RAW image directly obtained from the imaging chip in the imaging system, and this RAW image includes various imaging parameters; it can also be an RGB color image or an infrared image. In this regard, the embodiments of the present disclosure do not make specific limitations.

[0058] In the embodiments of the present disclosure, when the first image is a RAW format image, the second image can be a color image; when the first image is a color image, the second image can be an infrared image. In this regard, the embodiments of the present disclosure do not make specific limitations. Among them, when obtaining the first image and the second image, different brightness images can be obtained by controlling the exposure time of the imaging system, or by adding attenuation filters. In this regard, the embodiments of the present disclosure do not make specific limitations.

[0059] In a possible manner, the imaging system includes a first camera and a second camera. Obtaining the first image and the second image formed by the ambient light after passing through the beam splitter in the imaging system includes: Obtaining the first image formed by the ambient light entering the first camera after passing through the beam splitter in the imaging system, and the second image formed by the ambient light entering the second camera after the beam splitter.

[0060] It should be understood that in the imaging system, there are also a first camera and a second camera. Among them, the first camera can be used to form the first image, and the second camera is used to form the second image, or the first camera can be used to form the second image, and the second camera is used to form the first image. In the embodiments of the present disclosure, the first camera is used to form the first image, and the second camera is used to form the second image.

[0061] When the ambient light passes through the beam splitter, the light entering the first camera can form the first image, and the light entering the second camera can form the second image. Among them, the first camera can be a combination of a first lens and a first sub-camera, and the second camera can be a combination of a second lens and a second sub-camera. In this regard, the embodiments of the present disclosure do not make specific limitations. The first sub-camera and the second sub-camera can be cameras with the same parameters or cameras with different parameters.

[0062] In a possible manner, the sensitivity of the first camera is greater than the sensitivity of the second camera, or the sensitivity of the second camera is greater than the sensitivity of the first camera.

[0063] It should be understood that the higher the sensitivity of the camera, the clearer the captured image. When the first image or the second image is a color image, the obtained image quality is better. Thus, in the embodiments of the present disclosure, the sensitivity of the camera for capturing color images is higher than that of the camera for capturing other images. It can be that the sensitivity of the first camera is greater than the sensitivity of the second camera, or the sensitivity of the second camera is greater than the sensitivity of the first camera. When the sensitivities of the first camera and the second camera are different, the exposure times and gains of the first camera and the second camera can be adjusted, and thus the first image and the second image can be obtained.

[0064] Thus, the exposure time and gain of the first camera and the second camera can be adjusted, so that high-quality second images and low-illuminance first images can be acquired in low-light environments. Furthermore, when the first image and the second image are used for subsequent image pair construction, the quality of the image pair can be improved.

[0065] In a possible implementation, the imaging system further includes an attenuation filter, which is disposed between the first camera and the beam splitter. The step of obtaining the first image formed by the ambient light entering the first camera after passing through the beam splitter in the imaging system includes: Obtaining the first image formed by the ambient light entering the first camera after sequentially passing through the beam splitter and the attenuation filter in the imaging system.

[0066] It should be understood that when the ratio of the reflected light to the transmitted light of the beam splitter is the same, an attenuation filter can be added and disposed between the first camera and the beam splitter, thereby reducing the brightness of the light entering the first camera and making the brightness of the first image less than that of the second image.

[0067] Disposing an attenuation filter between the first camera and the beam splitter and allowing the ambient light passing through the beam splitter to enter the first camera after passing through the attenuation filter can make the brightness of the first image less than that of the second image.

[0068] In a possible implementation, the method further includes: Controlling the exposure signal generated by the first camera to be transmitted to the second camera to enable simultaneous exposure of the first camera and the second camera; or, Controlling the exposure signal generated by the second camera to be transmitted to the first camera to enable simultaneous exposure of the first camera and the second camera.

[0069] It should be understood that when acquiring the first image through the first camera and the second image through the second camera, the first camera and the second camera need to be exposed simultaneously. Thus, an exposure signal can be generated in the first camera, which can include the exposure time and the exposure duration. Then, control the exposure signal to be transmitted to the second camera, thereby enabling the first camera and the second camera to be exposed for the same duration at the same time, so that the field of view angles of the acquired first image and second image are the same. Alternatively, an exposure signal can be generated in the second camera, which can include the exposure time and the exposure duration. Then, control the exposure signal to be transmitted to the first camera, thereby enabling the first camera and the second camera to be exposed for the same duration at the same time, so that the field of view angles of the acquired first image and second image are the same.

[0070] By transmitting the exposure signal to the opposite camera, the field of view angles of the first image and the second image collected can be made the same. Thus, when the first image and the second image are used for subsequent construction of an image pair, the quality of the image pair can be improved.

[0071] In a possible way, constructing an image pair based on the first image and the second image includes: Converting the first image into a third image, where the third image is a color image; Aligning the spatial positions of the target pixel points in the second image with the spatial positions of the corresponding pixel points in the third image to obtain a fourth image; Constructing the image pair based on the fourth image and the first image.

[0072] It should be understood that when the first image is a RAW format image and the second image is a color image, when constructing an image pair based on the first image and the second image, the first image needs to be converted into a color image, registered with the second image according to the converted color image, and then an image pair is constructed based on the registered image and the first image, thereby improving the quality of the image pair.

[0073] In the embodiment of the present disclosure, first, the first image is converted into a color third image. Then, taking the third image as a reference object, the spatial positions of each pixel point in the second image are aligned with the corresponding pixel points in the third image to obtain a fourth image. Then, an image pair is constructed based on the fourth image and the first image, thereby improving the quality of the image pair.

[0074] In a possible way, converting the first image into the third image includes: Performing mosaic processing, or mosaic processing and white balance processing, on the first image to obtain a fifth image; Determining the third image according to the average brightness of the fifth image.

[0075] It should be understood that when converting the first image into the third image, mosaic processing or mosaic processing and white balance processing can be used to convert the first image into the third image. Among them, mosaic processing can be used to blur or block specific areas of the image. White balance processing can be used to adjust the RGB channel gains of the image so that the white areas in the image remain white under different illuminations.

[0076] Demosaicing the first image can restore the color information of the image. After that, performing white balance processing on the first image after demosaicing can restore the true color of the target in the image, and converting the first image with restored true color into an 8-bit image to obtain the fifth image. Then, the third image can be determined according to the average brightness of the fifth image, where the average brightness can be the average value of the brightness values corresponding to multiple pixel points in the fifth image.

[0077] In a possible way, the determining the third image according to the average brightness of the fifth image includes: When the average brightness of the fifth image is greater than or equal to the preset brightness threshold, determining the fifth image as the third image.

[0078] It should be understood that when determining the third image according to the average brightness of the fifth image, the average brightness of the fifth image can be compared with the preset brightness threshold. When the average brightness is greater than or equal to the preset brightness threshold, it can represent that the quality of the fifth image reaches the expected value at this time, and the fifth image can be determined as the third image.

[0079] In a possible way, the determining the third image according to the average brightness of the fifth image includes: When the average brightness of the fifth image is less than the preset brightness threshold, performing histogram equalization processing on the fifth image, and determining the fifth image after histogram equalization processing as the third image.

[0080] It should be understood that when the average brightness is less than the preset brightness threshold, it can represent that the brightness of the fifth image fails to meet the standard at this time, and histogram equalization processing needs to be performed on the fifth image to improve the brightness of the fifth image. Then, the fifth image after histogram equalization processing is determined as the third image, which can improve the registration rate between the second image and the third image, and further improve the quality of the image pair constructed by the first image and the fourth image.

[0081] In a possible way, the aligning the spatial positions of the target pixel points in the second image with the spatial positions of the corresponding pixel points in the third image to obtain the fourth image includes: Using the scale-invariant feature transform algorithm, with the first image as the reference image, calculating the transformation relationship matrix between the first image and the second image; According to the transformation relationship matrix, aligning the spatial positions of the target pixel points in the second image with the spatial positions of the corresponding pixel points in the third image.

[0082] It should be understood that the Scale-Invariant Feature Transform (SIFT) algorithm can be used to extract local features from an image and generate descriptors. In the embodiments of the present disclosure, when using the SIFT algorithm, the first image can be used as a reference image, and then the transformation relationship matrix between the first image and the second image is calculated through the SIFT algorithm to align the spatial positions of the target pixel points in the second image with the spatial positions of the corresponding pixel points in the third image. Among them, the target pixel point can be any one of all the pixel points in the second image.

[0083] In a possible way, the constructing the image pair according to the fourth image and the first image includes: Cropping the fourth image and the first image according to a preset corresponding position; Constructing the cropped fourth image and the first image with the same preset corresponding position into an image pair.

[0084] It should be understood that when constructing an image pair according to the first image and the fourth image, it can be constructed by cropping the fourth image and the first image correspondingly. The preset corresponding position can be the position where the fourth image is correspondingly cropped when cropping the first image. For example, if the first image needs to be cropped into nine images, the fourth image also needs to be cropped into nine images at the corresponding cropping position of the first image, so that the cropped first image and the cropped fourth image correspond one by one to form an image pair. That is to say, in this embodiment, the first image and the fourth image are respectively cropped into corresponding multiple small images, and then the multiple small images corresponding to the first image and the multiple small Figure 1 images corresponding to the fourth image are correspondingly paired to obtain an image pair composed of a low-light image and a color image.

[0085] Refer to Figure 5 , Figure 5 which is a schematic flowchart of an image pair construction method according to an exemplary embodiment of the present disclosure. As Figure 5 shown, the process of the image pair construction method includes the following steps.

[0086] S501: Set the working parameters of the first camera. The working parameters include exposure time, frame rate, gain, image format, and image size, and generate an exposure signal in the first camera to control the transmission of the exposure signal to the second camera. Its frame rate is set according to the vehicle speed during acquisition, the gain is set to 0, the image format is set to a data format of 16 bits or 8 bits, and the image size is set according to requirements. The exposure time is adjusted in combination with the attenuation rate of the attenuation sheet so that the brightness of the acquired first image is relatively dark.

[0087] S502: After the parameters are set, acquire the first image, where the first image is a RAW format image.

[0088] S503: Demosaic the first image to restore the color information of the image. Then perform white balance processing to restore the true color of the object in the image. Finally, convert the image with restored color into an 8-bit image to obtain the fifth image, and convert the fifth image into the sRGB domain for subsequent image processing.

[0089] S504: Calculate the average luminance of the fifth image.

[0090] S505: Determine the relationship between the average luminance and a preset luminance threshold. When the average luminance is greater than or equal to the preset luminance threshold, execute step S507; otherwise, execute step S506.

[0091] S506: When the average luminance is lower than the preset luminance threshold, perform histogram equalization on the fifth image to enhance the luminance of the image, and determine the processed image as the third image.

[0092] S507: Determine the fifth image as the third image.

[0093] S508: The working parameters of the second camera are the same as those of the first camera, and the second camera receives the exposure signal transmitted by the first camera to achieve synchronous exposure of the first camera and the second camera, obtaining a low-light prominent and corresponding color image, and setting the white balance of the second camera to automatic white balance.

[0094] S509: Acquire the second image through the second camera.

[0095] S510: Using the third image as a reference, register the third image and the second image using the SIFT algorithm to obtain a transformation relationship matrix.

[0096] S511: Use the image transformation matrix to transform the second image to the spatial position corresponding to the first image to obtain the fourth image, so as to achieve spatial alignment of the first image and the second image.

[0097] S512: Crop the first image and the fourth image into multiple small images and store them one by one to construct an image pair composed of a low-light image and a color image.

[0098] The specific implementation manners of the above process steps have been described in detail by way of examples above and will not be elaborated here. In addition, it should be understood that for the above system embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present disclosure is not limited by the action sequence described above. Secondly, those skilled in the art should also know that the embodiments described above are preferred embodiments, and the steps involved are not necessarily essential for the present disclosure.

[0099] Based on the same concept, this embodiment also discloses a model training method, as Figure 6 shown, Figure 6 FIG. Figure 6 is a schematic diagram showing a model training method according to an exemplary embodiment of the present disclosure. Referring to Figure 6 , it includes: S601: Training the low-light enhancement model with the image pairs constructed by the image pair construction method disclosed in this embodiment.

[0100] Training the low-light enhancement model with the image pairs constructed according to the image pair construction method can improve the performance of the low-light enhancement model and can also improve the accuracy of the low-light enhancement model. Furthermore, when the trained low-light enhancement model is used for vehicle parking, the accuracy of vehicle parking can be improved.

[0101] Based on the same concept, this embodiment also discloses an image pair construction device, as Figure 7 shown, Figure 7 FIG. Figure 7 is a schematic diagram showing an image pair construction device according to an exemplary embodiment of the present disclosure. Referring to Figure 7 , it includes an imaging system having a beam splitter 4. The imaging system is a light-shielding system provided with an opening. The beam splitter 4 is disposed at the entrance of the opening of the imaging system, and ambient light is obtained through the opening; The imaging system is configured to obtain a first image and a second image formed by ambient light after passing through the beam splitter 4 in the imaging system, and construct an image pair according to the first image and the second image, wherein the brightness of the first image is less than the brightness of the second image.

[0102] In a possible manner, the imaging system further includes a first camera 1 and a second camera 3; The first camera 1 is configured to obtain the transmitted light formed by ambient light after passing through the beam splitter 4 to obtain the first image; The second camera 3 is configured to obtain the reflected light formed by ambient light after passing through the beam splitter 4 to obtain the second image.

[0103] In a possible manner, the image pair construction device further includes an attenuation sheet 5, and the attenuation sheet 5 is disposed between the first camera 1 and the beam splitter 4; The first camera 1 is configured to obtain the light formed by ambient light after passing through the beam splitter 4 and the attenuation sheet 5 to obtain the first image.

[0104] It should be understood that, as Figure 8As shown, when the beam splitter 4 has a 50:50 splitting ratio, the area of the beam splitter 4 is determined by the field of view angles of the first camera 1 and the second camera 3. The attenuation filter 5 can reduce the light intensity of the transmitted light without changing the wavelength ratio of the transmitted light. The transmittance of the attenuation filter 5 can be 50% - 0.25%, and it can be fixed between the first camera 1 and the beam splitter 4 by threads, so as to simulate the imaging process in a low-light environment. The focal lengths of the first camera 1 and the second camera 3 can be the focal lengths commonly used in vehicle-mounted cameras, usually between 4 - 10 mm, and the adapted image sensors are usually 2 / 3 inch or 1 / 2.2 inch. The number of image pixels of the first camera 1 and the second camera 3 is greater than or equal to 2 million pixels, supporting the output of Raw images and RGB images, and supporting external trigger control.

[0105] During imaging, as Figure 8 shown, the ambient light is split into transmitted ambient light and reflected ambient light after passing through the beam splitter 4. The gray lines in the figure represent sunlight. They are respectively incident on the first camera 1 and the second camera 3 and form images in the first camera 1 and the second camera 3. Among them, an attenuation filter 5 with adjustable attenuation is installed between the first camera 1 and the beam splitter 4. The ambient light of this path is collected by the lens after passing through the beam splitter 4 and the attenuation filter 5 for imaging to acquire the first image. No attenuation filter 5 is installed between the second camera 3 and the beam splitter 4, and the ambient light directly enters the second camera 3 for imaging after passing through the beam splitter 4 to acquire the second image.

[0106] In a possible way, the size of the beam splitter 4 is obtained according to the distance from the focus of the optical axis of the beam splitter 4 to the center point of the outer surface of the second camera 3 and the viewing angle range of the first camera 1 and / or the second camera 3.

[0107] It should be understood that the size of the beam splitter 4 can be represented by the following calculation formula.

[0108]

[0109] In the formula, is the field of view angle in the viewing angle range of the first camera 1 and / or the second camera 3, d is the distance from the focus of the optical axis of the beam splitter 4 to the center point of the outer surface of the second camera, and L is the size of the beam splitter 4. If is the horizontal field of view angle in the viewing angle range of the first camera 1 and / or the second camera 3, then L is the length of the beam splitter 4. If is the vertical field of view angle in the viewing angle range of the first camera 1 and / or the second camera 3, then L is the height of the beam splitter 4. The thickness of the beam splitter 4 is generally 1 - 2 mm.

[0110] In a possible way, the imaging system further includes a plurality of light shields 2 and a bracket 6, and each side surface of the bracket 6 is provided with a groove 7; The multiple light-shielding plates 2 form a three-dimensional structure, and the edges of two adjacent light-shielding plates 2 are connected through the grooves 7 of the bracket 6 to form a light-shielding system. An opening is provided on one of the light-shielding plates 2 to allow ambient light to enter the beam splitter 4.

[0111] It should be understood that, as Figure 9 shown, the light-shielding plates 2 and the brackets 6 are provided to avoid the interference of ambient light on the imaging system. The light-shielding plates 2 and the brackets 6 form a cubic structure. The bracket 6 can be an aluminum profile bracket 6, and the light-shielding plate 2 can be an opaque plate. The aluminum profile bracket 6 can be a mature industrial fitting, and its four sides all have grooves 7. During use, a cubic framework is built through the matching connectors. The main function of the light-shielding plate 2 is to block light, and its material can be a black plate or a plate composed of light-absorbing materials such as black flannelette pasted on the inner side. During use, the light-shielding plate 2 is installed in the groove 7 of the cube built by the bracket 6 to achieve the light-shielding of the entire cube. In order to avoid the interference between parts and the occlusion of the imaging field of view, the opaque plates on each surface are grooved according to the actual structure to ensure normal imaging and debugging requirements.

[0112] By setting the light-shielding plate 2 and the bracket 6, when ambient light enters the beam splitter 4, other noises can be avoided from affecting the imaging of the imaging system. At the same time, maximum target bright-field imaging can be achieved, thereby improving the quality of the first image and the second image collected, and also improving the quality of the image pair.

[0113] In a possible way, the image pair construction device further includes a fixing structure for fixing the imaging system to the vehicle.

[0114] It should be understood that a fixing structure can also be provided on the image pair construction device, which can fix the imaging system to the vehicle. Then, when the vehicle is in operation, the imaging system can collect the ambient light around the vehicle while the vehicle is running, so as to achieve the collection of an image pair composed of low-light images and color images of the moving environment in the natural scene. An image pair can be constructed for the moving environment, which can enable technicians to collect low-light images during the vehicle's driving process, contribute to the research of low-light enhancement algorithms related to vehicle parking, and improve the vehicle's perception ability in low-light environments.

[0115] In a possible way, the fixing structure includes an optical breadboard 8, a suction cup device, and an optical fixing member; The optical fixing member is used to fix the first camera 1, the second camera 3, and the beam splitter 4 on the optical breadboard 8; The suction cup device is used to fix the optical breadboard 8 to the vehicle.

[0116] It should be understood that the suction cup devices can be set to four, which are respectively arranged at the four corners of the optical breadboard 8 for fixing the optical breadboard 8. The optical fixing members can fix the first camera 1, the second camera 3 and the beam splitter 4 on the optical breadboard 8 to collect the first image and the second image. By setting the optical fixing members to fix the optical breadboard 8 and thus fix the imaging system, the quality of the first image and the second image can be improved, and thus the quality of the image pair can be improved.

[0117] In practical applications, the fixing structure can fix the imaging system. Specifically, as Figure 10 shown in Figure 11 , the surface of the optical breadboard 8 has regularly arranged threaded through holes for quickly fixing components to the surface by threads. The suction cup devices can fix the optical breadboard 8 on the vehicle. During use, the upper ends of the four suction cup devices are fixed to the four corners of the optical breadboard 8 by threads, the suction cups at the lower ends are sucked to the smooth surface of the carrier, and the hinge is adjusted. After adjusting the optical breadboard 8 to be horizontal, the hinge is fixed.

[0118] In a possible manner, the optical fixing member includes a column 11 and a sleeve. A screw is provided on the side of the sleeve for adjusting the height of the column 11; The beam splitter 4, the first camera 1 and the second camera 3 are all connected to the corresponding sleeves through the column 11, and the sleeve connects the column 11 to the optical breadboard 8 to fix the beam splitter 4, the first camera 1 and the second camera 3.

[0119] It should be understood that multiple pairs of columns 11 and sleeves can be set, and each pair of columns 11 and sleeves can be used to fix a device. In the embodiment of the present disclosure, three sets can be set to respectively fix the first camera 1, the second camera 3 and the beam splitter 4.

[0120] In practical applications, as Figure 10 shown in Figure 11 , the sleeve can be fixed to the optical breadboard 8 by bolts. When the bolts are not tightened, the sleeve can rotate freely. After the bolts are tightened, the sleeve will be fixed. A set screw is installed on the side of the sleeve for adjusting the insertion depth and rotation angle of the column 11. There is a thread at the top of the column 11, which can be threadedly connected to optical components such as the first camera 1, the second camera 3 and the beam splitter 4. During installation, the column 11 is inserted into the sleeve, and the insertion depth and angle of the column 11 are manually adjusted so that the centers of the first camera 1, the second camera 3 and the beam splitter 4 are at the same height and specified angle, and then the set screw is tightened to fix the column 11.

[0121] Meanwhile, the light shield 2 and the bracket 6 are also fixed on the optical breadboard 8, and the first camera 1, the second camera 3, and the beam splitter 4 are protected within the light shielding system, which can prevent the beam splitter 4 from being interfered with.

[0122] In a possible manner, the suction cup device includes a double-hinged suction cup 9 and a rubber suction cup 10. One end of the double-hinged suction cup 9 is connected to the optical breadboard 8, and the other end of the double-hinged suction cup 9 is connected to the rubber suction cup 10; The rubber suction cup 10 is used to fix the optical breadboard 8 to the vehicle.

[0123] It should be understood that, as Figure 10 shown, the suction cup device is the double-hinged suction cup 9 and the rubber suction cup 10. The upper end of the double-hinged suction cup 9 has an external thread consistent with the specifications of the optical breadboard 8 for direct mechanical connection with the optical breadboard 8; the double-hinged suction cup 9 can be adjusted at any angle and fixed by a set screw. The lower end of the double-hinged suction cup 9 is the rubber suction cup 10, which can fix the optical breadboard 8 on a smooth surface, such as the engine hood of a car, through negative pressure.

[0124] Through the above technical solutions, compared with the characteristics of low frame rate image acquisition in the related art, efficient automatic image acquisition and processing can be achieved. The embodiments of the present disclosure utilize the soft and hard trigger capabilities and the image features during vehicle driving to set up an image acquisition and processing process including the first camera 1 and the second camera 3, the working parameters in each camera, image registration, image cropping, etc., and then can automatically realize the construction of image pairs. At the same time, a large data volume of image pair datasets during vehicle driving and parking can be established, and the image pairs include low-light images and color images.

[0125] Compared with the problem in the related art that it is difficult to apply to real vehicle image pair acquisition, an image pair construction device is set up, which can realize quick installation on the vehicle and image acquisition. The embodiments of the present disclosure utilize standard parts such as the suction cup device and the aluminum profile bracket 6, and combine the designed light shield 2 to build an image pair construction device, which can be installed and fixed to the roof or the front of the vehicle without damage quickly, and then can acquire images during vehicle driving. And this device can be quickly and conveniently installed above the vehicle, and image acquisition and real-time processing can be carried out, so that an efficient low-light image pair dataset can be established in the driving and parking environment.

[0126] Based on the same concept, this embodiment also discloses an image pair construction system. As Figure 12 shown, Figure 12 is a schematic diagram of an image pair construction system 1200 shown according to an exemplary embodiment of the present disclosure. Referring to Figure 12 , it includes: An image acquisition module 1201, configured to acquire a first image and a second image formed by ambient light after passing through a beam splitter in an imaging system, where the brightness of the first image is less than that of the second image, the imaging system is a light-shielding system with an opening, and the beam splitter acquires ambient light through the opening; An image pair construction module 1202, configured to construct an image pair according to the first image and the second image.

[0127] Optionally, the first image is a RAW format image and the second image is a color image; or, the first image is a color image and the second image is an infrared image.

[0128] Optionally, the imaging system includes a first camera and a second camera, and the image acquisition module 1201 is configured to: Acquire the first image formed by ambient light after passing through the beam splitter in the imaging system and entering the first camera, and the second image formed by the ambient light entering the second camera after the beam splitter.

[0129] Optionally, the sensitivity of the first camera is greater than that of the second camera, or the sensitivity of the second camera is greater than that of the first camera.

[0130] Optionally, the imaging system further includes an attenuation filter, the attenuation filter is disposed between the first camera and the beam splitter, and the image acquisition module is configured to: Acquire the first image formed by ambient light after passing through the beam splitter and the attenuation filter in the imaging system and entering the first camera.

[0131] Optionally, the image pair construction system 1200 further includes: A first control module, configured to control the exposure signal generated by the first camera to be transmitted to the second camera, so that the first camera and the second camera are exposed simultaneously; A second control module, configured to control the exposure signal generated by the second camera to be transmitted to the first camera, so that the first camera and the second camera are exposed simultaneously.

[0132] Optionally, the image pair construction module 1202 includes: A conversion module, configured to convert the first image into a third image, where the third image is a color image; A first alignment module, configured to align the spatial positions of target pixel points in the second image with the spatial positions of corresponding pixel points in the third image to obtain a fourth image; A construction module, configured to construct the image pair according to the fourth image and the first image.

[0133] Optionally, the conversion module includes: A processing module, configured to perform mosaic processing, or mosaic processing and white balance processing, on the first image to obtain a fifth image; A determination module, configured to determine the third image according to the average brightness of the fifth image.

[0134] Optionally, the determination module is configured to: When the average brightness of the fifth image is greater than or equal to the preset brightness threshold, determine the fifth image as the third image.

[0135] Optionally, the determination module is configured to: When the average brightness of the fifth image is less than the preset brightness threshold, perform histogram equalization processing on the fifth image, and determine the fifth image after histogram equalization processing as the third image.

[0136] Optionally, the first alignment module includes: A calculation module, configured to calculate a transformation relationship matrix between the first image and the second image by using the scale-invariant feature transform algorithm, with the first image as a reference image; A second alignment module, configured to align the spatial positions of the target pixel points in the second image with the spatial positions of the corresponding pixel points in the third image according to the transformation relationship matrix.

[0137] Optionally, the construction module includes: A cropping module, configured to crop the fourth image and the first image according to a preset corresponding position; A corresponding module, configured to construct the cropped fourth image and the first image with the same preset corresponding position into an image pair.

[0138] Based on the same concept, this embodiment also discloses a model training system, referring to Figure 13 , Figure 13 is a schematic diagram showing a model training system 1300 according to an exemplary embodiment of the present disclosure, as shown in Figure 13 shown, including: A model training module 1301, configured to train a low-light enhancement model by using the image pairs constructed by any one of the image pair construction methods provided in the first aspect of the present disclosure.

[0139] Based on the same concept, this embodiment also discloses an electronic device, including: A memory, on which a computer program is stored; A processor for executing the computer program in the memory to implement the steps of the image pair construction method or the model training method disclosed in this embodiment.

[0140] Figure 14 is a block diagram of an electronic device 1400 shown according to an exemplary embodiment. As Figure 14 shown, the electronic device 1400 may include: a processor 1401, a memory 1402. The electronic device 1400 may further include one or more of a multimedia component 1403, an input / output (I / O) interface 1404, and a communication component 1405.

[0141] Among them, the processor 1401 is used to control the overall operation of the electronic device 1400 to complete all or part of the steps in the above-mentioned image pair construction method and model training method. The memory 1402 is used to store various types of data to support the operation of the electronic device 1400. These data may include, for example, instructions for any application or method operating on the electronic device 1400, as well as application-related data, such as contact data, received and sent messages, pictures, audio, video, and so on. The memory 1402 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc. The multimedia component 1403 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 1402 or sent through the communication component 1405. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 1404 provides an interface between the processor 1401 and other interface modules, and the above-mentioned other interface modules may be a keyboard, a mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 1405 is used for wired or wireless communication between the electronic device 1400 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited herein. Therefore, the corresponding communication component 1405 may include: a Wi-Fi module, a Bluetooth module, an NFC module, and so on.

[0142] In one exemplary embodiment, the electronic device 1400 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components, and is used to execute the above-mentioned image pair construction method and model training method.

[0143] In another exemplary embodiment, a computer-readable storage medium including program instructions is further provided. When the program instructions are executed by a processor, the steps of the above-mentioned image pair construction method and model training method are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 1402 including program instructions, and the above-mentioned program instructions may be executed by the processor 1401 of the electronic device 1400 to complete the above-mentioned image pair construction method and model training method.

[0144] The preferred embodiments of the present disclosure have been described in detail above in conjunction with the accompanying drawings. However, the present disclosure is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present disclosure, various simple modifications can be made to the technical solutions of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0145] In addition, it should be noted that, in the above specific embodiments, the various specific technical features described can be combined in any suitable manner without conflict. To avoid unnecessary repetition, the present disclosure will not separately describe various possible combination methods.

[0146] In addition, any combination can be made between various different embodiments of the present disclosure as long as it does not violate the idea of the present disclosure, and it should also be regarded as the content disclosed by the present disclosure.

Claims

1. A method for constructing an image pair, characterized in that: include: Acquire a first image and a second image formed by ambient light after passing through a beam splitter in an imaging system, wherein the brightness of the first image is less than the brightness of the second image, the imaging system is a light shielding system provided with an opening, and the beam splitter acquires the ambient light through the opening; An image pair is constructed based on the first image and the second image.

2. The image pair construction method according to claim 1, characterized in that: The first image is a RAW format image, and the second image is a color image; or, The first image is a color image, and the second image is an infrared image.

3. The image pair construction method according to claim 1, characterized in that: The imaging system includes a first camera and a second camera, and acquires a first image and a second image formed by ambient light after passing through a beam splitter in the imaging system, including: The first image formed when the ambient light passes through the beam splitter in the imaging system and enters the first camera is acquired, and the second image formed when the ambient light passes through the beam splitter and enters the second camera is acquired.

4. The image pair construction method according to claim 3, characterized in that: The sensitivity of the first camera is greater than that of the second camera, or the sensitivity of the second camera is greater than that of the first camera.

5. The image pair construction method according to claim 3, characterized in that: The imaging system further includes an attenuation plate, which is disposed between the first camera and the beam splitter. The first image formed by obtaining the ambient light after passing through the beam splitter in the imaging system and entering the first camera includes: The first image formed by the ambient light passing through the beam splitter and the attenuation plate in the imaging system in sequence and then entering the first camera is acquired.

6. The image pair construction method according to claim 3, characterized in that: The method further comprises: controlling the exposure signal generated by the first camera to be transmitted to the second camera so that the first camera and the second camera are exposed simultaneously; or, The exposure signal generated by the second camera is controlled to be transmitted to the first camera, so that the first camera and the second camera are exposed simultaneously.

7. The image pair construction method according to claim 1, characterized in that: The step of constructing an image pair according to the first image and the second image includes: Converting the first image into a third image, wherein the third image is a color image; Aligning the spatial position of the target pixel in the second image with the spatial position of the corresponding pixel in the third image to obtain a fourth image; The image pair is constructed according to the fourth image and the first image.

8. The image pair construction method according to claim 7, characterized in that: The converting the first image into a third image comprises: Performing mosaic processing, or mosaic processing and white balance processing on the first image to obtain a fifth image; The third image is determined according to the average brightness of the fifth image.

9. The image pair construction method according to claim 8, characterized in that: The step of determining the third image according to the average brightness of the fifth image comprises: When the average brightness of the fifth image is greater than or equal to a preset brightness threshold, the fifth image is determined as the third image.

10. The image pair construction method according to claim 8, characterized in that: The step of determining the third image according to the average brightness of the fifth image comprises: When the average brightness of the fifth image is less than the preset brightness threshold, histogram equalization is performed on the fifth image, and the fifth image after the histogram equalization is determined as the third image.

11. The image pair construction method according to claim 7, characterized in that: The step of aligning the spatial position of the target pixel in the second image with the spatial position of the corresponding pixel in the third image to obtain the fourth image includes: By using a scale-invariant feature transformation algorithm, taking the first image as a reference image, calculating a transformation relationship matrix between the first image and the second image; According to the conversion relationship matrix, the spatial position of the target pixel point in the second image is aligned with the spatial position of the corresponding pixel point in the third image.

12. The image pair construction method according to claim 7, characterized in that: The step of constructing the image pair according to the fourth image and the first image includes: cropping the fourth image and the first image according to preset corresponding positions; The fourth image and the first image having the same preset corresponding position after being cropped are constructed into an image pair.

13. A model training method, characterized in that: include: The low-light enhancement model is trained using the image pairs constructed by the image pair construction method according to any one of claims 1 to 12.

14. An image pair construction device, characterized in that: An imaging system including a beam splitter, wherein the imaging system is a light shielding system provided with an opening, the beam splitter is arranged at the entrance of the opening of the imaging system, and obtains ambient light through the opening; The imaging system is used to obtain a first image and a second image formed by ambient light after passing through a beam splitter in the imaging system, and to construct an image pair based on the first image and the second image, wherein the brightness of the first image is less than the brightness of the second image.

15. The image pair construction device according to claim 14, characterized in that: The imaging system also includes a first camera and a second camera; The first camera is used to obtain the transmitted light formed by the ambient light after passing through the beam splitter to obtain the first image; The second camera is used to obtain the reflected light formed by the ambient light after passing through the beam splitter to obtain the second image.

16. The image pair construction device according to claim 15, characterized in that: The image pair construction device further comprises an attenuation plate, wherein the attenuation plate is arranged between the first camera and the beam splitter; The first camera is used to acquire light formed by the ambient light after passing through the beam splitter and the attenuation plate to obtain the first image.

17. The image pair construction device according to claim 14, characterized in that: The imaging system further comprises a plurality of light shielding plates and a bracket, each side of the bracket being provided with a groove; The plurality of shading plates form a three-dimensional structure, and the edges of two adjacent shading plates are connected through the grooves of the bracket to form a shading system, wherein an opening is provided on one of the shading plates to inject ambient light into the beam splitter.

18. The image pair construction device according to claim 15, characterized in that: The size of the beam splitter is obtained according to the distance between the focus of the optical axis of the beam splitter and the center point of the outer surface of the second camera and the viewing angle range of the first camera and / or the second camera.

19. The image pair construction device according to claim 15, characterized in that: The image pair construction device further comprises a fixing structure for fixing the imaging system to a vehicle.

20. The image pair construction device according to claim 19, characterized in that: The fixing structure includes an optical breadboard, a suction cup device and an optical fixing member; The optical fixing member is used to fix the first camera, the second camera and the beam splitter on the optical breadboard; The suction cup device is used to fix the optical breadboard to the vehicle.

21. The image pair construction device according to claim 20, characterized in that: The optical fixing member comprises a column and a sleeve, and a screw is provided on the side of the sleeve, and the screw is used to adjust the height of the column; The beam splitter, the first camera and the second camera are all connected to corresponding sleeves through a column, and the sleeve connects the column to the optical breadboard to fix the beam splitter, the first camera and the second camera.

22. The image pair construction device according to claim 20, characterized in that: The suction cup device comprises a double-hinge suction cup and a rubber suction cup, one end of the double-hinge suction cup is connected to the optical breadboard, and the other end of the double-hinge suction cup is connected to the rubber suction cup; The rubber suction cup is used to fix the optical breadboard to the vehicle.

23. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 13.

Images