Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an image processing method and device, electronic equipment and storage medium, relates to the technical field of computers, in particular to the technical field of image processing and map. The specific implementation scheme is: determining a to-be-processed image in at least one road image based on brightness information of each road image in the at least one road image; performing illumination enhancement on the to-be-processed image to obtain an enhanced image; and obtaining a road image base corresponding to the to-be-processed image based on the enhanced image. By adopting the technical scheme of the present disclosure, the visualization degree of the road image base can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to the technical field of image processing. BACKGROUND

[0002] The road image base map can be applied in the fields of electronic map, automatic driving, etc., and provides necessary information for positioning, navigation, trajectory prediction, etc. The images used for constructing the road image base map are generally collected by vehicles. Since the images have good visibility and the road traffic markings are clear and obvious under good lighting conditions, the images are generally collected in the daytime. SUMMARY

[0003] The present disclosure provides an image processing method, device, equipment and storage medium.

[0004] According to a first aspect of the present disclosure, an image processing method is provided, comprising:

[0005] determining a to-be-processed image in the at least one road image based on luminance information of each road image in the at least one road image;

[0006] performing illumination enhancement on the to-be-processed image to obtain an enhanced image;

[0007] obtaining a road image base map corresponding to the to-be-processed image based on the enhanced image.

[0008] According to a second aspect of the present disclosure, an image processing device is provided, comprising:

[0009] a determination module configured to determine a to-be-processed image in the at least one road image based on luminance information of each road image in the at least one road image;

[0010] an illumination enhancement module configured to perform illumination enhancement on the to-be-processed image to obtain an enhanced image;

[0011] a first image generation module configured to obtain a road image base map corresponding to the to-be-processed image based on the enhanced image.

[0012] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected with the at least one processor; wherein

[0015] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method in any embodiment of the present disclosure.

[0016] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method in any of the embodiments of the present disclosure.

[0017] According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method in any of the embodiments of the present disclosure.

[0018] In the technical solution of the present disclosure, the brightness information of the road image is used to determine the to-be-processed image with lower brightness from at least one road image, and the to-be-processed image is subjected to illumination enhancement to obtain an enhanced image with higher brightness, better visibility, and clear and obvious road traffic markings, which is conducive to restoring the detailed information, and the enhanced image is used to determine the road image base map corresponding to the to-be-processed image, so that the road image with lower brightness can be applied to the construction of the road image base map, and the visualization degree of the road image base map is improved.

[0019] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute limitations on the present disclosure. Among them:

[0021] Figure 1 is a flowchart of the image processing method according to the first embodiment of the present disclosure;

[0022] Figure 2 is a flowchart of step S120 according to the first embodiment of the present disclosure;

[0023] Figure 3 is a flowchart of step S110 according to the first embodiment of the present disclosure;

[0024] Figure 4 is a flowchart of the image processing method according to the second embodiment of the present disclosure;

[0025] Figure 5 is a flowchart of an application example of the image processing method according to the present disclosure;

[0026] Figure 6 is a block diagram of the image processing device according to the third embodiment of the present disclosure;

[0027] Figure 7 is a block diagram of an electronic device for implementing the image processing method according to the embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are meant to be exemplary. Therefore, it should be understood that various changes and modifications to the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted herein.

[0029] Figure 1 is a flowchart of an image processing method according to a first embodiment of the present disclosure. As shown in Figure 1 , the image processing method can include:

[0030] Step S110, determining a to-be-processed image in the at least one road image based on brightness information of each road image in the at least one road image;

[0031] Step S120, performing illumination enhancement on the to-be-processed image to obtain an enhanced image;

[0032] Step S130, obtaining a road image base corresponding to the to-be-processed image based on the enhanced image.

[0033] In the embodiments of the present disclosure, the at least one road image can be obtained by an image acquisition device, which includes but is not limited to a camera, a car recorder, a mobile phone, a tablet computer, a navigation device, etc. Alternatively, the image acquisition device can be installed on a vehicle, and the image acquisition device can acquire images of the road on which the vehicle travels during the driving of the vehicle. Exemplarily, the at least one road image can be any one of a front view, a rear view and a side view of the vehicle.

[0034] Alternatively, the brightness information of the road image can be used to represent the illumination intensity of the illumination environment in which the road is located. For example, when the road is in a bright environment such as daytime, the brightness of the road image is high; when the road is in a dark environment such as a tunnel, an overpass and night, the brightness of the road image is low. Based on the brightness information of each road image in the at least one road image, a to-be-processed image with low brightness can be determined from the at least one road image.

[0035] Alternatively, in the embodiments of the present disclosure, the illumination enhancement on the to-be-processed image can be illumination enhancement on the whole to-be-processed image, or illumination enhancement on a part of the to-be-processed image. By performing illumination enhancement on the to-be-processed image, the obtained enhanced image has high brightness, good visibility and clear road traffic markings. In addition, since the local area of the to-be-processed image is smaller than the whole area of the to-be-processed image, illumination enhancement on the local area of the to-be-processed image can reduce the calculation amount of the illumination enhancement processing and speed up the calculation.

[0036] In the related art, in the process of constructing a road image base map, the brightness of the road image acquired in a low-illumination environment is low, the detail information is seriously lost, and the visibility is poor, which reduces the accuracy and recall rate of subsequent image recognition, and is not conducive to the construction of the road image base map.

[0037] By adopting the method of the embodiments of the present disclosure, the image with lower brightness is determined from at least one road image based on the brightness information of the road image, the enhanced image with higher brightness and better visibility and clear and obvious road traffic markings can be obtained by performing illumination enhancement on the image to be processed, which helps to restore the detail information, and the road image base map corresponding to the image to be processed is determined by using the enhanced image, so that the road image with lower brightness can be applicable to the construction of the road image base map, and the visualization degree of the road image base map is improved. In the process of using the road image base map for subsequent image recognition, the accuracy and recall rate of image recognition are improved.

[0038] It should be noted that, since the vehicle flow is larger during the day, the road traffic markings are blocked, and the acquired road image is prone to invalid images. However, by adopting the method of the embodiments of the present disclosure, the road image acquired at night can be suitable for constructing the road image base map, so that the road image acquisition can be performed at night when the vehicle flow is smaller, the phenomenon of blocking the road traffic markings by vehicles is reduced, and the effectiveness of road image acquisition is improved.

[0039] In one embodiment, as shown in Figure 2 The step S120 of performing illumination enhancement on the image to be processed to obtain an enhanced image can include:

[0040] The step S210 of determining a target enhancement region in the image to be processed based on the vanishing point in the image to be processed;

[0041] The step S220 of performing enhancement on the target enhancement region based on the illumination information of the target enhancement region to obtain an enhanced image.

[0042] For example, the target enhancement region in the image to be processed is determined based on the vanishing point in the image to be processed, which includes: the region below the vanishing point in the image to be processed is taken as the target enhancement region. The vanishing point in the image to be processed can be the intersection point generated by the extension line of the road traffic markings, and the region below the vanishing point in the image to be processed is taken as the target enhancement region, which not only can retain the image of the road in the image to be processed, but also can remove the redundant images of the sky and road scenery.

[0043] In the above embodiment, the target enhancement region is determined by using the vanishing point of the image to be processed, which can reduce the calculation amount of the enhancement processing, so that the target enhancement region can be quickly enhanced based on the illumination information of the target enhancement region, and the speed of obtaining the enhanced image is accelerated.

[0044] In one embodiment, step S220, which involves enhancing the target enhancement region based on its illumination information to obtain an enhanced image, may include:

[0045] Illumination estimation is performed on the enhanced target region to obtain the corresponding illumination information.

[0046] Each pixel in the target enhancement region is processed based on illumination information and preset factors to obtain an enhanced image; wherein, the preset factors are determined based on atmospheric scattering information of multiple images in the road scene corresponding to the image to be processed.

[0047] Optionally, the LIME (Low-light Image Enhancement) algorithm is used to enhance the target enhancement region. For example, the maximum pixel value in the R (red) channel, G (green) channel, and B (blue) channel of each pixel in the target enhancement region is used as the illumination information of the corresponding pixel to obtain the illumination information T corresponding to the target enhancement region. T can be expressed by formula (1):

[0048]

[0049] Where x is a pixel, I is the target enhancement region, and c is each color channel of the target enhancement region.

[0050] The pixels of the target enhancement region in the R, G, and B channels are normalized respectively to obtain the normalized image N of the target enhancement region in each color channel. c N c It can be expressed using formula (2):

[0051]

[0052] Using the Retinex model, based on the illumination information T corresponding to the target enhancement region, the preset factor k, and the normalized image N of the target enhancement region in each color channel... c The reflection information E of the target enhancement region in the R, G, and B channels was determined respectively. c E c It can be expressed using formula (3):

[0053] E c =I one -[(I one -N c )-k*(I one -T)]. / T,c∈{R,G,B} (3)

[0054] Among them, I onedenotes an all-1 matrix, and. / T denotes pixel-wise division. The preset factor can be obtained based on statistical information of atmospheric scattering of a plurality of images in a road scene corresponding to the to-be-processed image. For example, the road scene corresponding to the to-be-processed image can be a road scene in a low-light condition such as a tunnel, an overpass, and night. Preferably, k = 0.905.

[0055] The reflection information E of the target enhancement region in the R channel R , the reflection information E in the G channel G , and the reflection information E in the B channel B are superimposed to obtain an enhanced image of the target enhancement region.

[0056] In the above embodiment, the target enhancement region is enhanced by using the illumination information and the preset factor, which makes the enhancement processing more suitable for the road scene in the low-light condition, and makes the enhanced image more natural.

[0057] In an embodiment, as shown in Figure 3 the step S110 of determining the to-be-processed image in the at least one road image based on the brightness information of each road image in the at least one road image comprises:

[0058] The step S310 of determining the brightness feature distribution information of the i-th road image based on the brightness information of the i-th road image in the at least one road image, wherein i is an integer greater than or equal to 1.

[0059] The step S320 of determining the low-brightness region proportion corresponding to the i-th road image based on the brightness feature distribution information.

[0060] The step S330 of determining the i-th road image as the to-be-processed image in the case where the low-brightness region proportion meets a preset condition.

[0061] Optionally, the step S310 can comprise: converting the i-th road image from an RGB color space image to an HSV color space image, the HSV color space mainly describing the i-th road image from three dimensions of hue, saturation, and brightness; obtaining a V channel image of the i-th road image; dividing the pixels of the V channel image of the i-th road image into N brightness regions; and statistically determining the proportion of each brightness region. In this way, the brightness feature distribution information of the i-th road image can be accurately extracted. In order to speed up the calculation, the V channel image of the i-th road image can also be sampled according to a preset step length, and then the pixels of the sampled V channel image are divided into brightness regions. The step length can be selected and adjusted according to actual needs, for example, the step length can include but is not limited to 4 pixels.

[0062] Exemplarily, the step S320 can include: counting proportions of the first luminance region to the nth luminance region, n and N are integers greater than or equal to 1, and n is less than N; the proportions of the first luminance region to the nth luminance region are taken as a low luminance region proportion p, which can be expressed by formula (4):

[0063]

[0064] wherein j is an integer greater than or equal to 1, and j is less than or equal to N, hist j represents the number of the jth luminance region. For example, N = 32, n = 6,

[0065] Exemplarily, the low luminance region proportion meeting the preset condition can be that the low luminance region proportion exceeds a first threshold value. For example, in the case that the proportions of the first luminance region to the sixth luminance region are greater than 0.65, it can be determined that the low luminance region proportion in the V channel image of the ith road image is large, the ith road image is dark as a whole, and the ith road image is a low luminance image. The ith road image can be determined as a to-be-processed image.

[0066] The above embodiment determines the low luminance region proportion of the ith road image through the luminance distribution information of the ith road image, and determines the ith road image as a to-be-processed image when the low luminance region proportion meets the preset condition, which can improve the accuracy of low luminance image determination.

[0067] In an embodiment, the above step S330, in the case that the low luminance region proportion meets the preset condition, includes:

[0068] determining a luminance standard deviation based on the luminance feature distribution information of the ith road image;

[0069] determining a darkness coefficient of the ith road image based on the luminance standard deviation and the low luminance region proportion;

[0070] determining the ith road image as a to-be-processed image in the case that the low luminance region proportion is greater than a first threshold value and the darkness coefficient is greater than a second threshold value.

[0071] Exemplarily, the luminance standard deviation can be denoted by δ, the first threshold value can be 0.65, and the second threshold value can be 6.0. The darkness coefficient r of the ith road image can be determined by formula (5) as follows:

[0072]

[0073] In a case that p is greater than or equal to 0.65 and r is greater than or equal to 6.0, it can be determined that the brightness of the i-th road image is low, and the i-th road image is too dark as a whole.

[0074] In the above embodiment, the i-th road image can be accurately determined as a too dark road image by using the low brightness distinguishing proportion of the i-th road image being greater than the first threshold value and the darkness coefficient being greater than the second threshold value, and the accuracy of the low brightness image determination can be further improved.

[0075] In an embodiment, the step S130 of obtaining the road image base map corresponding to the to-be-processed image based on the enhanced image comprises:

[0076] The enhanced image is projected based on the camera parameter of the image acquisition device corresponding to the to-be-processed image, the position information and the attitude information corresponding to the to-be-processed image, to obtain the road image base map corresponding to the to-be-processed image.

[0077] In the above embodiment, the enhanced image is projected in combination with the position information and the attitude information of the image acquisition device, so that the projection position and angle of the enhanced image are accurate, and the authenticity of the road image base map can be improved.

[0078] Optionally, the projection processing of the enhanced image can comprise: converting the enhanced image into an image A1 in a camera coordinate system based on the intrinsic matrix of the image acquisition device, and de-distorting the image A1 in the camera coordinate system by using a distortion coefficient to obtain an image A2. The image A2 in the camera coordinate system is converted into an image A3 in a pixel coordinate system, and the image A3 is the image obtained after the de-distortion of the enhanced image; and the de-distorted enhanced image is projected based on the position information and the attitude information corresponding to the to-be-processed image to obtain the road image base map corresponding to the to-be-processed image. In this way, the distortion of the road image base map obtained by projection can be avoided, and the road image base map is more authentic.

[0079] Exemplarily, the camera parameter of the image acquisition device can comprise camera intrinsic parameters and camera extrinsic parameters. The camera intrinsic parameters comprise an intrinsic matrix M1 and a distortion coefficient. The camera extrinsic parameters can comprise a rotation matrix R and a translation matrix t. The camera intrinsic parameters and the camera extrinsic parameters can be calibrated in advance by using a conventional calibration method. For example, the camera intrinsic parameters can be calibrated by using the Zhang Zhengyou camera calibration method to obtain the intrinsic matrix M1; and a space reference observation point is established, and an extrinsic matrix is determined by using the conversion relationship between the world coordinate system and the pixel coordinate system. For example, the conversion relationship between the world coordinate system and the pixel coordinate system can be expressed by the following formula (6):

[0080]

[0081] wherein u is a pixel coordinate of a pixel point of the two-dimensional image in the X-axis direction, v is a pixel coordinate of the pixel point of the two-dimensional image in the Y-axis direction, Z c is a coordinate of the pixel point of the two-dimensional image in the Z-axis direction in the camera coordinate system, M2 is an extrinsic matrix of the camera, X w , Y w and Z w are coordinates of the pixel point of the three-dimensional image in the X-axis, Y-axis and Z-axis directions in the world coordinate system, respectively.

[0082] Since M1 is determined, Z c can be established by canceling out the common factor, and the coordinates of the reference observation point in the world coordinate system and the pixel coordinates of the reference observation point in the pixel coordinate system can be determined by establishing a space reference observation point. Using M1, the coordinates of the reference observation point in the world coordinate system and the pixel coordinates of the reference observation point, M2 in the above formula (6) can be solved, and the calibration of the extrinsic matrix M2 is realized.

[0083] Exemplarily, the position information of the image to be processed can be collected by a Global Positioning System (GPS) and the attitude information can be collected by an Inertial Measurement Unit (IMU). For example, the GPS system and the IMU unit are installed on a vehicle together with the image collection device. During the driving of the vehicle, the GPS system collects the position information of the image collection device and the IMU unit collects the attitude information of the image collection device.

[0084] Optionally, in the case that the position information and the attitude information of the image to be processed are inconsistent with the collection time of the image to be processed, the position information and the attitude information of the image to be processed can be linearly interpolated respectively, so that the collection time of the position information and the attitude information of the image to be processed is consistent with the collection time of the image to be processed. Based on this, the interval between adjacent road image base maps can be prevented, and the accuracy of the projection position and angle can be improved.

[0085] In an embodiment, as Figure 4 shown, the method can further include:

[0086] Step S410, obtaining a bottom layer tile map corresponding to each of the at least one road image based on a road image base map corresponding to each of the at least one road image and position information corresponding to each of the at least one road image;

[0087] Step S420, performing multiple scaling on the bottom layer tile map to obtain a multi-layer tile map corresponding to the bottom layer tile map.

[0088] In step S430, a tile pyramid is obtained based on the multi-layer tile map, wherein the tile pyramid is used to reconstruct a scene corresponding to the at least one road image.

[0089] Exemplarily, the road image base map corresponding to each road image of the at least one road image can be one or more. Taking the case of multiple road base maps as an example, the multiple road base maps can be spliced according to the position information corresponding to the multiple road base maps, so as to obtain a bottom layer tile map corresponding to the road base map.

[0090] Exemplarily, the multiple scaling of the bottom layer tile map can include: taking the bottom layer tile map as a 0th layer tile map; scaling the 0th layer tile map according to a method of combining every 2*2 pixels into one pixel to generate a 1st layer tile map; scaling the 1st layer tile map according to the method of combining every 2*2 pixels into one pixel to generate a 2nd layer tile map. In this way, after the multiple scaling of the bottom layer tile map, a multi-layer tile map corresponding to the bottom layer tile map is obtained.

[0091] Exemplarily, the obtaining of the tile pyramid based on the multi-layer tile map can include: dividing each layer tile map in the multi-layer tile map into square tiles of the same size (for example, 256*256 pixels) respectively to form a tile matrix corresponding to each layer tile map; and taking the tile matrices corresponding to the multi-layer tile map as the tile pyramid.

[0092] Optionally, the scene corresponding to the at least one road image includes but is not limited to a high-definition map, positioning, navigation, trajectory prediction, etc.

[0093] In the above embodiments, the road image base map and the position information corresponding thereto are used to obtain a bottom layer tile map corresponding to the road image base map, and a tile pyramid is obtained based on the bottom layer tile map, so as to facilitate the storage, transmission and provision of the road image base map to an application program or other equipment, and the convenience of using the road image base map is improved.

[0094] In one embodiment, based on the road image base map corresponding to each road image in the at least one road image and the position information corresponding to each road image, a bottom layer tile map corresponding to the at least one road image is obtained, including:

[0095] Based on the road image base map corresponding to each road image in the at least one road image and the position information corresponding to each road image, an initial tile map is obtained.

[0096] Based on a preset algorithm, the initial tile map is subjected to neighborhood enhancement to obtain the bottom layer tile map corresponding to the at least one road image.

[0097] Exemplarily, a CLAHE (Contrast Limited Histogram Equalization) algorithm can be employed to perform neighborhood enhancement on the initial tile map, where the initial tile map can be divided into square tiles of a preset size (for example, 5*5 pixels) for enhancement processing.

[0098] The above scheme employs the CLAHE algorithm to perform neighborhood enhancement on the initial tile map, which can make the lane lines, lane edge lines, and turning arrows and other road traffic markings in the initial tile map more clear and obvious. In addition, the CLAHE algorithm can maintain the continuity of the image when improving the contrast of the initial tile map, so that the obtained bottom layer tile map has good visual continuity, thereby making the bottom layer tile map more real.

[0099] Figure 5 is a flow chart of an application example of the image processing method according to the present disclosure. As shown in Figure 5 , taking the front view of the vehicle collected by the image collection device as an example, the image processing method can include:

[0100] Step S510, acquiring the front view of the vehicle;

[0101] Step S520, performing illumination enhancement on the front view of the vehicle to obtain an enhanced image;

[0102] Step S530, acquiring position information and attitude information of the image collection device corresponding to the front view;

[0103] Step S540, aligning the sampling time of the position information and the attitude information with the sampling time of the front view, so that the sampling times of the front view, the position information, and the attitude information are consistent;

[0104] Step S550, performing de-warping on the enhanced image using a preset camera parameter, and projecting the de-warping enhanced image based on the position information and the attitude information to obtain a road image base map;

[0105] Step S560, obtaining an initial tile map based on the road image base map corresponding to each front view image in the at least one front view image and the position information corresponding to each road image base map;

[0106] Step S570, performing neighborhood enhancement on the initial tile map using a preset algorithm to obtain a bottom layer tile map corresponding to the at least one front view;

[0107] Step S580, performing multiple scaling on the bottom layer tile map to obtain multiple layer tile maps, and performing segmentation on each layer tile map to obtain a tile matrix corresponding to each layer tile map; taking the tile matrices corresponding to the multiple layer tile maps as a tile pyramid.

[0108] The method of the above example performs illumination enhancement through the front view of the vehicle, so that the front view of the vehicle collected in a low-illumination environment such as a tunnel, an overpass, and at night can have a good visual effect, and then a high-definition and high-precision road image base map can be generated using the enhanced image of the front view. In addition, after generating the initial tile map using the road image base map, the neighborhood enhancement of the initial tile map can also make the lane lines, edge lines, and turning arrows and other road traffic markings in the underlying tile map clearer and more obvious. Furthermore, generating a tile pyramid based on the underlying tile map also facilitates the storage, transmission, and use of the road image base map.

[0109] Figure 6 is a block diagram of an image processing apparatus according to a third embodiment of the present disclosure. As shown in Figure 6 the image processing apparatus 600 can include:

[0110] A determination module 610 is configured to determine, from the at least one road image, a to-be-processed image based on brightness information of each road image in the at least one road image.

[0111] An illumination enhancement module 620 is configured to perform illumination enhancement on the to-be-processed image to obtain an enhanced image.

[0112] A first image generation module 630 is configured to obtain a road image base map corresponding to the to-be-processed image based on the enhanced image.

[0113] In an implementation, the illumination enhancement module 620 includes:

[0114] A first determination sub-module is configured to determine, from the to-be-processed image, a target enhancement region based on a vanishing point in the to-be-processed image.

[0115] An enhancement processing sub-module is configured to perform enhancement on the target enhancement region based on illumination information of the target enhancement region to obtain the enhanced image.

[0116] In an implementation, the enhancement processing sub-module includes:

[0117] An illumination estimation unit is configured to perform illumination estimation on the target enhancement region to obtain illumination information corresponding to the target enhancement region.

[0118] An enhancement processing unit is configured to perform processing on each pixel of the target enhancement region based on the illumination information and a preset factor to obtain the enhanced image, wherein the preset factor is determined based on atmospheric scattering information of a plurality of images in a road scene corresponding to the to-be-processed image.

[0119] In an implementation, the determination module 610 includes:

[0120] The second determining sub-module is configured to determine brightness feature distribution information of the i-th road image based on brightness information of the i-th road image in the at least one road image, where i is an integer greater than or equal to 1.

[0121] The third determining sub-module is configured to determine a low-brightness area proportion corresponding to the i-th road image based on the brightness feature distribution information.

[0122] The fourth determining sub-module is configured to determine the i-th road image as a to-be-processed image in a case where the low-brightness area proportion meets a preset condition.

[0123] In an implementation, the fourth determining sub-module includes:

[0124] The first determining unit is configured to determine a brightness standard deviation based on the brightness feature distribution information of the i-th road image.

[0125] The second determining unit is configured to determine a darkness coefficient of the i-th road image based on the brightness standard deviation and the low-brightness area proportion.

[0126] The third determining unit is configured to determine the i-th road image as the to-be-processed image in a case where the low-brightness area proportion is greater than a first threshold value and the darkness coefficient is greater than a second threshold value.

[0127] In an implementation, the first image generation module 630 includes:

[0128] The projection processing sub-module is configured to perform projection processing on the enhanced image based on camera parameters of an image acquisition device corresponding to the to-be-processed image, position information and attitude information corresponding to the to-be-processed image, to obtain a road image base map corresponding to the to-be-processed image.

[0129] In an implementation, the device further includes:

[0130] The second image generation module is configured to obtain a bottom layer tile map corresponding to the at least one road image based on the road image base map corresponding to each road image in the at least one road image and position information corresponding to each road image.

[0131] The third image generation module is configured to perform multiple scaling on the bottom layer tile map to obtain a multi-layer tile map corresponding to the bottom layer tile map.

[0132] The fourth image generation module is configured to obtain a tile pyramid based on the multi-layer tile map, where the tile pyramid is used to reconstruct a scene corresponding to the at least one road image.

[0133] In an implementation, the second image generation module includes:

[0134] The first image generation sub-module is configured to obtain an initial tile map based on a road image corresponding to each of the at least one road image and position information corresponding to each of the at least one road image.

[0135] The second image generation sub-module is configured to perform neighborhood enhancement on the initial tile map based on a preset algorithm to obtain a bottom layer tile map corresponding to the at least one road image.

[0136] In the technical solution of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0137] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0138] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0139] As shown in Figure 7 The electronic device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0140] Various components in the electronic device 700 are connected to the I / O interface 705, including an input unit 706, such as a keyboard, a mouse, and the like; an output unit 707, such as various types of displays, speakers, and the like; a storage unit 708, such as a magnetic disk, an optical disk, and the like; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0141] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the image processing method. For example, in some embodiments, the image processing method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the image processing method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the image processing method by any other appropriate means, such as by means of firmware.

[0142] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0143] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0144] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0146] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0147] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0148] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology described in the present disclosure are achieved.

[0149] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image processing method, comprising: Based on the brightness information of each road image in at least one road image under the conditions of tunnels, overpasses, and nighttime, the image to be processed is determined in the at least one road image, including: determining the proportion of low brightness areas and the brightness standard deviation of the i-th road image based on the brightness feature distribution information of the i-th road image; determining the darkness coefficient of the i-th road image based on the brightness standard deviation and the proportion of low brightness areas; and determining the i-th road image as the image to be processed when the proportion of low brightness areas is greater than a first threshold and the darkness coefficient is greater than a second threshold, where i is an integer greater than or equal to 1. The region located below the vanishing point in the image to be processed is taken as the target enhancement region, wherein the vanishing point is the intersection point generated by the extension lines of the road traffic markings in the image to be processed; Illumination enhancement is performed on the target enhancement region in the image to be processed to obtain an enhanced image; Based on the enhanced image, a road image base map corresponding to the image to be processed is obtained.

2. The method according to claim 1, wherein, The target enhancement region in the image to be processed is illuminated to obtain an enhanced image, including: The target enhancement region is enhanced based on the illumination information of the target enhancement region to obtain an enhanced image.

3. The method according to claim 2, wherein, The enhancement of the target enhancement region based on the illumination information of the target enhancement region to obtain an enhanced image includes: Illumination estimation is performed on the target enhancement region to obtain the illumination information corresponding to the target enhancement region; Based on the illumination information and a preset factor, each pixel of the target enhancement region is processed to obtain an enhanced image; wherein, the preset factor is determined based on atmospheric scattering information of multiple images of the road scene corresponding to the image to be processed.

4. The method according to any one of claims 1-3, further comprising: Based on the brightness information of the i-th road image in the at least one road image, the brightness feature distribution information of the i-th road image is determined.

5. The method according to any one of claims 1-3, wherein, The step of obtaining the road image base map corresponding to the image to be processed based on the enhanced image includes: Based on the camera parameters of the image acquisition device corresponding to the image to be processed, the location information and attitude information corresponding to the image to be processed, the enhanced image is projected to obtain the road image base map corresponding to the image to be processed.

6. The method according to claim 5, further comprising: Based on the road image base map corresponding to each road image in the at least one road image and the location information corresponding to each road image, the underlying tile map corresponding to the at least one road image is obtained; The bottom layer tile map is scaled multiple times to obtain the multi-layer tile map corresponding to the bottom layer tile map; Based on the multi-layer tile map, a tile pyramid is obtained, wherein the tile pyramid is used to reconstruct the scene corresponding to the at least one road image.

7. The method according to claim 6, wherein, The step of obtaining the underlying tile map corresponding to the at least one road image based on the road image base map corresponding to each road image and the location information corresponding to each road image includes: An initial tile map is obtained based on the road image base map corresponding to each road image in the at least one road image and the location information corresponding to each road image; The initial tile map is enhanced with a preset algorithm to obtain the underlying tile map corresponding to the at least one road image.

8. An image processing apparatus, comprising: The determination module is used to determine the image to be processed from at least one road image based on the brightness information of each road image in at least one road image under the conditions of a tunnel, an overpass, and nighttime. An illumination enhancement module is used to enhance the illumination of a target enhancement region in the image to be processed, thereby obtaining an enhanced image; wherein, the illumination enhancement module includes a first determining submodule, used to take the region located below the vanishing point in the image to be processed as the target enhancement region, wherein the vanishing point is the intersection point generated by the extension lines of road traffic markings in the image to be processed; The first image generation module is used to obtain a road image base map corresponding to the image to be processed based on the enhanced image; The determining module includes: The third determining submodule is used to determine the proportion of low-brightness areas corresponding to the i-th road image based on the brightness feature distribution information of the i-th road image, where i is an integer greater than or equal to 1; The fourth determining submodule includes: a first determining unit, used to determine the brightness standard deviation based on the brightness feature distribution information of the i-th road image; a second determining unit, used to determine the darkness coefficient of the i-th road image based on the brightness standard deviation and the proportion of low brightness areas; and a third determining unit, used to determine the i-th road image as an image to be processed when the proportion of low brightness areas is greater than a first threshold and the darkness coefficient is greater than a second threshold.

9. The apparatus according to claim 8, wherein, The illumination enhancement module also includes: The enhancement processing submodule is used to enhance the target enhancement region based on the illumination information of the target enhancement region to obtain an enhanced image.

10. The apparatus according to claim 9, wherein, The enhanced processing submodule includes: An illumination estimation unit is used to perform illumination estimation on the target enhancement region to obtain illumination information corresponding to the target enhancement region; An enhancement processing unit is used to process each pixel of the target enhancement region based on the illumination information and a preset factor to obtain an enhanced image; wherein the preset factor is determined based on atmospheric scattering information of multiple images of the road scene corresponding to the image to be processed.

11. The apparatus according to any one of claims 8-10, wherein, The determining module further includes: The second determining submodule is used to determine the brightness feature distribution information of the i-th road image based on the brightness information of the i-th road image in the at least one road image.

12. The apparatus according to any one of claims 8-10, wherein, The first image generation module includes: The projection processing submodule is used to perform projection processing on the enhanced image based on the camera parameters of the image acquisition device corresponding to the image to be processed, the position information and attitude information corresponding to the image to be processed, to obtain the road image base map corresponding to the image to be processed.

13. The apparatus of claim 12, further comprising: The second image generation module is used to obtain the underlying tile map corresponding to the at least one road image based on the road image base map corresponding to each road image in the at least one road image and the location information corresponding to each road image. The third image generation module is used to scale the bottom tile image multiple times to obtain a multi-layer tile image corresponding to the bottom tile image; The fourth image generation module is used to obtain a tile pyramid based on the multi-layer tile image, wherein the tile pyramid is used to reconstruct the scene corresponding to the at least one road image.

14. The apparatus according to claim 13, wherein, The second image generation module includes: The first image generation submodule is used to obtain an initial tile image based on the road image base map corresponding to each road image in the at least one road image and the location information corresponding to each road image; The second image generation submodule is used to perform neighborhood enhancement on the initial tile map based on a preset algorithm to obtain the bottom tile map corresponding to the at least one road image.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

17. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images