Driving record image generation method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a driving record image generation method and device, equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a target long-exposure image and a target short-exposure image with synchronous exposure starting points; measuring a long exposure brightness value corresponding to a long exposure image feature obtained based on the target long exposure image, and a short exposure brightness value corresponding to a short exposure image feature obtained based on the target short exposure image; if the long exposure brightness value does not conform to a preset pixel fusion interval, determining the short exposure image feature as a target image feature, and if the short exposure brightness value does not conform to the pixel fusion interval, determining the long exposure image feature as the target image feature; otherwise, fusing the long-exposure image features and the short-exposure image features to obtain target image features; and splicing the plurality of target image features to obtain a target driving record image. The quality of the generated driving record image can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for generating driving record images. Background Art

[0002] Generally, a vehicle uses a dashcam to record its driving conditions, where the recorded video is composed of multiple frames of driving record images. When driving at night, the vehicle can only rely on its own headlights or surrounding ambient light for illumination. Under such extremely unstable lighting conditions, the recorded driving record images are often prone to overexposure or underexposure.

[0003] In the related art, long-exposure images and short-exposure images with different exposure times are obtained by sequentially shooting, and the long-exposure images and short-exposure images are fused to output a driving record image; however, there is a large misalignment between the long-exposure image and the short-exposure image obtained in this way, which makes it impossible to align the pixels in the long-exposure image and the short-exposure image, affecting the recognition of key identifiers such as license plates and signboards. That is, there is a problem of poor quality of the generated driving record images in the related art. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a method, device, equipment and storage medium for generating driving record images, which can improve the quality of the generated driving record images.

[0005] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a method for generating a driving record image, the method comprising:

[0006] Acquire a target long exposure image obtained by shooting with a first camera module based on a preset long exposure time, and a target short exposure image obtained by shooting with a second camera module based on a preset short exposure time, wherein the exposure starting points of the first camera module and the second camera module are synchronized;

[0007] Extracting long exposure image features and short exposure image features from the same target position of the target long exposure image and the target short exposure image, respectively, and measuring a long exposure brightness value corresponding to the long exposure image feature and a short exposure brightness value corresponding to the short exposure image;

[0008] If the long exposure brightness value does not match the preset pixel fusion interval, the short exposure image feature is determined to be the target image feature; if the short exposure brightness value does not match the pixel fusion interval, the long exposure image feature is determined to be the target image feature;

[0009] If the long exposure brightness value and the short exposure brightness value both match the pixel fusion interval, the long exposure image features and the short exposure image features are fused to obtain the target image features;

[0010] The target driving record image is obtained by splicing multiple target image features.

[0011] In some embodiments, after the target driving record image is obtained by splicing multiple target image features, the method further includes:

[0012] In a current time period, a plurality of long-exposure images obtained by shooting with a first camera module based on a long exposure time and a plurality of short-exposure images obtained by shooting with a second camera module based on a short exposure time are obtained, wherein the exposure starting points of the first camera module and the second camera module are synchronized;

[0013] Measuring and obtaining a plurality of image brightness values ​​corresponding to each long exposure image or each short exposure image, and calculating and obtaining a plurality of brightness difference values ​​representing the brightness difference between brightness values ​​of adjacent images;

[0014] Determine the environment area information jointly according to the multiple brightness difference values ​​and the multiple image brightness values, and dynamically determine the pixel fusion interval corresponding to the next time period according to the environment area information;

[0015] Based on the pixel fusion interval, a plurality of target image features are extracted from the target long exposure image and / or the target short exposure image captured in the next time period, and the target driving record image is obtained by splicing the plurality of target image features.

[0016] In some embodiments, determining the environment area information based on a plurality of brightness difference values ​​and a plurality of image brightness values ​​jointly includes:

[0017] If the plurality of brightness difference values ​​all belong to the preset small difference interval, and the plurality of image brightness values ​​belong to the preset bright value interval, it is determined that the environmental area information is the always-bright area information;

[0018] If the plurality of brightness difference values ​​all belong to the preset small difference interval, and the plurality of image brightness values ​​belong to the preset dim value interval, it is determined that the environmental area information is the normally dark area information;

[0019] If the plurality of brightness difference values ​​all belong to the preset large difference interval, it is determined that the environmental area information is stroboscopic area information.

[0020] In some embodiments, dynamically determining the pixel fusion interval corresponding to the next time period according to the environmental area information includes:

[0021] If the environmental area information is the always-on area information, determining that the pixel fusion interval corresponding to the next time period is the first pixel fusion interval;

[0022] If the environmental area information is the normally dark area information, it is determined that the pixel fusion interval corresponding to the next time period is the second pixel fusion interval, wherein the first pixel fusion interval and the second pixel fusion interval are different.

[0023] In some embodiments, the next time period includes a first capture moment and a second capture moment;

[0024] Dynamically determine the pixel fusion interval corresponding to the next time period according to the environmental area information, and also include:

[0025] If the environmental area information is stroboscopic area information, the shooting interval is determined based on any two long-exposure images or short-exposure images taken adjacently at the same time;

[0026] The pixel fusion interval corresponding to the first shooting moment is determined as the third pixel fusion interval, and the pixel fusion interval corresponding to the second shooting moment is determined as the fourth pixel fusion interval, wherein the difference between the first shooting moment and the second shooting moment is the shooting interval length, and the third pixel fusion interval and the fourth pixel fusion interval are different.

[0027] In some embodiments, multiple target image features are extracted from the target long exposure image and / or the target short exposure image captured in the next time period based on the pixel fusion interval, including:

[0028] Extracting long exposure image features and short exposure image features from the same target position of the target long exposure image and the target short exposure image, respectively, and measuring a long exposure brightness value corresponding to the long exposure image feature and a short exposure brightness value corresponding to the short exposure image;

[0029] If the long exposure brightness value does not match the preset pixel fusion interval, the short exposure image feature is determined to be the target image feature; if the short exposure brightness value does not match the pixel fusion interval, the long exposure image feature is determined to be the target image feature;

[0030] If the long exposure brightness value and the short exposure brightness value both match the pixel fusion interval, the long exposure image features and the short exposure image features are fused to obtain the target image features;

[0031] Based on the target image features of all target positions, multiple target image features are obtained.

[0032] In some embodiments, the long exposure image features and the short exposure image features are fused to obtain the target image features, including:

[0033] Based on a preset first weight, the long exposure image feature is updated to obtain a target long exposure image feature;

[0034] Based on a preset second weight, the short exposure image feature is updated to obtain a target short exposure image feature;

[0035] The long exposure image features and the short exposure image features are stitched together to obtain the target image features.

[0036] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a device for generating a driving record image, the device comprising:

[0037] An acquisition module, used to acquire a target long exposure image obtained by shooting with a first camera module based on a preset long exposure time, and a target short exposure image obtained by shooting with a second camera module based on a preset short exposure time, wherein the exposure starting points of the first camera module and the second camera module are synchronized;

[0038] A processing module, used to extract long exposure image features and short exposure image features from the same target position of the target long exposure image and the target short exposure image, respectively, and measure a long exposure brightness value corresponding to the long exposure image feature and a short exposure brightness value corresponding to the short exposure image;

[0039] A first determination module is used to determine the short exposure image feature as the target image feature if the long exposure brightness value does not match the preset pixel fusion interval, and to determine the long exposure image feature as the target image feature if the short exposure brightness value does not match the pixel fusion interval;

[0040] A second determination module is used to fuse the long exposure image features and the short exposure image features to obtain the target image features if both the long exposure brightness value and the short exposure brightness value are consistent with the pixel fusion interval;

[0041] The target generation module is used to stitch together multiple target image features to obtain a target driving record image.

[0042] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the method for generating a driving record image of the above-mentioned first aspect when executing the computer program.

[0043] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for generating a driving record image of the above-mentioned first aspect is implemented.

[0044] The present application proposes a method, device, equipment and storage medium for generating a driving record image, which obtains a target long exposure image obtained by shooting with a first camera module based on a preset long exposure time, and a target short exposure image obtained by shooting with a second camera module based on a preset short exposure time, wherein the exposure starting points of the first camera module and the second camera module are synchronized; extracts long exposure image features and short exposure image features from the same target position of the target long exposure image and the target short exposure image, respectively, and measures a long exposure brightness value corresponding to the long exposure image feature and a short exposure brightness value corresponding to the short exposure image; if the long exposure brightness value does not match the preset pixel fusion interval, determines the short exposure image feature as the target image feature, and if the short exposure brightness value does not match the pixel fusion interval, determines the long exposure image feature as the target image feature; if both the long exposure brightness value and the short exposure brightness value match the pixel fusion interval, fuses the long exposure image feature and the short exposure image feature to obtain the target image feature; and splices multiple target image features to obtain the target driving record image. The embodiment of the present application uses two camera modules to shoot at the same time, ensuring that the target objects of the long exposure image and the short exposure image are basically consistent in spatial position, and there is no need to consume too much computing power resources for global registration later; and the spatial parallax effect of the registered image is basically negligible, and a good overlapping fusion effect can be achieved, and there will be no problems such as the movement dislocation of the target object (such as the license plate) and the inability to restore after fusion due to the excessive delay between the long and short frames. That is, the embodiment of the present application uses two camera modules that shoot exposure images at the same time, and through the "space for time" method, it avoids the movement dislocation of the front and rear frames caused by the difference in the starting point of the long exposure time and the short exposure time in the related art, thereby improving the quality of the generated target driving record image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is an optional flow chart of the method for generating a driving record image provided in an embodiment of the present application;

[0046] Figure 2 It is an optional timeline schematic diagram of the method for generating a driving record image provided in an embodiment of the present application;

[0047] Figure 3 It is a schematic diagram of an optional functional module of the device for generating driving record images provided in an embodiment of the present application;

[0048] Figure 4 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0052] Generally, a vehicle uses a dashcam to record its driving conditions, where the recorded video is composed of multiple frames of driving record images. When driving at night, the vehicle can only rely on its own headlights or surrounding ambient light for illumination. Under such extremely unstable lighting conditions, the recorded driving record images are often prone to overexposure or underexposure.

[0053] In the related art, long-exposure images and short-exposure images with different exposure times are obtained by shooting successively, and the long-exposure images and the short-exposure images are fused to output a driving record image; however, there is a large misalignment phenomenon between the long-exposure image and the short-exposure image obtained in this way, which makes it impossible to align the pixels in the long-exposure image and the short-exposure image, affecting the recognition of key markers such as license plates and signboards. That is, there is a problem of poor quality of the generated driving record images in the related art. Based on this, the embodiments of the present application provide a method, device, equipment and storage medium for generating driving record images, aiming to improve the quality of the generated driving record images.

[0054] It should be noted first that in the embodiments of the present application, when it comes to information related to the user's identity or characteristics based on the user's basic information, user's geographic location information, etc., the user's permission or consent will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiments of the present application need to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained first. After the user's separate permission or consent is clearly obtained, the necessary data for the normal operation of the embodiments of the present application will be obtained. For example, when obtaining the target long-exposure image and the target short-exposure image recorded during driving, the consent of the relevant user will be obtained first, otherwise the target long-exposure image and the target short-exposure image used in the embodiments of the present application cannot be obtained.

[0055] The method, device, equipment and storage medium for generating driving record images provided in the embodiments of the present application are specifically explained through the following embodiments. First, the application scenario of the device for generating driving record images in the embodiments of the present application is described, wherein the method for generating driving record images provided in the embodiments of the present application (for the convenience of description, it may also be referred to as "generating method" hereinafter) can be applied to the device for generating driving record images (for the convenience of description, it may also be referred to as "generating device" hereinafter). In an application scenario, a target vehicle is provided with a driving recorder, and the driving recorder is provided with a generating device proposed in an embodiment of the present application. During the driving process of the vehicle, the generating device obtains a target long exposure image and a target short exposure image at different exposure times in the current time period, determines the corresponding long exposure image features and short exposure image features based on the same target position of the target long exposure image and the target short exposure image, and determines the long exposure brightness value and the short exposure brightness value corresponding to the long exposure image features and the short exposure image features; compares the long exposure brightness value and the short exposure brightness value with a preset pixel fusion interval, so as to determine how to obtain the target image features based on the long exposure image features and the short exposure image features, and then obtains the target driving record image by splicing multiple target image features. It can be understood that the target objects (the picture content in the captured image) in the long exposure image and the short exposure image obtained by the generating device proposed in the present application are basically consistent in spatial position, avoiding the phenomenon of front and rear frame motion misalignment caused by the difference in the starting point of the long exposure time and the short exposure time in the related art, thereby improving the quality of the generated driving record image.

[0056] After understanding the actual application scenarios of the processing device of the present application, the generation method proposed in the embodiment of the present application is specifically described through the following embodiments.

[0057] In the embodiment of the present application, the description will be made from the perspective of a device for generating a driving record image, which can be integrated into a related device, such as a driving recorder. Figure 1 As shown, Figure 1 is an optional flowchart of the method for generating a driving record image provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps 101 to 104. When the driving record image generation device executes the driving record image generation method, the specific process is as follows. It should be noted that the present embodiment Figure 1 The order of step 101 to step 104 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0058] Step 101, obtaining a target long exposure image shot by a first camera module based on a preset long exposure time, and a target short exposure image shot by a second camera module based on a preset short exposure time, wherein the exposure starting points of the first camera module and the second camera module are synchronized.

[0059] The following is a detailed description of step 101.

[0060] The first camera module and the second camera module refer to independent units or systems that integrate multiple components to complete the image capture function. Usually, the first camera module and the second camera module include the following key components:

[0061] (1) Image sensor: It is the core component of the camera module and is used to convert the received optical signal into an electrical signal to generate a digital image. For example, the image sensor can be a complementary metal oxide semiconductor (CMOS) and a charge coupled device (CCD) or other types of image sensors, which are not limited in the embodiments of the present application;

[0062] (2) Lens: responsible for focusing light onto the image sensor. The quality, focal length, aperture and other parameters of the lens will directly affect the clarity and quality of the captured image.

[0063] (3) Image processing chip: processes the raw image data output by the image sensor, including denoising, color correction, automatic exposure adjustment, automatic white balance, etc., to improve the image quality and visual effect;

[0064] (4) Other auxiliary components: such as flash, autofocus system, anti-shake mechanism, etc. These components can further enhance the functions and performance of the camera module.

[0065] Furthermore, the first camera module and the second camera module include the same components, and the parameters set for each component are also the same. The first camera module is responsible for capturing a long exposure image at a long exposure time, and the second camera module is responsible for capturing a short exposure image at a short exposure time, wherein the capture start time of each long exposure image and the corresponding short exposure image are the same.

[0066] Furthermore, the first camera module and the second camera module may both be arranged inside the generating device; or, the generating device may not be provided with a camera module inside, and the generating device is connected to a third-party camera module through an interface arranged by the generating device itself to realize the shooting of long-exposure images and short-exposure images. The specific adaptive adjustment may be made according to the actual situation, and the embodiments of the present application do not limit it.

[0067] For example, Figure 2 As shown, Figure 2 This is an optional timeline diagram of the method for generating driving record images provided in an embodiment of the present application. Set time A to time D as a current time period, and two camera modules are exposed at the same time to shoot synchronously. One camera module is responsible for shooting a long exposure image at a long exposure time (time A to time C), and the other camera module is responsible for shooting a short exposure image at a short exposure time (time A to time B). That is, the exposure starting point of the first camera module and the second camera module is the same (both are time A), and the exposure time of each camera module can be adaptively adjusted according to the actual situation.

[0068] Furthermore, the modes of the two camera modules are set in advance so that the two camera modules can enter the synchronization mode, thereby controlling the two to shoot synchronously from the same shooting start time. The two camera modules can be controlled to enter the synchronization mode (sync mode) in the following ways. Of course, different ways can also be used according to actual conditions to achieve synchronous shooting of each camera module, and the embodiments of the present application do not limit this:

[0069] (1) External trigger synchronization: Use an external device (such as a laser, flash, or dedicated visual trigger) to trigger two camera modules to shoot at the same time. This synchronization method requires a device that can send a trigger signal to both cameras at the same time. When the trigger signal arrives, the two cameras start shooting almost at the same time, thus achieving synchronization of the shooting start time;

[0070] (2) Time synchronization protocol: Based on the Precision Time Protocol (PTP), one camera is set as the master clock and the other camera is set as the slave clock. Then, the master clock periodically sends time synchronization information to the slave clock, and the slave clock adjusts its own time according to the received information to keep it consistent with the master clock. When the time of the two cameras is completely synchronized, the same shooting delay or trigger condition can be set in the software to ensure that the two camera modules can start shooting and exposing images at the same time.

[0071] (3) Software synchronization: If the camera module does not support external triggering or PTP time synchronization, shooting synchronization can also be achieved through software. Specifically, a timestamp for image shooting is added to each camera module so that each camera module can shoot the exposure image synchronously based on the set timestamp.

[0072] (4) Hardware synchronization: Use wiring to connect the synchronization signal interfaces of the two camera modules. The two camera modules achieve synchronized shooting by sending and receiving synchronization signals.

[0073] Furthermore, the first camera module and the second camera module are in the same plane when installed, and are arranged as close and parallel as possible to reduce the influence of parallax. The embodiment of the present application uses two camera modules for simultaneous shooting to ensure that the target objects of the obtained long exposure image and short exposure image are basically consistent in spatial position, and there is no need to consume too much computing power resources for global registration later; and the spatial parallax influence of the registered image is basically negligible, and a good overlapping fusion effect can be achieved, and there will be no problems such as the movement dislocation of the target object (such as the license plate) and the inability to restore after fusion due to the excessive delay between the long and short frames. That is, the embodiment of the present application uses two camera modules for simultaneous exposure image shooting, and avoids the movement dislocation of the front and rear frames caused by the difference in the starting point of the long exposure time and the short exposure time in the related art by the "space for time" method; and because the parallax of the multi-camera module can be basically ignored, the generation device does not need to consume too much computing power resources in the subsequent image processing, and also ensures the generation efficiency of the target driving record image under the condition of limited computing power resources.

[0074] Step 102 , extracting long exposure image features and short exposure image features from the same target position of the long exposure image and the short exposure image, respectively, and measuring the long exposure brightness value corresponding to the long exposure image feature and the short exposure brightness value corresponding to the short exposure image.

[0075] The following is a detailed description of step 102.

[0076] Wherein, both the long exposure image and the short exposure image include multiple target pixel blocks, and the image features of the target pixel blocks at the same target position in different exposure images are obtained to obtain the long exposure image features and the short exposure image features. For example, the target long exposure image includes target positions a1, a2 and a3, and the target short exposure image includes target positions b1, b2 and b3, then a1 and b1 are called the same target position, the image features extracted from a1 are called long exposure image features, and the image features extracted from b1 are called short exposure image features.

[0077] It can be understood that different image features correspond to different brightness values. Based on this, the brightness value corresponding to the long exposure image feature is further determined as the long exposure brightness value, and the brightness value corresponding to the short exposure image feature is determined as the short exposure brightness value.

[0078] Furthermore, after obtaining the target long exposure image and the target short exposure image, each exposure image can be firstly grayed to convert the image into a grayscale image; then, a filter (such as a Gaussian filter, a median filter, etc.) is used to remove the grayscale image noise and improve the grayscale image quality; then, a feature detection algorithm is used to identify key points in the image (such as corner points, edges, etc.) and corresponding feature descriptors, which usually contain information about the local structure of the image; then, the detected features are converted into a format suitable for subsequent processing, thereby obtaining long exposure image features and short exposure image features.

[0079] Among them, the feature detection algorithm can be a scale-invariant feature transform algorithm (ScaleInvariantFeatureTransform, SIFT), the SIFT algorithm realizes image feature extraction by constructing a scale space, detecting scale space extreme points, accurately locating key points, assigning key point directions, and calculating key point descriptors; the accelerated robust features algorithm (SpeededUpRobustFeatures, SURF), the SURF algorithm realizes feature extraction by constructing a scale space, detecting key points, assigning key point directions, and using a descriptor calculation method based on Haar wavelet response. Compared with SIFT, SURF is optimized in terms of calculation speed and robustness of descriptors; or, the feature detection algorithm can also be an algorithm for fast feature point extraction and description (OrientedFAST and RotatedBRIEF, ORB), which can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0080] Among them, the target pixel block can be a pixel point or a collection of multiple pixel points. The size of the target pixel block can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0081] Furthermore, in order to better determine the exposure brightness value corresponding to the image feature (including the long exposure brightness value and the short exposure brightness value), the exposure image can be first converted into a grayscale image, and the grayscale value of each target pixel block of the grayscale image can be used as the exposure brightness value; or, the color image can be separated into three color channels of red (R), green (G), and blue (B), and different channel weights are assigned to each channel. Based on the channel weights and the color distribution of the image features on the three channels, the exposure brightness value is calculated. Of course, the method for calculating the exposure brightness value can also be selected according to the actual situation, and the embodiment of the present application does not limit this.

[0082] Step 103: if the long exposure brightness value does not match the preset pixel fusion interval, the short exposure image feature is determined to be the target image feature; if the short exposure brightness value does not match the pixel fusion interval, the long exposure image feature is determined to be the target image feature.

[0083] Step 104 : If both the long exposure brightness value and the short exposure brightness value are consistent with the pixel fusion interval, the long exposure image feature and the short exposure image feature are fused to obtain the target image feature.

[0084] Step 103 and step 104 are described in detail below.

[0085] Among them, the pixel fusion interval represents a range of brightness values, and this range is set to determine whether it is necessary to fuse the long exposure image features and short exposure image features corresponding to the target long exposure image and the target short exposure image when generating the final target driving record image, so as to obtain better target image features for each target position. It can be understood that the image brightness value whose value falls closer to the middle part of the pixel fusion interval indicates that the brightness is moderate, thereby ensuring that the detail information in the image is well preserved. The image brightness value whose value is greater than the pixel fusion interval, or the image brightness value whose value is close to the right end of the pixel fusion interval indicates that the brightness is too bright, while the image brightness value whose value is less than the pixel fusion interval, or the image brightness value whose value is close to the left end of the pixel fusion interval indicates that the brightness is too dark. It can be understood that the image brightness value that is too bright or too dark will cause the details in the acquired exposure image to be lost. Therefore, accurately determining the pixel fusion interval helps to improve the quality of the target driving record image finally generated.

[0086] Furthermore, since the exposure time is unchanged and the start time of exposure is the same, under the same lighting environment, the brightness value of the short exposure image is usually not very low, while the brightness value of the long exposure image is very high. In other words, at least one exposure brightness value (long exposure brightness value or short exposure brightness value) falls into the preset pixel fusion interval, and the following three situations will occur accordingly:

[0087] Condition <1> : Select the short exposure image feature as the target image feature: If the long exposure brightness value does not match the pixel fusion interval, it means that the current driving lighting conditions are sufficient, so the long exposure image is overexposed. At this time, the brightness value of the short exposure image is more appropriate, so the short exposure image feature can be selected as the target image feature;

[0088] Condition <2> : Select the long exposure image feature as the target image feature: If the short exposure brightness value does not match the pixel fusion interval, it means that the current driving lighting conditions are insufficient, so the short exposure image is underexposed. At this time, the brightness value of the long exposure image is more appropriate, so the long exposure image feature can be selected as the target image feature;

[0089] Condition <3> : Fusion of long exposure image features and short exposure image features to obtain target image features: If the short exposure brightness value and the long exposure brightness value are consistent with the pixel fusion interval, it means that the short exposure brightness value and the long exposure brightness value are at the two ends of the pixel fusion interval, that is, the long exposure image is somewhat overexposed and the short exposure image is somewhat underexposed. Therefore, it is necessary to fuse the long exposure image features and the short exposure image to obtain the target image features with appropriate brightness.

[0090] Step 105 , stitching together multiple target image features to obtain a target driving record image.

[0091] Step 105 is described in detail below.

[0092] Further, after determining the target image features corresponding to each target position, the generating device splices the obtained target image features to generate a target driving record image. It can be understood that in an environment where the lighting conditions are unstable, such as driving at night, the embodiment of the present application can avoid a large misalignment between the long exposure image and the short exposure image captured based on the first camera module and the second camera module exposed at the same time, thereby improving the quality of the generated target driving record image.

[0093] In some embodiments, after the target driving record image is obtained by splicing multiple target image features, the following steps 201 to 204 are also included:

[0094] Step 201, within the current time period, obtain a plurality of long exposure images shot by a first camera module based on a long exposure time, and a plurality of short exposure images shot by a second camera module based on a short exposure time, wherein the exposure starting points of the first camera module and the second camera module are synchronized.

[0095] The following is a detailed description of step 201.

[0096] Among them, the current time period refers to a specific, ongoing time period. The time range of the current time period can be set according to actual conditions. For example, the current time period can be five seconds, half a minute or one minute. The embodiment of the present application does not limit the specific value of the current time period.

[0097] Among them, the short exposure time is less than the long exposure time, and the long exposure time is less than the current time period; usually, a current time period includes multiple long exposure times, so that multiple long exposure images and corresponding multiple short exposure images are captured in a current time period, and then the pixel fusion interval corresponding to the next time period is dynamically determined based on the multiple long exposure images and the corresponding multiple short exposure images. Of course, the long exposure time and the short exposure time can be determined according to the actual situation. The embodiment of the present application does not limit this.

[0098] Step 202 : obtaining a plurality of image brightness values ​​corresponding to each long exposure image or each short exposure image by measurement, and obtaining a plurality of brightness difference values ​​representing the brightness difference between each adjacent image brightness value by calculation.

[0099] Step 202 is described in detail below.

[0100] In some embodiments, after obtaining multiple long-exposure images and multiple short-exposure images taken within the current time period, the image brightness values ​​corresponding to all the long-exposure images are measured, or the image brightness values ​​corresponding to all the short-exposure images are measured, so as to subsequently generate a final target driving record image based on different night driving conditions.

[0101] The image brightness value may be the average brightness value of each pixel of the long exposure image and the short exposure image, may be the brightness value of any pixel taken as the image brightness value, or may be the median brightness value of each pixel of the long exposure image and the short exposure image. Of course, the multiple image brightness values ​​corresponding to each long exposure image or each short exposure image may also be determined by other methods. It should be noted that the determination method of the image brightness value of each long exposure image or each short exposure image should be unified, and the specific determination method may also be adaptively adjusted according to the actual situation, and the embodiments of the present application do not limit this.

[0102] For example, assuming that the target vehicle is driving at night, the generating device obtains a long exposure image A based on the long exposure time when the driving ambient light is bright; and obtains a long exposure image B based on the long exposure time when the driving ambient light is dark; since the long exposure time and the short exposure time are unchanged, the brightness value of the long exposure image A is higher than the brightness value of the long exposure image B. In other words, it is unreasonable to use a fixed pixel fusion interval to fuse the long exposure image and the short exposure image. When the driving ambient light is bright, more relevant information of the short exposure image should be retained, and when the driving ambient light is dark, more relevant information of the long exposure image should be retained. Therefore, the embodiment of the present application needs to determine the corresponding image brightness value based on the exposure image obtained under the same exposure time, so that the pixel fusion interval can be dynamically determined based on these image brightness values ​​later.

[0103] Further, the difference between the brightness values ​​of two adjacent images is calculated to obtain the brightness difference value, and then the brightness difference values ​​corresponding to the brightness values ​​of all images are calculated using the same method. For example, a long exposure image A1 is captured at the first moment, a long exposure image A2 is captured at the second moment, and a long exposure image A3 is captured at the third moment, and the first moment, the second moment, and the third moment are adjacent; the image brightness value corresponding to the long exposure image A1 is the image brightness value a1, the image brightness value corresponding to the long exposure image A2 is the image brightness value a2, and the image brightness value corresponding to the long exposure image A3 is the image brightness value a3; multiple brightness difference values ​​can be calculated to be the brightness difference value B1 (image brightness value a2-image brightness value a1) and the brightness difference value B2 (image brightness value a3-image brightness value a2).

[0104] Step 203 : jointly determine the environment area information according to the multiple brightness difference values ​​and the multiple image brightness values, and dynamically determine the pixel fusion interval corresponding to the next time period according to the environment area information.

[0105] The following is a detailed description of step 203.

[0106] In some embodiments, after determining the image brightness value and brightness difference value corresponding to the long exposure image or the short exposure image, the environment area information of the target vehicle when it is currently driving is jointly determined based on multiple image brightness values ​​and multiple brightness difference values ​​in the current time period, wherein the environment area information is used to characterize the brightness situation around the target vehicle when it is currently driving. Then, based on the different environment area information corresponding to the target vehicle, the pixel fusion interval corresponding to the next time period can be dynamically adjusted.

[0107] In some embodiments, determining the environment area information based on multiple brightness difference values ​​and multiple image brightness values ​​includes the following steps 301 to 303:

[0108] Step 301: If the plurality of brightness difference values ​​all belong to a preset small difference interval, and the plurality of image brightness values ​​belong to a preset bright value interval, it is determined that the environmental area information is always bright area information.

[0109] Step 302: If the plurality of brightness difference values ​​all belong to a preset small difference interval, and the plurality of image brightness values ​​belong to a preset dim value interval, it is determined that the environmental area information is normally dark area information.

[0110] Step 303: If the plurality of brightness difference values ​​all belong to the preset large difference interval, it is determined that the environmental area information is stroboscopic area information.

[0111] Steps 301 to 303 are described in detail below.

[0112] In some embodiments, the lighting conditions at night are very complex, so the target vehicle often encounters the following three situations when driving:

[0113] Lighting condition ①: Always on. The target vehicle is traveling on a road with sufficient and reasonably distributed street lights. In this case, the brightness values ​​of the long-exposure image and the short-exposure image captured by the generating device are brighter overall.

[0114] Lighting condition ②: Often dark. The target vehicle is traveling on a road with reasonable distribution of street lights but insufficient lighting, or on a road with no street lights and only its own headlights for road lighting. In this case, the brightness values ​​of the long-exposure image and the short-exposure image captured by the generating device are generally dark.

[0115] Lighting condition ③: sometimes bright and sometimes dark. The target vehicle is driving on a road with street lights but with unreasonable distribution of street lights, such as a tunnel. In this case, there will be obvious dark and bright areas on the road surface. In the bright area, the exposure image obtained by the generating device is brighter overall, while in the dark area, the exposure image obtained by the generating device is darker overall.

[0116] Based on this, the embodiment of the present application determines different environmental area information corresponding to different driving conditions:

[0117] Furthermore, after determining multiple image brightness values, the difference intervals to which the multiple brightness difference values ​​belong are determined, and the difference intervals are further divided into small difference intervals and large difference intervals; wherein the small difference intervals are used to characterize intervals with smaller differences in image brightness values, and the large difference intervals are used to characterize intervals with larger differences in image brightness values.

[0118] Furthermore, the bright value interval is used to characterize that the exposure image corresponding to the acquired image brightness value is taken under relatively bright conditions at night, corresponding to the bright areas in case ① and case ③ of the three conditions of night driving mentioned above; and the dim value interval is used to characterize that the exposure image corresponding to the acquired image brightness value is taken under relatively dark conditions at night, corresponding to the dark areas in case ② and case ③ of the three conditions of night driving mentioned above. It should be noted that the specific division of the bright value interval and the dim value interval can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0119] For example, the small difference interval is set to [0,5), the large difference interval is set to [5, +∞), +∞ represents positive infinity; the bright value interval is [100, 255], and the dim value interval is [0, 100):

[0120] Scenario (1): Assume that the brightness values ​​of multiple images are 100 and 102, which belong to the bright value range; and the corresponding brightness difference is calculated to be 2, which belongs to the small difference range; this means that the target vehicle is traveling on a road with sufficient and reasonably distributed street lights (corresponding to lighting condition ①). The image brightness values ​​of the exposure images taken are relatively high, and the differences between the brightness values ​​of each image are not large. Therefore, it can be determined that the environmental area information is the always-bright area information.

[0121] Scenario (2): Assume that the brightness values ​​of multiple images are 90 and 92, respectively, which belong to the dim value range; and the corresponding brightness difference is calculated to be 2, which belongs to the small difference range; this means that at this time the target vehicle is traveling on a road with reasonable street light distribution but insufficient lighting, or there are no street lights and the road conditions can only be illuminated by its own headlights (corresponding to lighting condition ②). The image brightness values ​​of the exposure images taken are relatively low, and the differences between the brightness values ​​of each image are not large. Therefore, it can be determined that the environmental area information is the normally dark area information.

[0122] Scenario (3): Assume that the brightness values ​​of multiple images are 105 and 92 respectively, that is, some of the brightness values ​​of the images belong to the bright value range, and some belong to the dim value range; and the corresponding brightness difference is calculated to be 14, which belongs to the large difference range; this means that the target vehicle is traveling on a road with street lights but unreasonable distribution of street lights (corresponding to lighting condition ③). The image brightness values ​​of the exposure images taken are sometimes high and sometimes low, and the difference between the brightness values ​​of each image is large. Therefore, it can be determined that the environmental area information is the stroboscopic area information.

[0123] It should be noted that the specific division of the large difference interval and the small difference interval can be set according to actual conditions, and the embodiments of the present application do not limit this.

[0124] In some embodiments, dynamically determining the pixel fusion interval corresponding to the next time period according to the environmental area information includes the following steps 401 to 402:

[0125] Step 401: If the environmental area information is the always-on area information, determine that the pixel fusion interval corresponding to the next time period is the first pixel fusion interval.

[0126] Step 402 : If the environment area information is the normally dark area information, determine that the pixel fusion interval corresponding to the next time period is the second pixel fusion interval, wherein the first pixel fusion interval and the second pixel fusion interval are different.

[0127] Steps 401 to 402 are described in detail below.

[0128] In some embodiments, since the captured long-exposure image and short-exposure image monitor the same area, for darker areas, the long-exposure image can improve the visibility of dark details; and for brighter areas, the short-exposure image can avoid overexposure to retain bright details. That is, when driving at night, the long-exposure image or the short-exposure image may be able to fully and clearly display the details of the picture. In this case, the long-exposure image or the short-exposure image can be selected to generate the target driving record image; if the long-exposure image or the short-exposure image cannot display the details of the picture well, the long-exposure image and the short-exposure image are fused to generate the target driving record image; in this way, there is no need to fuse each frame of the long-exposure image and the short-exposure image, reducing the waste of computing resources.

[0129] Furthermore, since the exposure time remains unchanged, for lighting condition ①, the image brightness values ​​of both the long exposure image and the short exposure image will be higher, so the generating device expects the pixel fusion interval to be biased towards a lower value to ensure that the short exposure image that meets the brightness requirements can be selected; and for lighting condition ②, the image brightness values ​​of both the long exposure image and the short exposure image will be lower, so the generating device expects the pixel fusion interval to be biased towards a higher value to ensure that the long exposure image that meets the brightness requirements can be selected; and in the related art, the pixel fusion interval is usually fixed and unchanged. When the exposure time remains unchanged, it is easy to cause the image brightness values ​​of the long exposure image and the short exposure image to not fall within the preset pixel fusion interval, resulting in the need to frequently fuse the long exposure image and the short exposure image (for the sake of ease of description, they can also be collectively referred to as "exposure images") to generate the target driving record image, thereby causing a waste of computing resources.

[0130] Furthermore, if the environmental area information is the always-lit area information, it can be determined that the target vehicle's driving environment has sufficient lighting conditions in the current time period. Therefore, it is predicted that in the next time period adjacent to the current time period, the target vehicle's driving environment also has sufficient lighting conditions. Therefore, the pixel fusion interval corresponding to the next time period is determined to be the first pixel fusion interval.

[0131] Furthermore, if the environmental area information is the normally dark area information, it can be determined that the target vehicle's driving environment has insufficient lighting conditions in the current time period, and thus it is predicted that the target vehicle's driving environment also has insufficient lighting conditions in the next time period adjacent to the current time period, and thus the pixel fusion interval corresponding to the next time period is determined to be the second pixel fusion interval. Among them, the left end value of the first pixel fusion interval is less than the left end value of the second pixel fusion interval, and the right end value of the first pixel fusion interval is less than the right end value of the second pixel fusion interval.

[0132] It should be noted that the specific values ​​of the left end and the right end of the first pixel fusion interval and the second pixel fusion interval can be set according to actual conditions, and the implementation of this application does not impose any restrictions on this.

[0133] In some embodiments, dynamically determining the pixel fusion interval corresponding to the next time period according to the environmental area information further includes the following steps 501 to 502:

[0134] Step 501 : If the environmental area information is stroboscopic area information, the shooting interval is determined based on any two long exposure images or short exposure images taken at adjacent times.

[0135] Step 502, determining the pixel fusion interval corresponding to the first shooting moment as the third pixel fusion interval, and determining the pixel fusion interval corresponding to the second shooting moment as the fourth pixel fusion interval, wherein the difference between the first shooting moment and the second shooting moment is the shooting interval length, and the third pixel fusion interval and the fourth pixel fusion interval are different.

[0136] Steps 501 to 502 are described in detail below.

[0137] In some embodiments, the next time period includes a first shooting moment and a second shooting moment, and the current time period includes a third shooting moment and a fourth shooting moment. Usually, the time intervals between the first shooting moment, the second shooting moment, the third shooting moment and the fourth shooting moment are the same. Therefore, the environmental area information corresponding to the current time period can be used as a reference for the driving environment of the next time period, and the pixel fusion interval of the next time period is determined based on the environmental area information of the current time period, so as to process the exposure image obtained in the next time period, and then better generate the final target driving record image.

[0138] Furthermore, if the environmental area information is stroboscopic area information, it means that the target vehicle has staggered bright and dark areas in the current driving environment, and the exposure image taken in the bright area is overall brighter, and the exposure image taken in the dark area is overall darker. Furthermore, the shooting interval between two shooting moments can be determined from any two long-exposure images or short-exposure images taken by the generating device, and the pixel fusion intervals corresponding to the two shooting moments are different. Since the shooting interval of the next time period is the same as the shooting interval of the current time period, the pixel fusion interval of each shooting moment in the current time period and the pixel fusion interval of each shooting moment in the next time period can be synchronously set, so that the generating device can process the exposure image based on different pixel fusion intervals at different shooting moments.

[0139] Furthermore, the first shooting moment of the next time period corresponds to the third shooting moment of the current time period, which also indicates that the target vehicle is in the bright area; the second shooting moment of the next time period corresponds to the fourth shooting moment of the current time period, which also indicates that the target vehicle is in the dark area.

[0140] Further, if the environmental area information is stroboscopic area information, and the exposure image captured by the generating device indicates that the target vehicle is in a bright area at the third shooting moment of the current time period, the pixel fusion interval of the first shooting moment of the next time period can be set as the third pixel fusion interval; if the environmental area information is stroboscopic area information, and the exposure image captured by the generating device indicates that the target vehicle is in a dark area at the fourth shooting moment of the current time period, the pixel fusion interval of the second shooting moment of the next time period can be set as the fourth pixel fusion interval, and the interval ranges of the third pixel fusion interval and the fourth pixel fusion interval are different.

[0141] Furthermore, if the environmental area information is stroboscopic area information, it can be determined that the lighting conditions of the driving environment of the target vehicle in the current time period are unstable, and thus it is predicted that in the next time period adjacent to the current time period, the lighting conditions of the driving environment of the target vehicle are also unstable, so the pixel fusion interval corresponding to the next time period is determined to be an alternating setting of the third pixel fusion interval and the fourth pixel fusion interval. Among them, the left end value of the third pixel fusion interval is less than the left end value of the fourth pixel fusion interval, and the right end value of the third pixel fusion interval is less than the right end value of the fourth pixel fusion interval.

[0142] Furthermore, since there is usually light from the bright area on both sides of the dark area of ​​the stroboscopic area, the brightness of the dark area of ​​the stroboscopic area is usually higher than the brightness of the normally dark area, so the value range of the third pixel fusion interval is greater than the value range of the first pixel fusion interval. Of course, the specific value ranges of the third pixel fusion interval and the fourth pixel fusion interval can also be set according to actual conditions, and the implementation of this application does not limit this.

[0143] Step 204 , extracting multiple target image features from the target long exposure image and / or the target short exposure image captured in the next time period based on the pixel fusion interval, and splicing the multiple target image features to obtain the target driving record image.

[0144] Step 204 is described in detail below.

[0145] In some embodiments, after determining the pixel fusion interval corresponding to the next time period, the generating device can process the target long exposure image and the target exposure image acquired in the next time period based on the pixel fusion interval. Specifically, the generating device can determine based on the pixel fusion interval whether to retain the features extracted from the target long exposure image, or retain the features extracted from the target short exposure image, or to fuse the features extracted from both to obtain the target image features corresponding to the current pixel block; and then obtain the target driving record image by splicing multiple target image features.

[0146] It is understandable that in the case of unstable lighting conditions and limited computing resources, the embodiment of the present application can avoid unnecessary fusion of long-exposure images and short-exposure images by dynamically adjusting the pixel fusion interval, and can still ensure the output of high-quality target driving record images while saving computing resources to the greatest extent. In this way, it can ensure that a clear target driving record image can be obtained even when the resources of the target vehicle are about to be exhausted, which has extremely high practical application value.

[0147] In some embodiments, the target long exposure image and the target short exposure image are obtained by the following steps:

[0148] In the next time period, a target long exposure image shot by the first camera module based on a preset long exposure time and a target short exposure image shot by the second camera module based on a preset short exposure time are obtained, wherein the exposure starting points of the first camera module and the second camera module are synchronized.

[0149] The method for acquiring the target exposure image in the next time period is the same as the method for acquiring the exposure image in the current time period, and is also obtained by shooting with the first camera module and the second camera module at the same starting time, and the setting of the camera module here is the same as step 101, which will not be repeated here. The target exposure image includes a target long exposure image and a target short exposure image.

[0150] In some embodiments, extracting multiple target image features from the target long exposure image and / or the target short exposure image captured in the next time period based on the pixel fusion interval includes the following steps 601 to 605:

[0151] Step 601, extracting long exposure image features and short exposure image features from the same target position of the target long exposure image and the target short exposure image, respectively, and measuring the long exposure brightness value corresponding to the long exposure image features and the short exposure brightness value corresponding to the short exposure image.

[0152] Step 602: if the long exposure brightness value does not match the preset pixel fusion interval, determine the short exposure image feature as the target image feature; if the short exposure brightness value does not match the pixel fusion interval, determine the long exposure image feature as the target image feature.

[0153] Step 603: If both the long exposure brightness value and the short exposure brightness value are consistent with the pixel fusion interval, the long exposure image feature and the short exposure image feature are fused to obtain the target image feature.

[0154] Step 604: obtaining a plurality of target image features based on the target image features at all target positions.

[0155] Steps 601 to 604 are described in detail below.

[0156] Wherein, the long exposure image and the short exposure image both include a plurality of target pixel blocks, and the image features of the same target pixel block in different exposure images are obtained to obtain the long exposure image features and the short exposure image features. It can be understood that different image features correspond to different brightness values, based on which, the brightness value corresponding to the long exposure image feature is further determined to be the long exposure brightness value, and the brightness value corresponding to the short exposure image feature is determined to be the short exposure brightness value.

[0157] Furthermore, after obtaining multiple long exposure brightness values ​​and short exposure brightness values, the long exposure brightness values ​​and the short exposure brightness values ​​are compared with the pixel fusion interval dynamically determined by the current time period to determine how to obtain the target image features based on the long exposure image features and the short exposure image features; the method for determining the target image features is similar to steps 103 to 104 and will not be repeated here.

[0158] In some embodiments, the long exposure image features and the short exposure image features are fused to obtain the target image features, including the following steps 701 to 703:

[0159] Step 701: Based on a preset first weight, the long exposure image feature is updated to obtain a target long exposure image feature.

[0160] Step 702: Based on a preset second weight, the short exposure image feature is updated to obtain a target short exposure image feature.

[0161] Step 703: stitching the long exposure image features and the short exposure image features to obtain the target image features.

[0162] Steps 701 to 703 are described in detail below.

[0163] Among them, since the long exposure image is good at capturing dark details, while the short exposure image is better at presenting bright details, by setting the first weight and the second weight respectively, the image features are extracted based on different proportions, so that the image of the target image features obtained by the final fusion retains the dark details without losing the bright details.

[0164] It can be understood that by setting different weights for the long exposure image features and the short exposure image features respectively, the brightness of the target image features obtained can be emphasized or suppressed to different degrees, so that the brightness distribution of the target image features is more uniform. This helps to reduce or avoid the problem of overexposure or underexposure of the target image features obtained due to simple fusion.

[0165] It should be noted that the specific values ​​of the first weight and the second weight can be adaptively adjusted according to actual conditions, and the embodiments of the present application do not limit this.

[0166] like Figure 3 As shown, Figure 3 : is an optional functional module diagram of a device for generating a driving record image provided in an embodiment of the present application, wherein the device for generating a driving record image may include:

[0167] The acquisition module 801 is used to acquire a target long exposure image shot by a first camera module based on a preset long exposure time, and a target short exposure image shot by a second camera module based on a preset short exposure time, wherein the exposure starting points of the first camera module and the second camera module are synchronized.

[0168] The processing module 802 is used to extract the long exposure image features and the short exposure image features from the same target position of the target long exposure image and the target short exposure image, respectively, and measure the long exposure brightness value corresponding to the long exposure image features and the short exposure brightness value corresponding to the short exposure image.

[0169] The first determination module 803 is used to determine that the short exposure image feature is the target image feature if the long exposure brightness value does not match the preset pixel fusion interval, and to determine that the long exposure image feature is the target image feature if the short exposure brightness value does not match the pixel fusion interval.

[0170] The second determination module 804 is used to fuse the long exposure image features and the short exposure image features to obtain the target image features if both the long exposure brightness value and the short exposure brightness value are consistent with the pixel fusion interval.

[0171] The target generation module 805 is used to combine multiple target image features to obtain a target driving record image.

[0172] The first camera module and the second camera module refer to independent units or systems that integrate multiple components to complete the image capture function. The first camera module and the second camera module include the same components, and the parameters set for each component are also the same. The first camera module is responsible for capturing a long exposure image at a long exposure time, while the second camera module is responsible for capturing a short exposure image at a short exposure time, wherein the capture start time of each long exposure image and the corresponding short exposure image are the same.

[0173] Furthermore, the first camera module and the second camera module may both be arranged inside the generating device; or, the generating device may not be provided with a camera module inside, and the generating device is connected to a third-party camera module through an interface arranged by the generating device itself to realize the shooting of long-exposure images and short-exposure images. The specific adaptive adjustment may be made according to the actual situation, and the embodiments of the present application do not limit it.

[0174] For example, Figure 2 As shown, Figure 2 This is an optional timeline diagram of the method for generating driving record images provided in an embodiment of the present application. Set time A to time D as a current time period, and two camera modules are exposed at the same time to shoot synchronously. One camera module is responsible for shooting a long exposure image at a long exposure time (time A to time C), and the other camera module is responsible for shooting a short exposure image at a short exposure time (time A to time B). That is, the exposure starting point of the first camera module and the second camera module is the same (both are time A), and the exposure time of each camera module can be adaptively adjusted according to the actual situation.

[0175] Furthermore, the modes of the two camera modules are set in advance so that the two camera modules can enter the synchronization mode, thereby controlling the two to shoot synchronously from the same shooting start time. The two camera modules can be controlled to enter the synchronization mode (sync mode) in the following ways. Of course, different ways can also be used according to actual conditions to achieve synchronous shooting of each camera module, and the embodiments of the present application do not limit this:

[0176] (1) External trigger synchronization: Use an external device (such as a laser, flash, or dedicated visual trigger) to trigger two camera modules to shoot at the same time. This synchronization method requires a device that can send a trigger signal to both cameras at the same time. When the trigger signal arrives, the two cameras start shooting almost at the same time, thus achieving synchronization of the shooting start time;

[0177] (2) Time synchronization protocol: Based on the Precision Time Protocol (PTP), one camera is set as the master clock and the other camera is set as the slave clock. Then, the master clock periodically sends time synchronization information to the slave clock. The slave clock adjusts its own time according to the received information to keep it consistent with the master clock. When the time of the two cameras is completely synchronized, the same shooting delay or trigger condition can be set in the software to ensure that the two camera modules can start shooting and exposing images at the same time.

[0178] (3) Software synchronization: If the camera module does not support external triggering or PTP time synchronization, shooting synchronization can also be achieved through software. Specifically, a timestamp for image shooting is added to each camera module so that each camera module can shoot the exposure image synchronously based on the set timestamp.

[0179] (4) Hardware synchronization: Use wiring to connect the synchronization signal interfaces of the two camera modules. The two camera modules achieve synchronized shooting by sending and receiving synchronization signals.

[0180] Furthermore, the first camera module and the second camera module are in the same plane when installed, and are arranged as close and parallel as possible to reduce the influence of parallax. The embodiment of the present application uses two camera modules for simultaneous shooting to ensure that the target objects of the obtained long exposure image and short exposure image are basically consistent in spatial position, and there is no need to consume too much computing power resources for global registration later; and the spatial parallax influence of the registered image is basically negligible, and a good overlapping fusion effect can be achieved, and there will be no problems such as the movement dislocation of the target object (such as the license plate) and the inability to restore after fusion due to the excessive delay between the long and short frames. That is, the embodiment of the present application uses two camera modules for simultaneous exposure image shooting, and avoids the movement dislocation of the front and rear frames caused by the difference in the starting point of the long exposure time and the short exposure time in the related art by the "space for time" method; and because the parallax of the multi-camera module can be basically ignored, the generation device does not need to consume too much computing power resources in the subsequent image processing, and also ensures the generation efficiency of the target driving record image under the condition of limited computing power resources.

[0181] Wherein, both the long exposure image and the short exposure image include multiple target pixel blocks, and the image features of the target pixel blocks at the same target position in different exposure images are obtained to obtain the long exposure image features and the short exposure image features. For example, the target long exposure image includes target positions a1, a2 and a3, and the target short exposure image includes target positions b1, b2 and b3, then a1 and b1 are called the same target position, the image features extracted from a1 are called long exposure image features, and the image features extracted from b1 are called short exposure image features.

[0182] It can be understood that different image features correspond to different brightness values. Based on this, the brightness value corresponding to the long exposure image feature is further determined as the long exposure brightness value, and the brightness value corresponding to the short exposure image feature is determined as the short exposure brightness value.

[0183] Furthermore, after obtaining the target long exposure image and the target short exposure image, each exposure image can be firstly grayed to convert the image into a grayscale image; then, a filter (such as a Gaussian filter, a median filter, etc.) is used to remove the grayscale image noise and improve the grayscale image quality; then, a feature detection algorithm is used to identify key points in the image (such as corner points, edges, etc.) and corresponding feature descriptors, which usually contain information about the local structure of the image; then, the detected features are converted into a format suitable for subsequent processing, thereby obtaining long exposure image features and short exposure image features.

[0184] Among them, the feature detection algorithm can be a SI FT algorithm, which realizes image feature extraction by constructing a scale space, detecting scale space extreme points, accurately locating key points, assigning key point directions, and calculating key point descriptors; a SURF algorithm, which realizes feature extraction by constructing a scale space, detecting key points, assigning key point directions, and using a descriptor calculation method based on Haar wavelet response. Compared with SI FT, SURF is optimized in terms of calculation speed and robustness of descriptors; or, the feature detection algorithm can also be an ORB algorithm, which can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0185] Among them, the target pixel block can be a pixel point or a collection of multiple pixel points. The size of the target pixel block can be set according to actual conditions, and the embodiment of the present application does not limit this.

[0186] Furthermore, in order to better determine the exposure brightness value corresponding to the image feature (including the long exposure brightness value and the short exposure brightness value), the exposure image can be first converted into a grayscale image, and the grayscale value of each target pixel block of the grayscale image can be used as the exposure brightness value; or, the color image can be separated into three color channels of red (R), green (G), and blue (B), and different channel weights are assigned to each channel. Based on the channel weights and the color distribution of the image features on the three channels, the exposure brightness value is calculated. Of course, the method for calculating the exposure brightness value can also be selected according to the actual situation, and the embodiment of the present application does not limit this.

[0187] Among them, the pixel fusion interval represents a range of brightness values, and this range is set to determine whether it is necessary to fuse the long exposure image features and short exposure image features corresponding to the target long exposure image and the target short exposure image when generating the final target driving record image, so as to obtain better target image features for each target position. It can be understood that the image brightness value whose value falls closer to the middle part of the pixel fusion interval indicates that the brightness is moderate, thereby ensuring that the detail information in the image is well preserved. The image brightness value whose value is greater than the pixel fusion interval, or the image brightness value whose value is close to the right end of the pixel fusion interval indicates that the brightness is too bright, while the image brightness value whose value is less than the pixel fusion interval, or the image brightness value whose value is close to the left end of the pixel fusion interval indicates that the brightness is too dark. It can be understood that the image brightness value that is too bright or too dark will cause the details in the acquired exposure image to be lost. Therefore, accurately determining the pixel fusion interval helps to improve the quality of the target driving record image finally generated.

[0188] Furthermore, since the exposure time is unchanged and the start time of exposure is the same, under the same lighting environment, the brightness value of the short exposure image is usually not very low, while the brightness value of the long exposure image is very high. In other words, at least one exposure brightness value (long exposure brightness value or short exposure brightness value) falls into the preset pixel fusion interval, and the following three situations will occur accordingly:

[0189] Condition <1> : Select the short exposure image feature as the target image feature: If the long exposure brightness value does not match the pixel fusion interval, it means that the current driving lighting conditions are sufficient, so the long exposure image is overexposed. At this time, the brightness value of the short exposure image is more appropriate, so the short exposure image feature can be selected as the target image feature;

[0190] Condition <2> : Select the long exposure image feature as the target image feature: If the short exposure brightness value does not match the pixel fusion interval, it means that the current driving lighting conditions are insufficient, so the short exposure image is underexposed. At this time, the brightness value of the long exposure image is more appropriate, so the long exposure image feature can be selected as the target image feature;

[0191] Condition <3> : Fusion of long exposure image features and short exposure image features to obtain target image features: If the short exposure brightness value and the long exposure brightness value are consistent with the pixel fusion interval, it means that the short exposure brightness value and the long exposure brightness value are at the two ends of the pixel fusion interval, that is, the long exposure image is somewhat overexposed and the short exposure image is somewhat underexposed. Therefore, it is necessary to fuse the long exposure image features and the short exposure image to obtain the target image features with appropriate brightness.

[0192] Further, after determining the target image features corresponding to each target position, the generating device splices the obtained target image features to generate a target driving record image. It can be understood that in an environment where the lighting conditions are unstable, such as driving at night, the embodiment of the present application can avoid a large misalignment between the long exposure image and the short exposure image captured based on the first camera module and the second camera module exposed at the same time, thereby improving the quality of the generated target driving record image.

[0193] The specific implementation of the driving record image generation device is substantially the same as the specific implementation of the driving record image generation method described above, and will not be described in detail herein.

[0194] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned method for generating a driving record image when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0195] like Figure 4 As shown, Figure 4 : is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application, the electronic device includes:

[0196] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0197] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the method for generating a driving record image in the embodiment of this application;

[0198] Input / output interface 903, used to implement information input and output;

[0199] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0200] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0201] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0202] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for generating the driving record image is implemented.

[0203] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0204] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0205] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0206] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0207] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0208] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0209] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0210] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0211] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0212] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0213] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0214] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for generating a driving record image, characterized in that: The method comprises: Acquire a target long exposure image obtained by shooting with a first camera module based on a preset long exposure time, and a target short exposure image obtained by shooting with a second camera module based on a preset short exposure time, wherein the exposure starting points of the first camera module and the second camera module are synchronized; Extracting a long exposure image feature and a short exposure image feature from the same target position of the target long exposure image and the target short exposure image, respectively, and measuring a long exposure brightness value corresponding to the long exposure image feature and a short exposure brightness value corresponding to the short exposure image; If the long exposure brightness value does not match the preset pixel fusion interval, the short exposure image feature is determined to be the target image feature; if the short exposure brightness value does not match the pixel fusion interval, the long exposure image feature is determined to be the target image feature; If both the long exposure brightness value and the short exposure brightness value are consistent with the pixel fusion interval, the long exposure image feature and the short exposure image feature are fused to obtain a target image feature; A plurality of the target image features are spliced ​​together to obtain a target driving record image.

2. The method according to claim 1, characterized in that: After the target driving record image is obtained by splicing the plurality of target image features, the method further includes: In a current time period, a plurality of long-exposure images captured by a first camera module based on the long exposure time and a plurality of short-exposure images captured by a second camera module based on the short exposure time are acquired, wherein the exposure starting points of the first camera module and the second camera module are synchronized; Measuring and obtaining a plurality of image brightness values ​​corresponding to each of the long exposure images or each of the short exposure images, and calculating and obtaining a plurality of brightness difference values ​​representing the brightness difference between each of the adjacent image brightness values; jointly determining environmental area information according to the plurality of brightness difference values ​​and the plurality of image brightness values, and dynamically determining the pixel fusion interval corresponding to the next time period according to the environmental area information; A plurality of target image features are extracted from the target long exposure image and / or the target short exposure image captured in the next time period based on the pixel fusion interval, and the target driving record image is obtained by splicing the plurality of target image features.

3. The method according to claim 2, characterized in that The jointly determining the environmental area information according to the plurality of brightness difference values ​​and the plurality of image brightness values ​​comprises: If the plurality of brightness difference values ​​all belong to a preset small difference interval, and the plurality of image brightness values ​​belong to a preset bright value interval, determining that the environmental area information is always bright area information; If the plurality of brightness difference values ​​all belong to a preset small difference interval, and the plurality of image brightness values ​​belong to a preset dim value interval, it is determined that the environmental area information is normally dark area information; If the plurality of brightness difference values ​​all belong to a preset large difference interval, it is determined that the environmental area information is stroboscopic area information.

4. The method according to claim 3, characterized in that The dynamically determining the pixel fusion interval corresponding to the next time period according to the environmental area information includes: If the environmental area information is the always-on area information, determining that the pixel fusion interval corresponding to the next time period is the first pixel fusion interval; If the environmental area information is the normally dark area information, it is determined that the pixel fusion interval corresponding to the next time period is a second pixel fusion interval, wherein the first pixel fusion interval is different from the second pixel fusion interval.

5. The method according to claim 4, characterized in that The next time period includes a first shooting moment and a second shooting moment; The step of dynamically determining the pixel fusion interval corresponding to the next time period according to the environmental area information further includes: If the environmental area information is the stroboscopic area information, determining the shooting interval length based on any two of the long-exposure images or the short-exposure images taken at adjacent times; Determine the pixel fusion interval corresponding to the first shooting moment as the third pixel fusion interval, and determine the pixel fusion interval corresponding to the second shooting moment as the fourth pixel fusion interval, wherein the difference between the first shooting moment and the second shooting moment is the shooting interval length, and the third pixel fusion interval is different from the fourth pixel fusion interval.

6. The method according to claim 5, characterized in that The extracting a plurality of target image features from the target long exposure image and / or the target short exposure image captured in the next time period based on the pixel fusion interval includes: Extracting a long exposure image feature and a short exposure image feature from the same target position of the target long exposure image and the target short exposure image, respectively, and measuring a long exposure brightness value corresponding to the long exposure image feature and a short exposure brightness value corresponding to the short exposure image; If the long exposure brightness value does not match the preset pixel fusion interval, the short exposure image feature is determined to be the target image feature; if the short exposure brightness value does not match the pixel fusion interval, the long exposure image feature is determined to be the target image feature; If both the long exposure brightness value and the short exposure brightness value are consistent with the pixel fusion interval, the long exposure image feature and the short exposure image feature are fused to obtain a target image feature; Based on the target image features of all the target positions, a plurality of target image features are obtained.

7. The method according to claim 6, characterized in that The fusing the long exposure image features and the short exposure image features to obtain target image features includes: Based on a preset first weight, updating the long exposure image feature to obtain a target long exposure image feature; Based on a preset second weight, updating the short exposure image feature to obtain a target short exposure image feature; The long exposure image features and the short exposure image features are concatenated to obtain target image features.

8. A device for generating a driving record image, characterized in that: The device comprises: An acquisition module, used to acquire a target long exposure image obtained by shooting with a first camera module based on a preset long exposure time, and a target short exposure image obtained by shooting with a second camera module based on a preset short exposure time, wherein the exposure starting points of the first camera module and the second camera module are synchronized; a processing module, configured to extract a long exposure image feature and a short exposure image feature from the same target position of the target long exposure image and the target short exposure image, respectively, and to measure a long exposure brightness value corresponding to the long exposure image feature and a short exposure brightness value corresponding to the short exposure image; A first determination module, configured to determine that the short exposure image feature is a target image feature if the long exposure brightness value does not match a preset pixel fusion interval, and to determine that the long exposure image feature is a target image feature if the short exposure brightness value does not match the pixel fusion interval; A second determination module is configured to fuse the long exposure image feature and the short exposure image feature to obtain a target image feature if both the long exposure brightness value and the short exposure brightness value are consistent with the pixel fusion interval; The target generation module is used to splice multiple target image features to obtain a target driving record image.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method for generating a driving record image according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for generating a driving record image according to any one of claims 1 to 7 is implemented.

Images