Method and device for avoiding shooting loss of traffic lights

Abstract

The invention aims to provide a method and equipment for avoiding shooting loss of a traffic signal lamp. The method comprises the following steps: capturing a video image of the traffic signal lamp through a shooting sensor; frame data output by a shooting sensor in a high dynamic range mode is obtained, and the frame data comprises long exposure area data and short exposure area data; and fusing the long exposure area data and the short exposure area data according to the idle time of the shooting sensor, and outputting a shot image according to the fused data. Therefore, the problem that traffic lights are not turned on in daytime is avoided, and traffic safety is guaranteed.

Description

Technical Field

[0001] The present application relates to the field of traffic lights, and in particular to a method and device for avoiding the loss of traffic light photography. Background Art

[0002] In actual daytime scenes, traffic lights will turn on and off. The current software measures to prevent LED traffic lights from not lighting up mainly rely on the frame rate. For example, in a 60hz area, if a 30 frame rate (fps) is selected, the traffic lights will not light up easily during the day. If a 50hz area selects 25fps, the traffic lights will not light up easily during the day. Therefore, if some areas have both 50hz and 60hz AC power, some 27 or 28fps categories will appear, which means that they do not fall on the 50hz frequency or the 60hz frequency. Let the traffic lights flash continuously during the day, which will cause the lights to flash at night. For sensors with single-frame output, the frame rate can only be modified to capture traffic lights.

[0003] In order to improve the shooting quality of the sensor, some sensors integrate the high dynamic range (HDR) function, so that the output video image can present a larger dynamic range, but it cannot solve the problem of traffic lights not flashing when the exposure time is insufficient in broad daylight. Summary of the invention

[0004] One purpose of the present application is to provide a method and device for avoiding the loss of traffic light photography, thereby solving the problem in the prior art.

[0005] According to one aspect of the present application, a method for avoiding traffic light shooting loss is provided, the method comprising:

[0006] Capture video images of traffic lights by shooting sensors;

[0007] Acquire frame data output by the shooting sensor in a high dynamic range mode, wherein the frame data includes long exposure area data and short exposure area data;

[0008] The long exposure area data and the short exposure area data are fused according to the idle time of the shooting sensor, and the shooting image is output according to the fused data.

[0009] Optionally, the long exposure area data and the short exposure area data are fused according to the idle time of the shooting sensor, including:

[0010] Determine the staggered time according to the idle time of the shooting sensor;

[0011] Separate the long exposure area data and the short exposure area data according to the staggered time;

[0012] The separated long exposure area data and short exposure area data are fused.

[0013] Optionally, the separated long exposure area data and short exposure area data are fused, including:

[0014] When the data of the separated short exposure area is less than the brightness threshold, it is determined that the short exposure is in the dark area; when it is greater than the brightness threshold, it is determined that the short exposure is in the bright area;

[0015] When the short exposure is in the dark area, the long exposure area data is used;

[0016] When the short exposure is in a bright area, the short exposure area data is used.

[0017] Optionally, the method comprises:

[0018] The built-in AE control is driven by the shooting sensor to determine whether to trigger the offset compensation mechanism. When the long exposure time is less than the staggered time, the offset compensation mechanism is triggered and compensation is performed according to the set offset time.

[0019] Optionally, compensation is performed according to the set offset time, including:

[0020] Calculate each line time of the exposure time of the shooting sensor, and calculate the exposure offset time according to each line time;

[0021] Determine a long exposure offset time and a short exposure offset time set by a user, wherein the short exposure offset time is obtained by adding the long exposure offset time and the exposure offset time;

[0022] The time offset of the frame data is removed according to the corresponding offset time.

[0023] Optionally, the method comprises:

[0024] Determine the time interval between each frame captured by the sensor;

[0025] The idle time of the capturing sensor is determined by the duration of the long exposure and the time interval between each frame.

[0026] Optionally, determining the staggered time according to the idle time of the shooting sensor includes:

[0027] A time position point is selected according to the idle time of the shooting sensor and the occupied time of the long exposure, and the staggered time is determined according to the time position point.

[0028] According to another aspect of the present application, a device for avoiding loss of traffic light photography is provided, the device comprising:

[0029] one or more processors; and

[0030] A memory storing computer readable instructions, which, when executed, cause the processor to perform the operations of the method as described above.

[0031] According to another aspect of the present application, a computer-readable medium is provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described above.

[0032] Compared with the prior art, the present application uses a shooting sensor to capture video images of traffic lights; obtains frame data output by the shooting sensor in high dynamic range mode, wherein the frame data includes long exposure area data and short exposure area data; fuses the long exposure area data and the short exposure area data according to the idle time of the shooting sensor, and outputs the shot image according to the fused data. This avoids the problem of traffic lights not lighting up during the day and ensures traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0034] Figure 1 A schematic flow chart of a method for avoiding loss of traffic light photography provided according to one aspect of the present application is shown;

[0035] Figure 2 A schematic diagram showing the exposure time and idle time of a shooting sensor in one embodiment of the present application is shown;

[0036] Figure 3 A schematic diagram of long and short exposure time nodes in an embodiment of the present application is shown;

[0037] Figure 4 An original image and a processed image of a traffic light in an embodiment of the present application are shown.

[0038] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0039] The present application is described in further detail below in conjunction with the accompanying drawings.

[0040] In a typical configuration of the present application, the terminal, the device of the service network and the trusted party all include one or more processors (eg, a central processing unit (CPU)), an input / output interface, a network interface and a memory.

[0041] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0042] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, Phase-Change RAM (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of random access memory (RAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. According to the definition in this article, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0043] Figure 1 A schematic flow chart of a method for avoiding loss of traffic light photography provided according to one aspect of the present application is shown, the method comprising: steps S11 to S13, wherein:

[0044] Step S11, capturing a video image of the traffic light by using a shooting sensor;

[0045] When capturing a traffic light, a shooting sensor is used to shoot it; here, a video image of the traffic light LED is captured, and a shooting sensor is used to shoot it to obtain a video image, and then the video image is subsequently analyzed, wherein the shooting sensor can be a vehicle-mounted driving recorder or other shooting device, which can capture the traffic light in front.

[0046] Step S12, obtaining frame data output by the shooting sensor in high dynamic range mode, wherein the frame data includes long exposure area data and short exposure area data; here, in the embodiment of the present application, the sensor uses high dynamic range HDR mode, and HDR imaging technology is a technology for capturing and presenting a wider range of brightness and color. This technology expands the dynamic range of the camera by combining multiple images with different exposures, so that the obtained high dynamic range HDR image can show the full range from the brightest details to the darkest details. The exposure area is the brightness of different areas of the subject, which is called the exposure area. These exposure areas are reflected in grayscale values ​​on the final captured image. The grayscale value can reflect the lighting conditions in the natural environment, that is, the subject and the background should be within a reasonable brightness range. The sensor outputs two frames of data in HDR mode, including short exposure area data and long exposure area data. The short exposure area data is also the data of a video frame, including short exposure images or short frame images, which are images taken by the sensor when the exposure time is less than the preset time; the long exposure area data includes long exposure images, also called long frame images, which are images taken by the sensor when the exposure time is greater than the preset time.

[0047] Step S13, according to the idle time of the shooting sensor, the long exposure area data and the short exposure area data are fused, and the shooting image is output according to the fused data. Here, the two frames of data are fused using the idle time of the shooting sensor to obtain a suitable shooting image. The vertical blanking (Vertical Blanking, Vblanking) of the shooting sensor is the idle time of the sensor after the exposure is completed, such as Figure 2As shown in the figure, the black vertical line is the exposure time, and the white part is the idle time of the sensor. The idle time of 10fps and 15fps is different. Therefore, the idle time is used to fuse the captured long exposure area data and short exposure area data. The fusion can use the HDR fusion algorithm, where the HDR fusion algorithm can be used to determine a group of images with different exposure values ​​through different exposure times, and then fuse the above group of images with different exposure values ​​through the fusion algorithm to determine an HDR image containing a complete brightness range, fuse the pixel values ​​of the original images taken with different exposures (for example, weighted summation), calculate the value index of each pixel of each image as the corresponding weight, and finally obtain HDR through weighted fusion. For example, the long exposure time is generally preset to T, the short exposure time is T / 16, and the data is all 8 bits, then the brightness range of the long exposure is (0-255), and the exposure range of the short exposure is (0-255)*16=(0-4096). When fusion is performed, for example, if the preset value is 128, short exposure is used at the beginning and completely after 192, then short exposure data is used linearly from 128, and there is some long exposure between 128-192, and all data after 192 brightness is short exposure. A new fused video image is obtained based on the fused data, that is, the image of the traffic light LED obtained by the vehicle-mounted equipment is output, so as to avoid the loss of LED light signals in broad daylight and correctly identify the traffic light, thereby avoiding traffic risks.

[0048] In some embodiments of the present application, in step S13, the staggered time is determined according to the idle time of the shooting sensor; the long exposure area data and the short exposure area data are separated according to the staggered time; and the separated long exposure area data and short exposure area data are merged. Here, taking the 50hz area as an example, when the exposure time is less than 10ms, it means that there is a chance that both the long and short exposures are in the trough state, then it is necessary to separate the short exposure and the long exposure so that they are staggered by 10ms. In this way, for devices such as traffic lights with a duty cycle of 50%, one of the long and short exposures must be in the bright area. Among them, the staggered 10ms is determined according to the idle time of the shooting sensor.

[0049] Continuing with the above embodiment, when the fusion is performed, when the data of the short exposure area after separation is less than the brightness threshold, it is determined that the short exposure is in the dark area, and when it is greater than the brightness threshold, it is determined that the short exposure is in the bright area; when the short exposure is in the dark area, the long exposure area data is used; when the short exposure is in the bright area, the short exposure area data is used. Here, when the short exposure is in the dark area, the long exposure area data will be used (at this time, the long exposure area data is the data of the long exposure in the bright area), and when the short exposure is in the bright area, the data of the short exposure area itself will be used, so that there will be light after the fusion of the long and short exposures, rather than a completely dark situation. Figure 3As shown in the figure, by staggering the long and short exposures, different flashes can be achieved while ensuring that the signal light is always on. The red vertical line is the long exposure time node, and the blue vertical line is the short exposure time node. The determination of the dark area and the bright area is determined by the brightness threshold. By comparing the long and short frames of data, when the brightness is less than the brightness threshold L (for example, 128), it means that it is in the dark area and the long exposure data is used. When the brightness is less than the brightness threshold, the short exposure data is used, so the signal light information collection in the whole cycle is completed. Figure 4 As shown, the upper left side and the lower left side are original images, and some LED lights are not on. However, through the method of the present application, they can be processed into new images at the upper right side and the lower right side, which can make the LED lamp beads always on without flickering.

[0050] In some embodiments of the present application, the built-in AE control is driven by the shooting sensor to determine whether the offset compensation mechanism is triggered. When the long exposure time is less than the offset time, the offset compensation mechanism is triggered, and compensation is performed according to the set offset time. Here, the AE control of the shooting sensor is Auto Exposure automatic exposure control, and its main function is to make the imaging screen have a suitable brightness. Through AE control, it is determined whether the exposure when capturing the video image triggers the offset compensation mechanism. Whether there is an offset is determined based on the length of the exposure time. When the long exposure time is short, vblanking is relatively large, and vblanking is roughly equal to the frame interval minus the long exposure time. Therefore, when the long exposure time is long, no compensation is required to capture the signal light information. At this time, no offset is required, so the offset needs to be slowly removed; and when the long exposure time is short, an offset is required, which triggers the offset compensation mechanism, and compensation is performed according to the offset time set by the user. For example, in a certain area, the national power grid frequency is 50 Hz, and one cycle is 20 ms. If the traffic light is on for 10 ms and off for 10 ms, and the sensor has 25 frames and a cycle of 40 ms, if the exposure time is less than 10 ms, for example, the exposure time is 1 ms, then the idle time is 39 ms, and offset compensation is required. According to the multi-frame HDR mode, one of the frames can be staggered to avoid the 10 ms when the traffic light is off, thereby avoiding the traffic light from being completely off.

[0051] Continuing with the above embodiment, the exposure time of each line of the shooting sensor is calculated, and the exposure offset time is calculated according to the exposure time of each line; the long exposure offset time and the short exposure offset time set by the user are determined, wherein the short exposure offset time is obtained by adding the long exposure offset time and the exposure offset time; and the frame data is time-offset removed according to the corresponding offset time.

[0052] Here, one scale of the sensor is a row, and a row is row_time (time per row), which is converted into time by calculating the row. Calculate the sensor exposure time row_time (time per row), and calculate the exposure offset line offset_line = offset_time / row_time. Set the long exposure offset line long_start_line, and set the short exposure offset line short_start_line = long_start_line + offset_line. The offset time offset_time is selected according to the user's anti-flash setting. For example, for 60hz areas, it is set to 1000ms / 60 / 2 (half a square wave cycle) = 8.33ms, and for 50hz areas, it is set to 1000ms / 50 / 2 = 10ms. Users select their country's industrial frequency through options, such as 50hz in domestic areas and 60hz in N system in foreign areas, and then set different offsets according to the frequencies of power grids in different countries.

[0053] In some embodiments of the present application, the time interval between each frame of the shooting sensor is determined; the idle time of the shooting sensor is determined according to the occupied time of the long exposure and the time interval between each frame. The time position point is selected according to the idle time of the shooting sensor and the occupied time of the long exposure, and the staggered time is determined according to the time position point.

[0054] The time interval between each frame of the shooting sensor is determined by the local industrial frequency. For example, taking a 25fps sensor as an example, the interval between each frame is 40ms. If the long exposure consumes 10ms, that is, the long exposure occupancy time is 10ms, then the sensor idle time vblanking is 40ms-10ms=30ms. Then the short exposure can be set to any one between 10ms and 40ms. Since the long exposure during the day is too short, such as only 1ms, there are more positions where the short exposure can be set. The staggered time with the long exposure is determined according to the selected position and time point; when the long exposure itself is 10ms, the short exposure does not need to be offset so much, because the long exposure itself can capture the short exposure signal light data.

[0055] The method described in this application utilizes the physical idle time vblanking of the sensor itself and the fusion algorithm, combined with user settings, to avoid the problem of traffic light shooting loss; it is not only applicable to dual-frame HDR, but also to multi-frame HDR. In addition, the flickering effect of changing the frequency can be achieved without losing the frame rate by changing the offset time of each frame.

[0056] In addition, an embodiment of the present application further provides a computer-readable medium having computer-readable instructions stored thereon, and the computer-readable instructions can be executed by a processor to implement the aforementioned method for avoiding the loss of traffic light photography.

[0057] Corresponding to the method described above, the present application also provides a terminal, which includes a terminal capable of executing the above Figure 1 or Figure 2 or Figure 3 or Figure 4 Or the modules or units of the method steps described in each embodiment, these modules or units can be implemented by hardware, software or a combination of hardware and software, and this application does not limit it. For example, in one embodiment of this application, a device for avoiding the loss of traffic light photography is also provided, and the device includes:

[0058] one or more processors; and

[0059] A memory storing computer readable instructions, which, when executed, cause the processor to perform the operations of the method as described above.

[0060] For example, the computer readable instructions, when executed, cause the one or more processors to:

[0061] Capture video images of traffic lights by shooting sensors;

[0062] Acquire frame data output by the shooting sensor in a high dynamic range mode, wherein the frame data includes long exposure area data and short exposure area data;

[0063] The long exposure area data and the short exposure area data are fused according to the idle time of the shooting sensor, and the shooting image is output according to the fused data.

[0064] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0065] It should be noted that the present application can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present application can be executed by a processor to implement the steps or functions described above. Similarly, the software program of the present application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0066] In addition, a part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. The program instruction for calling the method of the present application may be stored in a fixed or removable recording medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium, and / or stored in a working memory of a computer device that runs according to the program instruction. Here, according to an embodiment of the present application, a device is included, the device including a memory for storing computer program instructions and a processor for executing program instructions, wherein, when the computer program instruction is executed by the processor, the device is triggered to run the method and / or technical solution based on the aforementioned multiple embodiments according to the present application.

[0067] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic features of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. A method for avoiding the loss of traffic light photography, characterized in that: The method comprises: Capture video images of traffic lights by shooting sensors; Acquire frame data output by the shooting sensor in a high dynamic range mode, wherein the frame data includes long exposure area data and short exposure area data; The long exposure area data and the short exposure area data are fused according to the idle time of the shooting sensor, and the shooting image is output according to the fused data.

2. The method according to claim 1, characterized in that The long exposure area data and the short exposure area data are fused according to the idle time of the shooting sensor, including: Determine the staggered time according to the idle time of the shooting sensor; Separate the long exposure area data and the short exposure area data according to the staggered time; The separated long exposure area data and short exposure area data are fused.

3. The method according to claim 2, characterized in that The separated long exposure area data and short exposure area data are fused, including: When the data of the separated short exposure area is less than the brightness threshold, it is determined that the short exposure is in the dark area; when it is greater than the brightness threshold, it is determined that the short exposure is in the bright area; When the short exposure is in the dark area, the long exposure area data is used; When the short exposure is in a bright area, the short exposure area data is used.

4. The method according to claim 2, characterized in that: The method comprises: The built-in AE control is driven by the shooting sensor to determine whether to trigger the offset compensation mechanism. When the long exposure time is less than the staggered time, the offset compensation mechanism is triggered and compensation is performed according to the set offset time.

5. The method according to claim 4, characterized in that Compensation is performed according to the set offset time, including: Calculate each line time of the exposure time of the shooting sensor, and calculate the exposure offset time according to each line time; Determine a long exposure offset time and a short exposure offset time set by a user, wherein the short exposure offset time is obtained by adding the long exposure offset time and the exposure offset time; The time offset of the frame data is removed according to the corresponding offset time.

6. The method according to claim 2, characterized in that The method comprises: Determine the time interval between each frame captured by the sensor; The idle time of the capturing sensor is determined by the duration of the long exposure and the time interval between each frame.

7. The method according to claim 6, characterized in that The staggered time is determined according to the idle time of the shooting sensor, including: A time position point is selected according to the idle time of the shooting sensor and the occupied time of the long exposure, and the staggered time is determined according to the time position point.

8. A device for avoiding the loss of traffic light photography, characterized in that: The device comprises: one or more processors; and A memory storing computer readable instructions which, when executed, cause the processor to perform the operations of the method as claimed in any one of claims 1 to 7.

9. A computer-readable medium having computer-readable instructions stored thereon, wherein the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1 to 7.

Images