Information processing device, information processing method, and computer program

Abstract

To enable the type of a light source of a signal light unit to be autonomously determined.SOLUTION: According to an aspect of the disclosure, a device is an information processing device including an acquisition part for acquiring image data of a moving image generated by a road-side camera, and a control part for determining a current lamp color of a signal light unit including a plurality of signal lights from the image data and executing prescribed processing by using the determined current lamp color. The prescribed processing includes type determination for determining a type of the light source of the signal light unit on the basis of the existence / nonexistence of the periodicity of luminance fluctuation in pixels corresponding to the signal lights.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and a computer program.

Background Art

[0002] Patent Document 1 describes an in-vehicle camera that can prevent misidentifying an LED traffic signal as being turned off by acquiring the blinking cycle of the LED traffic signal through vehicle-to-roadside communication and setting the frame rate to synchronize with the acquired blinking cycle. Patent Document 2 describes a traffic signal discrimination apparatus that can surely discriminate traffic signals based on image processing of a plurality of prepared lighting patterns in advance.

[0003] Patent Document 3 describes a traffic signal recognition method that specifies the lit portion of a target traffic signal in a region of interest (ROI) as a lit region and recognizes the lit color of the target traffic signal based on the color of the specified lit region.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the road network of our country, LED traffic signals and incandescent traffic signals are currently mixed. Since the operations of LED traffic signals and incandescent traffic signals are different during lighting, it is preferable to identify both of them in advance in order to accurately determine the light color of the traffic signal. However, the aforementioned patent documents do not describe a method for determining whether a photographed traffic light is an LED traffic light or an incandescent light.

[0006] In view of the aforementioned conventional problems, this disclosure aims to provide an information processing device, etc., that can autonomously determine the type of light source of a traffic signal. [Means for solving the problem]

[0007] An information processing apparatus according to one aspect of the present disclosure comprises: an acquisition unit that acquires image data of a video generated by a roadside camera; and a control unit that determines the current light color of a traffic signaling device including a plurality of signal lights from the image data and performs a predetermined process using the determined current light color, wherein the predetermined process includes a type determination that determines the type of light source of the traffic signaling device based on whether or not there is periodicity in the brightness fluctuations in the pixels corresponding to the signal lights.

[0008] A computer program according to one aspect of the present disclosure is an information processing method comprising, as a step executed by an information processing device, a step of acquiring image data of a video generated by a roadside camera, and a step of determining the current light color of a traffic signal light including a plurality of signal lights from the image data, and performing a predetermined process using the determined current light color, wherein the predetermined process includes a type determination that determines the type of light source of the traffic signal light based on whether or not there is periodicity in the brightness fluctuations in the pixels corresponding to the signal lights.

[0009] A method according to one aspect of the present disclosure is a computer program that causes a computer to function as an information processing device comprising: an acquisition unit that acquires image data of a video generated by a roadside camera; and a control unit that determines the current light color of a signal light device including a plurality of signal lights from the image data and performs a predetermined process using the determined current light color, wherein the predetermined process includes a type determination that determines the type of light source of the signal light device based on whether or not there is periodicity in the brightness fluctuations in the pixels corresponding to the signal lights.

[0010] This disclosure can be implemented not only as a system and apparatus having the characteristic configuration described above, but also as a program for causing a computer to execute such characteristic configuration. Furthermore, this disclosure can be implemented as a semiconductor integrated circuit that implements part or all of the system and apparatus. [Effects of the Invention]

[0011] According to this disclosure, the type of light source of a traffic signal can be determined autonomously. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a road plan showing an example of the configuration of the information provision system. [Figure 2] Figure 2 is a block diagram showing an example of an edge computer hardware configuration. [Figure 3] Figure 3 is a block diagram showing an example of the functional configuration of an edge computer. [Figure 4] Figure 4 is an explanatory diagram showing an example of how to set the reference point for the traffic light area. [Figure 5] Figure 5 is a time chart showing the relationship between the brightness change of the first lamp and the exposure time. [Figure 6] Figure 6 is a time chart showing the relationship between the brightness change of the second light and the exposure time. [Figure 7] Figure 7 is an explanatory diagram showing variations in the color detection pattern of the third traffic light. [Figure 8] Figure 8 is a flowchart showing an example of type determination. [Figure 9] Figure 9 is a flowchart showing an example of timing adjustment. [Figure 10] Figure 10 is a flowchart showing an example of time detection. [Figure 11] Figure 11 is a flowchart showing an example of updating the timetable. [Modes for carrying out the invention]

[0013] <Overview of Embodiments of the Present Invention> The overview of the embodiments of the present invention will be listed and described below. (1) An apparatus according to an aspect of the present embodiment is an information processing apparatus including an acquisition unit that acquires image data of a video generated by a roadside camera, and a control unit that determines the current light color of a signal light device including a plurality of signal lights from the image data and executes a predetermined process using the determined current light color. The predetermined process includes a type determination that determines the type of the light source of the signal light device based on the presence or absence of periodicity in the luminance variation at pixels corresponding to the signal lights.

[0014] According to the information processing apparatus of the present embodiment, since the above-described predetermined process includes a type determination that determines the type of the light source of the signal light device based on the presence or absence of periodicity in the luminance variation at pixels corresponding to the signal lights, the type of the light source of the signal light device can be autonomously determined.

[0015] (2) In the information processing apparatus of the present embodiment, when the type determined by the type determination is an LED, the predetermined process may include a timing adjustment that synchronizes the exposure time of the roadside camera with the timing at which the luminance of the pixel corresponding to the signal light becomes a predetermined value or more. In this case, since the exposure time of the roadside camera is synchronized with the timing at which the luminance of the pixel corresponding to the signal light becomes a predetermined value or more, the determination accuracy of the current light color is improved as compared with the asynchronous case.

[0016] (3) In the information processing apparatus of the present embodiment, the timing adjustment may include a process of setting the generation cycle of the exposure time to an integer multiple of the cycle of the luminance variation of the LED that is lit by an AC power supply. The reason is that when the generation cycle of the exposure time is not an integer multiple of the cycle of the luminance variation of the LED, periodic fluctuations (flicker phenomenon) occur in the luminance of the pixel corresponding to the signal light, and the determination accuracy of the light color decreases.

[0017] (4) In the information processing device of this embodiment, if the type determined by the type determination is an incandescent bulb, the predetermined process may include time detection, where the detection time of a lower-than-normal brightness is set as the time of the signal lamp color change. In this case, the detection error for the time of the light color change can be made smaller than the reciprocal of the frame rate (for example, 33ms if the fps is 30).

[0018] (5) In the information processing device of this embodiment, the predetermined processing may include processing to update a time table that includes the transition order of the light colors for each inflow path and the duration of each light color, based on the detection time. In this case, the timetable is updated based on the detection time with the smallest error, thus improving the accuracy of the timetable compared to before the update.

[0019] (6) In the information processing device of this embodiment, the predetermined processing may include the process of generating signal information representing the operating state of the signal lamp based on the determined current lamp color. In this case, signal information useful for vehicle driving assistance can be generated.

[0020] (7) A method according to one aspect of this embodiment is an information processing method executed by the information processing device described in (1) to (6) above. Therefore, the information processing method of this embodiment has the same effects as the information processing devices described in (1) to (6) above.

[0021] (8) A computer program according to one aspect of this embodiment is a computer program that causes a computer to function as an information processing device as described in (1) to (6) above. Therefore, the computer program of this embodiment has the same effects as the information processing devices described in (1) to (6) above.

[0022] <Details of Embodiments of the Invention> The embodiments of the present invention will be described in detail below with reference to the drawings. At least some of the embodiments described below may be combined in any way.

[0023] [Overall System Configuration] Figure 1 is a road plan showing an example configuration of the information provision system 1 of this embodiment. Figure 2 is a block diagram showing an example hardware configuration of the edge computer 3. As shown in Figures 1 and 2, the information provision system 1 includes roadside cameras 2 installed at appropriate locations at intersection J, an edge computer (information processing device) 3, and terminal devices 4 and 5 capable of wireless communication with the edge computer 3.

[0024] A traffic signal 6 will be installed at intersection J. The traffic signal 6 will be installed on a support pole located on the side of the road at intersection J. The traffic signal 6 is a display device that includes multiple signal lights to indicate whether a moving object such as a vehicle 7 has the right of way or not, based on the color of the lights when illuminated. The traffic signal 6 illustrated in Figure 1 is a vehicle signal that includes at least a blue light, a yellow light, and a red light. The vehicle signal may also have arrow lights, such as a right-turn arrow light. Although not shown in Figure 1, pedestrian signals may also be included at intersection J.

[0025] Roadside camera 2 is installed on a support pole or similar structure near intersection J. Roadside camera 2 is a digital camera with an image sensor capable of recording video. Roadside camera 2 can be adjusted to enlarge and reduce its field of view, and is set up so that the traffic signal lights 6 fit within the field of view. Therefore, the image data output by the roadside camera 2 will be a landscape image that includes at least one traffic light 6 installed at intersection J.

[0026] The roadside camera 2 may be set up so that one traffic light 6 fits within the field of view of one roadside camera 2, or it may be set up so that two or more traffic light 6 fit within the field of view of one roadside camera 2. In Figures 1 and 2, two roadside cameras 2 are installed, but the number of roadside cameras 2 can be one or three or more.

[0027] The roadside camera 2 is a digital camera that can adjust the frame rate (in fps) and exposure time of video recording using an external control signal C1 (see Figure 3). The frame rate adjustment range is, for example, 25 to 60 fps. In this embodiment, the default frame rate of the roadside camera 3 is set to 30 fps.

[0028] The edge computer 3 is an information processing device that is connected to the roadside camera 2 in a communicative manner and is installed on a support pole or similar structure near the intersection J. The connection method between the roadside camera 2 and the edge computer 3 may be either wired or wireless. The edge computer 3 has a setting function for the roadside camera 2, a function to generate predetermined downlink information from video image data input from the roadside camera 2, and a function to distribute the generated downlink information to terminal devices 4 and 5.

[0029] Terminal device 4 is an in-vehicle terminal permanently or temporarily installed in vehicle 7. Vehicle 7 refers to all vehicles that travel on roads. Therefore, in addition to automobiles, light vehicles, and trolleybuses, motorcycles are also considered vehicles. The drive system of Vehicle 7 is not limited to internal combustion engines; electric vehicles and hybrid cars are also included. Vehicle 7 may be a regular vehicle driven by the occupants themselves, or it may be an autonomous vehicle of Level 3 or higher.

[0030] Terminal device 5 is a user terminal carried by pedestrian 8 (see Figure 2). The user terminal may be, for example, a smartphone, a tablet computer, or a notebook computer. Terminal devices 4 and 5 are capable of wireless communication with a wireless base station 9 (e.g., a mobile base station). The wireless base station 9 can communicate with the edge computer 3 via a public communication network 10, which includes the core network of mobile communications and the internet.

[0031] The edge computer 3 transmits a communication packet containing downlink information to terminal devices 4 and 5 via the public communication network 10. Therefore, the communication packet containing downlink information is distributed to terminal devices 4 and 5 via the wireless base station 9. The communication equipment for distributing communication packets may not rely on the public communication network 10, but may also be a DSRC (Dedicated Short Range Communication) communication device (not shown). The DSRC method is also called a narrow-range communication method.

[0032] The downlink information includes signal information S1 for intersection J. Signal information S1 is information representing the color state of the signal light 6. The signal information S1 of this embodiment is, for example, information that represents the current light color state as well as the light color state for a predetermined period in the near future (e.g., 2 cycles). Specifically, the signal information S1 of intersection J includes the following information 1 to 5.

[0033] Information 1: Location and name of intersection J to be provided. Information 2: Location and name of the access road to which the traffic signal 6 provided grants the right of way. Information 3: Types of signal lights currently illuminated (which signal lights are on) Information 4: The sequence of traffic light illumination from this point forward and the directions in which traffic is permitted when the lights are on. Information 5: The timing for turning each signal light on or off from this point in time (either absolute time or remaining seconds from the reference time)

[0034] The signal information S1 for intersection J provided to terminal devices 4 and 5 may be information that represents only the current color state of the signal lights 6. In this case, the signal information S1 includes the information 1 to 3 described above.

[0035] [Edge computer hardware configuration] As shown in Figure 2, the edge computer 3 comprises a control unit 31, a storage unit 32, a communication unit 33, a synchronization processing unit 34, and multiple types of databases 35, 36. Databases 35 and 36 are electronic data constructed in the storage unit 32 in a predetermined data arrangement. However, some or all of databases 35 and 36 may be constructed in an external storage device (not shown) connected to the edge computer 3.

[0036] The control unit 31 is an arithmetic processing unit that includes a CPU (Central Processing Unit) and RAM (Random Access Memory). The control unit 31 may also include an integrated circuit such as an FPGA (Field-Programmable Gate Array). The control unit 31 reads the computer program 38 stored in the memory unit 32 into the main memory (RAM) and performs various information processing according to the program 38. The information processing includes the process of generating signal information S1 from the image data of the roadside camera 2.

[0037] The storage unit 32 is an auxiliary storage device that includes non-volatile memory such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive). The storage unit 32 may include flash ROM (Read Only Memory), USB (Universal Serial Bus) memory, or an SD card.

[0038] The communication unit 33 is a communication module that performs public communication via the public communication network 10 and dedicated communication with the roadside camera 2. The communication unit 33 may be composed of multiple communication boards, each with a different communication standard. When the communication unit 33 receives image data from the roadside camera 2, it outputs the received image data to the control unit 31. Therefore, the communication unit 33 functions as an acquisition unit that takes in video image data from the roadside camera 2, which is the source of the information, into the edge computer 3.

[0039] The control unit 31 determines the color of the currently lit signal light 6 based on the input image data. The control unit 31 measures the duration of the determined color and creates a "timetable" for the traffic signals currently in use at intersection J from the measured duration. The timetable is a table-formatted data set that includes the cycle start time, the sequence of light color transitions for each inflow path within one cycle, and the duration of each light color. The control unit 31 creates the timetable based on the duration of each light color for a predetermined period (e.g., one hour) for multiple signal lights 6.

[0040] The control unit 31 generates signal information S1 based on the timetable. Specifically, the control unit 31 determines the aforementioned information 1 to 3 from the elapsed time from the cycle start time to the current time, and calculates the aforementioned information 4 and information 5 from the remaining seconds for each light color at the current time. Furthermore, if only the current color status of signal lamp 6 is to be transmitted, the control unit 31 will not calculate information 4 and information 5.

[0041] The control unit 31 outputs the generated signal information S1 to the communication unit 33. The communication unit 33 generates a communication packet destined for terminal devices 4 and 5, which includes the input signal information S1, and transmits the generated communication packet to the wireless base station 9. Therefore, the communication unit 33 functions as a distribution unit that distributes signal information S1 to external devices such as terminal devices 4 and 5. Note that the destination external device may be not only terminal devices 4 and 5, but also a server (for example, a server of an insurance company that conducts accident investigations).

[0042] The synchronization processing unit 34 is a processing unit that synchronizes the time with the roadside camera 2 and other communication nodes such as terminal devices 4 and 5 using a predetermined synchronization method. The synchronization method of the synchronization processing unit 34 may employ, for example, a synchronization method based on the output of a GNSS (Global Navigation Satellite System) receiver, or a synchronization method using communication frames such as NTP (Network Time Protocol) and PTP (Precision Time Protocol).

[0043] The multiple types of databases 35 and 36 include the member database (DB) 35 and the timetable database (DB) 36. The member database 35 records personal information of registered members (e.g., the owner of vehicle 7) who receive the signal information S1 service, as well as identification information of the registered member's communication terminal (e.g., MAC address). The timetable database 36 records the aforementioned timetable.

[0044] [Functional Configuration of Edge Computers] Figure 3 is a block diagram showing an example of the functional configuration of edge computer 3. As shown in Figure 3, the control unit 31 of the edge computer 3 includes a data input unit 41, an area setting unit 42, a lamp extraction unit 43, a lamp color determination unit 44, an information generation unit 45, a training unit 46, and a camera control unit 47. These functional units 41 to 47 are realized by the execution of a computer program 38 by the control unit 31.

[0045] The data input unit 41 receives image data received by the communication unit 33 from the roadside camera 2. The data input unit 41 outputs the received image data to the lamp extraction unit 43. The area setting unit 42 performs the process of setting a reference point for the area in which the signal light 6 exists (hereinafter referred to as the "light area") among all the pixels that make up the image data. This process is performed, for example, by the user's initial settings for the edge computer 3.

[0046] Figure 4 is an explanatory diagram showing an example of a method for setting the reference point in a traffic light area. As shown in Figure 4, any of the following methods 1 to 4 can be used to set the reference point. Method 1: Specify the center point P1 of the signal light 6 (center point of the yellow light). Method 2: Specify the center points P1 to P3 of the yellow / blue / red lights. Method 3: Specify the four vertices Pa to Pd when the signal light 6 is approximated as a rectangle. Method 4: Specify the two diagonal vertices Pa and Pd when the signal light 6 is approximated as a rectangle.

[0047] In Method 2, if the signal lamp 6 has an arrow light, the center point of the arrow light may be specified. Also, in Method 4, the two diagonal vertices may be Pb and Pc. The method for inputting the reference points for the light area is not particularly limited, but for example, it can be achieved by transmitting the coordinate values ​​of the planar coordinates assigned to each pixel of the image data from an input device such as a keyboard connected to the edge computer 3.

[0048] The light extraction unit 43 determines a pixel range (hereinafter referred to as the "extraction range") for extracting signal lights (such as red lights, blue lights, and yellow lights in the case of vehicle lights) included in the signal light 6, based on a reference point in the designated light area, and extracts each signal light within the determined extraction range. For example, if only one center point P1 is specified, the lamp extraction unit 43 will use a rectangular area of ​​a predetermined length centered on center point P1 as the extraction range, and if three center points P1 to P3 are specified, the extraction range will be a circular area of ​​a predetermined radius centered on each of the three center points P1 to P3.

[0049] Furthermore, when four vertices Pa to Pd are specified, the lamp extraction unit 43 uses the rectangular area enclosed by the four vertices Pa to Pd as its extraction range, and when two vertices Pa and Pc are specified, it uses the rectangular area with the two vertices Pa and Pd as its diagonals as its extraction range. The lamp extraction unit 43 extracts the coordinate values ​​of pixels corresponding to the signal lamps included in the signal lamp unit 6 (hereinafter referred to as "lamp coordinates") based on the feature quantities representing the signal lamps within the extraction range determined as described above, and outputs the extracted lamp coordinates to the lamp color determination unit 44 and the training unit 46.

[0050] Furthermore, the light extraction unit 43 considers a light to be the same signal light if the difference between the light coordinates in the previous frame and the light coordinates in the current frame is within a predetermined range. This allows the system to track the signal light once it has been extracted, even if the signal light 6 vibrates due to strong winds, for example. Therefore, it is possible to suppress the increase in processing load caused by repeatedly extracting the light coordinates.

[0051] The lamp color determination unit 44 determines the lamp color of the signal lamp 6 based on the lamp coordinates input from the lamp extraction unit 43. Specifically, the light color determination unit 44 determines the color of the currently lit signal light from the RGB values ​​contained in the pixel data corresponding to the light coordinate. The light color determination unit 44 outputs the determined current light color (hereinafter referred to as "current light color") to the information generation unit 45.

[0052] The information generation unit 45 creates the aforementioned timetable from the current duration of the light color and records the created timetable in the timetable database 36. The information generation unit 45 generates signal information S1 from the timetables recorded in the timetable database 36, and outputs the generated signal information S1 to the communication unit 33.

[0053] Training unit 46 also determines the color of the currently lit signal light from the RGB values ​​contained in the pixel data corresponding to the light coordinates. The training unit 46 is capable of performing training processes using the colors of signal lights. These training processes include the following processes 1 to 4.

[0054] Process 1: A process to determine the type of light source (LED or incandescent bulb) of the signal light 6 based on the presence or absence of periodicity in brightness fluctuations in the pixels corresponding to the signal light (hereinafter referred to as "type determination"). Process 2: When the light source of the signal lamp 6 is an LED, the exposure time of the roadside camera 2 is synchronized to the timing when the brightness of the pixel corresponding to the signal lamp exceeds a predetermined value (hereinafter referred to as "timing adjustment").

[0055] Process 3: When the light source of the signal lamp 6 is an incandescent bulb, the process of detecting a lower-than-normal brightness time is set as the time when the signal lamp 6 changes color (hereinafter referred to as "time detection"). Process 4: Based on the detection time obtained in Process 3, the process of updating the timetable that serves as the source data for signal information S1 (hereinafter referred to as "updating the timetable").

[0056] Processes 1 and 2 are performed during the training period (e.g., 10 minutes) before the distribution of signal information S1 (before operation). Processes 3 and 4 are performed after the distribution of signal information S1. The training unit 46 decides which of processes 2 through 4 to perform based on the result of the type determination (process 1). Specifically, if the determination result is "LED", it performs timing adjustment (process 2), and if the determination result is "light bulb", it performs time detection (process 3) and updating the timetable (process 4).

[0057] When the training unit 46 performs timing adjustment (process 2), it communicates with the roadside sensor 2 via the camera control unit 47. The commands of the control signal C1 for control communication include notifications for starting calibration mode, notifications for ending calibration mode, instructions for changing the frame rate, and instructions for changing the exposure time. When the timing adjustment (process 2) is completed, the training unit 46 outputs an output permission control signal C2 to the light color determination unit 44. The light color determination unit 44 outputs the current light color to the information generation unit 45 based on the input of the control signal C2.

[0058] [Variations in signal light types] The traffic signals currently in use on Japan's road network consist of a mix of the following types 1 through 3. Type 1: This type has signal lights that use multiple LEDs (Light-Emitting Diodes) as light sources and is driven by a 60Hz AC power supply. LED traffic lights installed in the Kansai region are of Type 1. Hereafter, signal light 6 of Type 1 will be referred to as "Signal Light 6A".

[0059] Type 2: This type has signal lights that use multiple LEDs as light sources and is driven by a 50Hz AC power supply. LED traffic lights installed in the Kanto region are Type 1. Hereafter, signal light 6 of Type 2 will be referred to as "Signal Light 6B No. 2". Type 3: This type uses incandescent light bulbs as the light source for signal lights. Type 3 signals are installed in suburban areas where LED conversion has not yet progressed. Hereafter, signal light 6 of Type 3 will be referred to as "Third Signal Light 6C".

[0060] [Problems and solutions for the first type of light fixture] Figure 5 is a time chart showing the relationship between the brightness change of the first lamp 6A and the exposure time. Here, if the exposure start time is denoted as ts and the exposure end time as te, then the "exposure time" is defined as the time (ts + te) / 2. However, in the case of high-speed shooting where the exposure time (shutter speed) is sufficiently small compared to the reciprocal of the frame rate, the exposure time may be set to ts or te. This is also true for the case of Figure 6.

[0061] In the first traffic light 6A in Figure 5, "G" represents a blue light, "Y" represents a yellow light, and "R" represents a red light. The frame rate of the roadside camera 2 is set to the default value (30fps). The meanings of the variables shown in Figure 5 are as follows: Ta: Period of change in brightness of a lit signal light (=1 / 60 ≈ 17ms) Tf: Period of exposure time (= 1 / 30 ≈ 33 ms) bi(i=1,2…): Data sequence of exposure times when detecting maximum brightness. di(i=1,2…): A data sequence of exposure times delayed by Ta / 4 from bi.

[0062] As shown in Figure 5, in the case of the first lamp 6A, the generation period Tf is twice the change period Ta. Therefore, if, for example, the exposure time bi is synchronized with the timing of maximum brightness, the illumination of the signal lamp can be appropriately determined. However, if the exposure time di is synchronized with the timing of minimum brightness, there is a high possibility of misinterpreting an illuminated signal light as being off. Therefore, in the case of the first light 6A, the exposure time bi should be set to the time when brightness above a predetermined value is detected.

[0063] [Challenges and solutions for the second type of traffic light] Figure 6 is a time chart showing the relationship between the brightness change of the second lamp 6B and the exposure time. In Figure 6, in the second traffic light 6B, "G" represents a blue light, "Y" represents a yellow light, and "R" represents a red light. The frame rate of the roadside camera 2 is assumed to be set to the default value (30fps). The meanings of the variables shown in Figure 6 are as follows:

[0064] Tb: Period of change in brightness of a lit signal light (=1 / 50=20ms) Tf: Period of exposure time (= 1 / 30 ≈ 33 ms) bi(i=1,2…): Data sequence of exposure times when maximum brightness is detected on the first exposure.

[0065] As shown in Figure 6, in the case of the second light 6B, the generation period Tf is not an integer multiple of the change period Tb. Therefore, for example, as with exposure time bi, even if the maximum brightness is detected on the first exposure, the brightness may decrease on the second exposure or become zero on the third exposure. In this way, periodic fluctuations (flicker phenomenon) occur in the brightness of the pixels corresponding to the signal lights, which reduces the accuracy of light color determination. Therefore, in the case of the second lamp 6B, the generation period Tf should be set to an integer multiple of the change period Tb (for example, 2 times), and the exposure time bi should be set to the time when a brightness of a predetermined value or higher is detected.

[0066] [Challenges and Solutions for the Third-Party Lighting] Figure 7 is an explanatory diagram showing variations in the color detection pattern of the third lamp 6C. In Figure 7, detection pattern 1 represents the case where the following events are detected at time x1 and time x2, which is one frame later (33ms later). x1: Detects blue light at normal brightness x2: Yellow light detected at normal brightness

[0067] Detection pattern 2 represents the case where the following events are detected at time y1 and at time y2, which is one frame later (33ms later). y1: Blue light detected at normal brightness y2: Detect blue light at low brightness (pale blue) or off, or detect yellow light at off or at low brightness (pale yellow).

[0068] As in detection pattern 1, if a blue light of normal brightness is detected at time x1 and a yellow light of normal brightness is detected at time x2, it is not possible to determine at what point between time x1 and time x2 the color change occurred from blue to yellow. Therefore, there may be an error of up to 33ms in the detected time of the color change from blue to yellow.

[0069] On the other hand, while the brightness of an incandescent light bulb fluctuates very little while lit, due to transient phenomena in the filament, it completely turns off after a predetermined fall time Δd and completely turns on after a predetermined rise time Δu. Therefore, if blue or yellow light can be detected at a lower brightness than usual, as in the case of detection pattern 2 at time y2, this detection time y2 is considered to be in close agreement with the actual time of the light color change from blue to yellow, and the error is considered to be at least less than 33ms.

[0070] Therefore, if the above time y2 can be detected, that time y2 should be set as the time for the light color change. Specifically, if a predetermined light color can be detected at a level below the higher first threshold TH1 (e.g., 30% of normal brightness) and above the lower second threshold TH2 (e.g., zero), then the determined time y2 should be determined as the time of the light color change.

[0071] [Specific examples of type determination] Figure 8 is a flowchart showing an example of type determination performed by the control unit 31 (training unit 46) of the edge computer 3. As shown in Figure 8, the control unit 31 sets the frame rate of the roadside camera 2 to 30fps using the control signal C1 (step ST11), and then detects the brightness fluctuation of the lamp coordinates corresponding to the signal lamps extracted by the lamp extraction unit 43 (step ST12).

[0072] Next, the control unit 31 determines whether the brightness fluctuation is periodic or not (step ST13). If the result of step ST13 is positive, the control unit 31 determines that the signal lamp 6 is the second lamp 6B (step ST14) and terminates the process. If the determination result in step ST13 is negative, the control unit 31 determines that the signal lamp 6 is either the first lamp 6A or the third lamp 6C (step ST15).

[0073] Next, the control unit 31 sets the frame rate of the roadside camera 2 to 25fps using the control signal C1 (step ST16), and then detects the brightness fluctuation of the lamp coordinates corresponding to the signal lamps extracted by the lamp extraction unit 43 (step ST17).

[0074] Next, the control unit 31 determines whether the brightness fluctuation is periodic or not (step ST18). If the result of step ST18 is positive, the control unit 31 determines that the signal lamp 6 is the first lamp 6A (step ST19) and terminates the process. If the determination result in step ST18 is negative, the control unit 31 determines that the signal lamp 6 is the third lamp 6C (step ST20) and terminates the process.

[0075] [Specific examples of timing adjustment] Figure 9 is a flowchart showing an example of timing adjustment performed by the control unit 31 (training unit 46) of the edge computer 3. As shown in Figure 9, the control unit 31 notifies the roadside camera 2 of the start of calibration mode using the control signal C1 (step ST21), and then determines the type of signal light 6 (in this case, only LED signal lights) (step ST22).

[0076] If the determination result in step ST22 is "Type 1" (first light 6A), the control unit 31 sends a control signal C1 to the roadside sensor 2 to set the frame rate to 30fps (step ST23). This sets the generation period of the exposure time bi to an integer multiple (specifically, 2 times) of the brightness change period of the LED driven by a 60Hz AC power supply.

[0077] If the determination result in step ST21 is "Type 2" (second traffic light 6B), the control unit 31 sends a control signal C1 to the roadside sensor 2 to set the frame rate to 25fps (step ST24). This sets the generation period of the exposure time bi to an integer multiple (specifically, 2 times) of the brightness change period of the LED driven by a 50Hz AC power supply.

[0078] Next, the control unit 31 determines whether the brightness L of the signal light is equal to or greater than a predetermined threshold TH (step ST25). Brightness L is the brightness of the lamp coordinates corresponding to the signal lamp extracted by the lamp extraction unit 43. Threshold TH is a predetermined set value, for example, the minimum brightness value required to accurately perform lamp color determination.

[0079] If the determination result in step ST25 is negative, the control unit 31 instructs the roadside camera 2 to change the exposure time using the control signal C1 (step ST26). Instructions to change the exposure time may include, for example, an instruction to delay the exposure time by a predetermined time Δt (e.g., 2ms) from the current time, or conversely, an instruction to advance the exposure time by a predetermined time Δt from the current time.

[0080] The control unit 31 repeats the instruction to change the exposure time (step ST26) until the determination result of step ST25 becomes positive. If the determination result in step ST25 is positive, the control unit 31 notifies the roadside camera 2 of the end of the calibration mode by control signal C1 (step ST27).

[0081] [Specific examples of time detection] Figure 10 is a flowchart showing an example of time detection performed by the control unit 31 (training unit 46) of the edge computer 3. As shown in Figure 10, the control unit 31 constantly monitors the luminance L (step ST31). The luminance L is the luminance of the lamp coordinates corresponding to the signal lamp extracted by the lamp extraction unit 43.

[0082] Next, the control unit 31 determines whether the luminance L is less than or equal to the first threshold TH1 (higher value) and greater than or equal to the second threshold TH2 (lower value) (step ST32). The first threshold TH1 is set to, for example, 30% of the continuously monitored luminance statistics (such as the mean or median). The second threshold TH2 is set to, for example, zero.

[0083] If the determination result in step ST32 is positive, the control unit 31 records the following detection information 1 to detection information 3 into memory (step ST33). Detection information 1: Detection time t1 (time y2 in Figure 7) of the brightness L that satisfies step ST32. Detection Information 2: The color of the light immediately before detection time t1 (hereinafter referred to as "previous color"). Detection Information 3: The color of the light immediately after detection time t1 (hereinafter referred to as "subsequent color").

[0084] [Specific examples of updating the timetable] Figure 11 is a flowchart showing an example of updating the timetable performed by the control unit 31 (training unit 46) of the edge computer 3. As shown in Figure 11, the control unit 31 extracts the detection time t1 detected by time detection (Figure 10), and the lamp color switching time t2 corresponding to the preceding and succeeding colors from the currently operating timetable (step ST41).

[0085] For example, if the preceding color is red and the following color is blue, the detection time t1 is the time when the signal changes from red to green. In this case, the control unit 31 calculates the red end time (or blue start time) from the timetable and sets the calculated time as the light color change time t2 corresponding to the detected time t1.

[0086] Next, the control unit 31 determines whether the absolute value of the difference between time t1 and time t2 (t1-t2) is less than or equal to a predetermined value ε (for example, 3ms) (step ST42). If the result of step ST42 is negative, the control unit 31 performs a correction to the timetable (step ST43). This correction is performed, for example, by increasing or decreasing the duration of each light color included in the timetable.

[0087] Specifically, if the difference (t1-t2) is a positive number, the control unit 31 reduces the duration of each light color in the timetable by the absolute value of the difference (t1-t2). Conversely, if the difference (t1-t2) is a negative number, the control unit 31 increases the duration of each light color in the timetable by the absolute value of the difference (t1-t2). If the determination result in step ST42 is positive, the control unit 31 skips the correction in step ST43 and terminates the process.

[0088] [First variation] In the above embodiment, an edge computer 3 that does not perform type determination (Figure 8) may be used. Specifically, depending on the type of light source of the signal lamp 6 determined manually, either the first computer 3A or the second computer 3B may be used.

[0089] Computer 1A: This is an edge computer 3 specifically for LED traffic signals. The first computer 3A performs timing adjustment (Figure 9), but does not perform type determination (Figure 8), time detection (Figure 10), or updating of the timetable (Figure 11). Computer 3B (second computer): This is edge computer 3 specifically for incandescent traffic signals. The second computer 3B performs time detection (Figure 10) and updating of the timetable (Figure 11), but does not perform type determination (Figure 8) or timing adjustment (Figure 9).

[0090] [Other variations] The embodiments described above are illustrative and not restrictive in all respects. The scope of the present invention includes all modifications within the scope equivalent to the configurations described in the claims. For example, in the embodiment described above, the roadside camera 2 and the edge computer (information processing device) 3 were shown as separate devices, but both devices may be housed in a single enclosure as an integrated information processing device. [Explanation of symbols]

[0091] 1. Information Provision System 2 Roadside cameras 3. Edge Computers (Information Processing Devices) 4. Terminal devices (in-vehicle terminals) 5. Terminal device (user terminal) 6. Traffic lights 6A 1st light device (LED: 60Hz power supply) 6B 2nd light device (LED: 50Hz power supply) 6C Third light fixture (light bulb) 7 vehicles 8 Pedestrians 9 Wireless base stations 10 Public telecommunications network 31 Control Unit 32 Storage section 33 Communications Department (Acquisition Department, Distribution Department) 34 Synchronization Processing Unit 35 Member Database 36. Timetable Database 38 Computer Programs 41 Data Input Section 42 Area setting section 43 Light equipment extraction part 44 Light color determination section 45 Information generation section 46 Training Department 47 Camera Control Unit J intersection S1 Signal Information C1 control signal C2 control signal

Claims

1. An acquisition unit that acquires image data of videos generated by roadside cameras, An information processing apparatus comprising: a control unit that extracts pixels corresponding to multiple signal lights included in a signal light from the image data, The control unit, Based on the presence or absence of periodicity in brightness fluctuations in the aforementioned pixels, a type determination is performed to determine the type of light source of the signal lamp. An information processing device that, given that the type determined by the type determination is an incandescent bulb, performs time detection, setting the time at which a brightness lower than a threshold is detected as the time when the signal light changes color.

2. An acquisition unit that acquires image data of a video generated by a roadside camera, An information processing apparatus comprising: a control unit that extracts pixels corresponding to multiple signal lights included in a signal light from the image data, The control unit, Based on the presence or absence of periodicity in brightness fluctuations in the aforementioned pixels, a type determination is performed to determine whether the light source type of the signal lamp is an LED or an incandescent bulb. If the type determined by the type determination is the LED, The control unit, The information processing apparatus according to claim 1, which performs timing adjustment to synchronize the exposure time of the roadside camera with the timing at which the brightness of the pixel corresponding to the signal light exceeds a predetermined value.

3. The aforementioned timing adjustment is, The information processing apparatus according to claim 2, comprising the process of setting the period of occurrence of the exposure time to an integer multiple of the period of brightness fluctuation of the LED that is lit by an AC power supply.

4. The control unit, The information processing apparatus according to claim 1, which performs a process to update a time table including the transition order of light colors for each inflow path and the duration of each light color based on the detection time.

5. The control unit, The information processing device according to any one of claims 1 to 4, which performs a process of determining the current light color of the signal lamp from the image data and generating signal information for the intersection based on the determined current light color.

6. As steps performed by the information processing device, The steps include acquiring image data from the video generated by the roadside camera, The steps include extracting pixels corresponding to multiple signal lights included in the signal light from the aforementioned image data, A step of determining the type of light source of the signal lamp based on whether or not there is periodicity in the brightness fluctuations of the pixels, An information processing method comprising the step of performing a time detection, where the time at which a brightness lower than a threshold is detected is set to be the time when the signal light changes color, provided that the type determined by the type determination is an incandescent bulb.

7. An acquisition unit that acquires video image data generated by a roadside camera, and A computer program that causes a computer to function as an information processing device comprising a control unit that extracts pixels corresponding to multiple signal lights included in a signal light from the aforementioned image data, The control unit, Based on the presence or absence of periodicity in brightness fluctuations in the aforementioned pixels, a type determination is performed to determine the type of light source of the signal lamp. A computer program that performs time detection, where the time at which a brightness level lower than a threshold is detected is set as the time when the signal light changes color, provided that the type determined by the type determination is an incandescent bulb.

8. As steps performed by the information processing device, The steps include acquiring image data from the video generated by the roadside camera, The steps include extracting pixels corresponding to multiple signal lights included in the signal light from the aforementioned image data, A step of determining whether the light source of the signal lamp is an LED or an incandescent bulb based on the presence or absence of periodicity in the brightness fluctuations of the aforementioned pixels, An information processing method comprising the step of, if the type determined by the type determination is the LED, performing timing adjustment to synchronize the exposure time of the roadside camera with the timing at which the brightness of the pixel corresponding to the signal light becomes equal to or greater than a predetermined value.

9. An acquisition unit that acquires video image data generated by a roadside camera, and A computer program that causes a computer to function as an information processing device comprising a control unit that extracts pixels corresponding to multiple signal lights included in a signal light from the aforementioned image data, The control unit, Based on the presence or absence of periodicity in brightness fluctuations in the aforementioned pixels, a type determination is performed to determine whether the light source type of the signal lamp is an LED or an incandescent bulb. If the type determined by the type determination is the LED, The control unit, A computer program that adjusts the timing of the exposure time of the roadside camera to synchronize it with the timing at which the brightness of the pixels corresponding to the signal lights exceeds a predetermined value.

Images