Information processing device, information processing method, and computer program

Abstract

A device according to one embodiment of the present invention comprises: a communication unit that receives image data in which a traffic signal at an intersection is included in the depicted scene; and a control unit that generates signal information representing the color of the traffic signal light on the basis of the image data and transmits the signal information to the communication unit, wherein the image data is image data generated by a roadside camera installed so as to include the traffic signal in the angle of view.

Description

Information processing device, information processing method, and computer program

[0001] This disclosure relates to an information processing device, an information processing method, and a computer program. This application claims priority to Japanese Application No. 2022-56321 filed on March 30, 2022, and incorporates by reference all of the contents of that application.

[0002] Patent Literature 1 describes a technology that acquires the position information of a traffic light from map information, and uses image data acquired by an in-vehicle camera that photographs the traffic light at that position to determine the current light color of the traffic light. Patent Literature 2 describes a technology that prompts a vehicle to stop when the current light color of a traffic light determined by an in-vehicle camera is yellow.

[0003] JP 2020-24715 A JP 2017-37528 A

[0004] An apparatus according to one aspect of the present disclosure is an information processing apparatus including a communication unit that receives image data in which a traffic light at an intersection is included in the scenery, and a control unit that generates signal information representing the light color state of the traffic light based on the image data and transmits the signal information to the communication unit, wherein the image data is image data generated by a roadside camera installed so as to include the traffic light in its angle of view.

[0005] An apparatus according to another aspect of the present disclosure is an information processing apparatus including a communication unit that receives data indicating a change in the light color of a traffic light at an intersection, and a control unit that generates signal information indicating the light color state of the traffic light based on the data and causes the signal information to be transmitted to the communication unit, wherein the communication unit is capable of transmitting a communication packet including the signal information, and the communication packet includes additional information regarding the data used to generate the signal information.

[0006] A computer program according to one aspect of the present disclosure is a computer program that causes a computer to function as an information processing device that includes a communication unit that receives image data in which a traffic light at an intersection is included in the scenery, and a control unit that generates signal information representing the light color state of the traffic light based on the image data and transmits the signal information to the communication unit, wherein the image data is image data generated by a roadside camera installed so as to include the traffic light in its angle of view.

[0007] A method according to one aspect of the present disclosure is an information processing method including, as steps executed by an information processing device, receiving image data in which a traffic light at an intersection is included in the scenery, generating signal information representing the light color state of the traffic light based on the image data, and transmitting the signal information to an external device, wherein the image data is image data generated by a roadside camera installed so as to include the traffic light in its angle of view.

[0008] The present disclosure can be realized not only as a system and device having the above-described characteristic configuration, but also as a program for causing a computer to execute such characteristic configuration. Furthermore, the present disclosure can be realized as a semiconductor integrated circuit that realizes part or all of the system and device.

[0009] FIG. 1 is a road plan view showing an example of the configuration of an information provision system. FIG. 2 is a block diagram showing an example of the hardware configuration of an edge computer. FIG. 3 is a block diagram showing an example of the functional configuration of an edge computer. FIG. 4 is an explanatory diagram showing an example of a method for setting a reference point for a light area. FIG. 5 is a flowchart showing an example of a first check process. FIG. 6 is a flowchart showing an example of a second check process. FIG. 7 is a flowchart showing an example of a third check process. FIG. 8 is a flowchart showing an example of a fourth check process. FIG. 9 is a flowchart showing an example of a fifth check process. FIG. 10 is a flowchart showing an example of a sixth check process. FIG. 11 is an explanatory diagram showing an example of the data structure of a communication packet for traffic light information.

[0010] <Problem to be Solved by the Present Disclosure> In Patent Documents 1 and 2, the current light color of a traffic light is determined based on image data captured by an onboard camera. However, it is difficult for an onboard camera to capture a traffic light due to a large vehicle ahead, and even if it does capture the light, the capture time is short. This makes it difficult to accurately determine the current light color of a traffic light, and there is a possibility that inaccurate traffic light information will be distributed. In view of these conventional problems, the present disclosure aims to provide an information processing device and the like that can distribute accurate traffic light information.

[0011] <Effects of the Present Disclosure> According to the present disclosure, accurate signal information can be distributed.

[0012] <Outline of Embodiments of the Present Disclosure> The following is a list and description of outlines of embodiments of the present disclosure. (1) A device according to one aspect of the present embodiment is an information processing device including a communication unit that receives image data in which a traffic light at an intersection is included in a scene, and a control unit that generates signal information representing a lighting color state of the traffic light based on the image data and causes the signal information to be transmitted to the communication unit, wherein the image data is image data generated by a roadside camera installed so as to include the traffic light in its angle of view.

[0013] According to the information processing device of this embodiment, the image data used to generate traffic light information is generated by a roadside camera installed so that the traffic light is included in its angle of view, so that more accurate image data can be obtained with a longer capture time that reliably captures the traffic light compared to when image data acquired by an in-vehicle camera is used. This improves the accuracy of determining the current light color of the traffic light, enabling accurate traffic light information to be distributed.

[0014] (2) In the information processing device of this embodiment, the communication unit may be capable of transmitting a communication packet including the signal information, and the communication packet may include additional information related to a generation method of the signal information.

[0015] In this case, for example, by defining additional information indicating the source of image data as a method for generating traffic light information, a vehicle receiving a communication packet can determine based on the additional information that the traffic light information is based on image data from a roadside camera, thereby notifying the vehicle that the traffic light information is highly reliable.

[0016] (3) In the information processing device of this embodiment, the control unit may determine an extraction range of the signal light, which is a pixel range including a reference point set by the user in the image data, and determine the current light color of the signal light based on the pixel data within the extraction range.

[0017] In this way, compared to determining the current light color of a signal light from all pixel data contained in the image data, it is possible to reduce extraction errors of the signal light and determine the current light color with high accuracy, and it is also possible to determine the current light color quickly and with low load.

[0018] (4) In the information processing device of this embodiment, the control unit may execute a time consistency check to check the temporal consistency of the operation of the signal lamp device.

[0019] In this way, it is possible to prevent signal information from being generated based on the operation of a signal lamp that lacks temporal consistency, thereby improving the accuracy of the signal information.

[0020] (5) In the information processing device of this embodiment, the time consistency check may include a first check to check whether a change in the light color of the signal lamp corresponds to a prohibited operation.

[0021] In this way, it is possible to prevent signal information from being generated based on an inaccurate change in light color that corresponds to a prohibited operation, thereby improving the accuracy of the signal information.

[0022] (6) In the information processing device of this embodiment, the time consistency check may include a second check to check whether the change in light color of the signal lamp is simultaneous with a pre-calculated estimated time limit table. Note that if a time limit table describing the light color changes of the signal lamp is disclosed by the operation manager of the signal lamp, the time limit table may be used instead of the estimated time limit table.

[0023] This prevents traffic signal information from being generated based on inaccurate changes in light color that are not synchronized with the estimated time limit table, thereby improving the accuracy of the traffic signal information.

[0024] (7) In the information processing device of this embodiment, the control unit may execute a spatial consistency check to check the spatial consistency of the operation of the signal lamp device.

[0025] In this way, it is possible to prevent signal information from being generated based on the operation of signal lamps that lack spatial consistency, thereby improving the accuracy of the signal information.

[0026] (8) In the present embodiment, the spatial consistency check may include a third check that checks for synchronicity of changes in light color of a pair of the signal lamp devices that are oriented in directions facing each other.

[0027] This prevents signal information from being generated based on inaccurate light color changes of a pair of opposing signal lamps that are not synchronized, thereby improving the accuracy of the signal information.

[0028] (9) In the information processing device of this embodiment, the spatial consistency check may include a fourth check that checks whether the change in light color of a pair of signal lights facing in directions that intersect with each other is in a forbidden relationship.

[0029] In this way, it is possible to prevent signal information from being generated based on inaccurate light color changes related to a pair of signal lamps that are in a forbidden relationship, thereby improving the accuracy of the signal information.

[0030] (10) In the information processing device of this embodiment, the signal lights may include a vehicle light and a pedestrian light, and the spatial consistency check may include a fifth check that checks whether the relationship between the change in light color of the vehicle light and the change in light color of the pedestrian light violates a transition rule.

[0031] This prevents signal information from being generated based on inaccurate light color changes for vehicle and pedestrian lights that violate the transition rules, thereby improving the accuracy of the signal information.

[0032] (11) In the information processing device of this embodiment, the spatial consistency check may include a sixth check of checking whether the change in light color of the signal light is consistent with the behavior of a vehicle that is located upstream of and closest to the stop line in the direction in which the signal light grants the right of way.

[0033] In this way, it is possible to confirm whether the light color determination of the signal lamp based on image data is correct or not, based on the behavior of the actual vehicle (leading vehicle) about to pass through the intersection.

[0034] (12) In the information processing device of this embodiment, the communication unit may be capable of transmitting a communication packet including the signal information, and the communication packet may include identification information indicating whether or not the signal information is corrected and the type of correction.

[0035] In this way, a vehicle that receives a communication packet can determine whether the signal information is information that involves correction of light color determination, and if correction is involved, the type of correction.

[0036] (13) In the information processing device of this embodiment, if the time consistency check determines that there is no consistency and if a spatial consistency check, which checks the spatial consistency of the operation of the signal light, is not possible, the control unit may determine that the light color state of the signal light is indefinite.

[0037] This prevents signal information from being generated based on an inaccurate light color state, thereby improving the accuracy of the signal information.

[0038] (14) A device according to another aspect of this embodiment is an information processing device that includes a communication unit that receives data indicating a change in the light color of a traffic light at an intersection, and a control unit that generates signal information indicating the light color state of the traffic light based on the data and causes the communication unit to transmit the signal information, wherein the communication unit is capable of transmitting a communication packet including the signal information, and the communication packet includes additional information related to a method for generating the signal information.

[0039] According to the information processing device of this embodiment, for example, by defining additional information indicating the information source of image data as a method for generating traffic light information, a vehicle receiving a communication packet can recognize from the additional information that the traffic light information is based on image data from a roadside camera, and can therefore be notified that the traffic light information is highly reliable.

[0040] (15) A computer program according to one aspect of this embodiment is a computer program that causes a computer to function as the information processing device described above in (1) to (13). Therefore, the computer program of this embodiment has the same effects as the information processing device described above in (1) to (13).

[0041] (16) A method according to one aspect of the present embodiment is an information processing method executed by the information processing device described above in (1) to (13). Therefore, the information processing method according to the present embodiment has the same effects as the information processing device described above in (1) to (13).

[0042] <Details of Embodiments of the Present Disclosure> Hereinafter, details of embodiments of the present disclosure will be described with reference to the drawings. Note that at least some of the embodiments described below may be combined in any manner.

[0043] [Overall System Configuration] Fig. 1 is a road plan view showing an example configuration of an information provision system 1 according to this embodiment. Fig. 2 is a block diagram showing an example hardware configuration of an edge computer 3. As shown in Figs. 1 and 2, the information provision system 1 includes a roadside camera 2 installed at an appropriate location at an intersection J, an edge computer (information processing device) 3, and terminal devices 4 and 5 capable of wireless communication with the edge computer 3.

[0044] At intersection J, a signal light 6 is installed to indicate whether or not there is right-of-way on the approach road. The signal light 6 is installed on a pole located on the side of the road at intersection J. The signal light 6 illustrated in FIG. 1 is a vehicle light including at least a blue light, a yellow light, and a red light. The vehicle light may also have an arrow signal light such as a right-turn arrow light. Although not shown in FIG. 1, intersection J may also include a pedestrian light.

[0045] The roadside camera 2 is installed on a pole or the like near the intersection J. The roadside camera 2 is a digital camera equipped with an image sensor capable of capturing video. The roadside camera 2 is capable of adjusting the angle of view to enlarge or reduce it, and is installed so that the traffic light 6 fits within the angle of view. Therefore, the image data output by the roadside camera 2 includes at least one traffic light 2 installed at the intersection J as part of the scenery reflected in the image data.

[0046] The roadside cameras 2 may be installed so that one traffic light 6 fits within the angle of view of one roadside camera 2, or so that two or more traffic light devices 6 fit within the angle of view of one roadside camera 2. Also, although two roadside cameras 2 are installed in Fig. 1, the number of roadside cameras 2 installed may be one or three or more.

[0047] The edge computer 3 is an information processing device that is communicatively connected to the roadside camera 2, and is installed on a support pole or the like near the intersection J. The roadside camera 2 and the edge computer 3 may be connected by wire or wirelessly. The edge computer 3 has a function for setting the roadside camera 2, a function for generating predetermined downlink information from image data input from the roadside camera 2, and a function for distributing the generated downlink information to the terminal devices 4 and 5.

[0048] The terminal device 4 is an on-board terminal permanently or temporarily installed in the vehicle 7. The vehicle 7 refers to all vehicles traveling on roads. Therefore, in addition to automobiles, light vehicles, and trolleybuses, motorcycles also fall under the category of vehicles. The drive system of the vehicle 7 is not limited to internal combustion engines, and electric vehicles and hybrid cars are also included in the vehicle. The vehicle 7 may be a normal vehicle driven by a passenger, or may be an autonomous vehicle of level 3 or higher.

[0049] The terminal device 5 is a user terminal carried by a pedestrian 8 (see FIG. 2 ). The user terminal is, for example, a smartphone, a tablet PC, or a laptop PC. The terminal devices 4 and 5 are capable of wireless communication with a wireless base station 9 (e.g., a mobile base station). The wireless base station 9 is capable of communication with the edge computer 3 via a public communication network 10, which includes a mobile communication core network and the Internet.

[0050] The edge computer 3 transmits communication packets containing downlink information addressed to the terminal devices 4 and 5 to the public communication network 10. Therefore, the communication packets containing downlink information are distributed to the terminal devices 4 and 5 via the wireless base station 9. Here, a DSRC (Dedicated Short Range Communication) communication device (not shown) may be used as a means for transmitting the communication packets, instead of the public wireless network 10. The DSRC method is also called a short-range communication method. The downlink information includes traffic light information S1 of the intersection J. The traffic light information S1 is information indicating the light color status of the traffic light device 6. In this embodiment, the traffic light information S1 indicates the light color status at the current time as well as the light color status for a predetermined period (e.g., two cycles) into the near future. Specifically, the traffic light information S1 of the intersection J includes the following information 1 to information 5.

[0051] Information 1: Location and name of intersection J to be provided; Information 2: Location and name of the oncoming road to which the signal lamp 6 to be provided grants the right of way; Information 3: Current light color of the oncoming road (which lamp is on); Information 4: Lighting sequence of the oncoming road from the current time onwards and passable direction when the lamp is on; Information 5: Timing to turn on or off each lamp (can be either absolute time or the number of seconds remaining from a reference time)

[0052] However, the traffic light information S1 of the intersection J may be information that indicates only the current light color state of the traffic light device 6, in which case the above information 1 to 3 is provided to the terminal devices 4 and 5.

[0053] 2, the edge computer 3 includes a control unit 31, a storage unit 32, a communication unit 33, a synchronization processing unit 34, and multiple types of databases 35 to 37. The databases 35 to 37 are electronic data constructed in a predetermined data array in the storage unit 32. However, some or all of the databases 35 to 37 may be constructed in an external storage device (not shown) connected to the edge computer 3.

[0054] The control unit 31 is an arithmetic processing device including a CPU (Central Processing Unit) and RAM (Random Access Memory). The control unit 31 may include an integrated circuit such as an FPGA (Field-Programmable Gate Array). The control unit 31 reads a computer program 38 stored in the storage unit 32 into the main memory (RAM) and executes various information processing operations in accordance with the program 38. The information processing operations include generating traffic signal information S1 from image data captured by the roadside camera 2.

[0055] The storage unit 32 is an auxiliary storage device including a nonvolatile memory such as a hard disk drive (HDD) and a solid state drive (SSD). The storage unit 32 may also include a flash read-only memory (ROM), a universal serial bus (USB) memory, an SD card, or the like.

[0056] The communication unit 33 is a communication interface that performs public communication via the public communication network 10 and dedicated communication with the roadside camera 2. The communication unit 33 may be composed of two communication modules. When the communication unit 33 receives image data from the roadside camera 2, it outputs the received image data to the control unit 31. Therefore, the communication unit 33 functions as an acquisition unit that imports image data whose information source is the roadside camera 2 into the edge computer 3.

[0057] The control unit 31 generates traffic light information S1 based on the input image data and outputs it to the communication unit 33. The communication unit 33 generates a communication packet addressed to the terminal devices 4 and 5, including the traffic light information S1 input from the control unit 31, and transmits the generated communication packet to the wireless base station 9. Therefore, the communication unit 33 functions as a distribution unit that distributes the traffic light information S1 to external devices such as the terminal devices 4 and 5. Note that the external device to which the traffic light information S1 is distributed may be not only the terminal devices 4 and 5 but also a server (for example, a server of an insurance company that investigates accidents).

[0058] The synchronization processing unit 34 is a processing unit for achieving time synchronization with other communication nodes such as the terminal devices 4 and 5 using a predetermined synchronization method. The synchronization method used by the synchronization processing unit 34 may be, for example, a synchronization method based on the output of a Global Navigation Satellite System (GNSS) receiver, or a synchronization method using communication frames such as Network Time Protocol (NTP) and Precision Time Protocol (PTP).

[0059] The multiple types of databases 35 to 37 include a prohibition database (prohibition DB) 35, a time table database (time table DB) 36, and a transition rule database (transition rule DB) 37. The prohibition database 35 records prohibited actions and prohibition relationships that must not occur in the signal light device 6. A prohibited action is a light color change that is prohibited for a single signal light device 6, and includes, for example, an action of transitioning from blue to red (without passing through yellow).

[0060] The forbidden relationship refers to a combination of light color states that are prohibited for multiple signal lights 6, and includes, for example, a combination of light color states in which a pair of signal lights 6, 6 facing mutually intersecting directions (e.g., northbound and eastbound) simultaneously show green lights. The time limit table database 36 stores an estimated time limit table. The estimated time limit table is data defined as a table that represents the order of light color transitions for each incoming lane for one cycle, estimated from the light color changes for all signal lights 6 included in intersection J over a predetermined period of time (e.g., one hour).

[0061] The transition rule database 37 stores the transition rules of the signal lamps 6. The transition rules are predefined rules for the transition of light colors between vehicle lamps and pedestrian lamps. The transition rules include, for example, the following rules 1 and 2.

[0062] Rule 1: In the case of intersection J where pedestrians and vehicles are separated, the right of way is not given to both vehicle and pedestrian lights at the same time. Rule 2: In the case of intersection J where pedestrians and vehicles are not separated, the right of way of the vehicle light is lost (i.e., the light changes to yellow or red) within a specified time from the start of pedestrian flashing for vehicle and pedestrian lights in the same direction of travel.

[0063] [Functional Configuration of Edge Computer] Fig. 3 is a block diagram showing an example of the functional configuration of the edge computer 3. As shown in Fig. 3, the control unit 31 of the edge computer 3 includes a data input unit 41, an area setting unit 42, a lamp extraction unit 43, a lamp color determination unit 44, a lamp color check unit 45, an information generation unit 46, a sensor information input unit 47, and a behavior determination unit 48. These functional units 41 to 48 are realized by the control unit 31 executing a computer program 38.

[0064] The data input unit 41 receives image data received by the communication unit 33 from the roadside camera 2. The data input unit 41 outputs the received image data to the lamp extraction unit 43 and the behavior determination unit 48. The area setting unit 42 performs processing to set a reference point for an area where a traffic light 6 exists (hereinafter referred to as a "lamp area") among all pixels constituting the image data. This processing is executed, for example, by a user's initial setting of the edge computer 3.

[0065] Figure 4 is an explanatory diagram showing an example of a method for setting the reference point of a light unit area. As shown in Figure 4, any of the following methods 1 to 4 can be used to set the reference point. Method 1: Specify the center point P1 of the signal light unit 6 (center point of the yellow light). Method 2: Specify center points P1 to P3 of the yellow light, blue light, and red light. Method 3: Specify four vertices Pa to Pd when the signal light unit 6 is approximated as a rectangle. Method 4: Specify two diagonal vertices Pa and Pd when the signal light unit 6 is approximated as a rectangle.

[0066] In Method 2, in the case of a signal lamp 6 having an arrow light, the center point of the arrow light may be specified. Furthermore, in Method 4, the two diagonal vertices may be Pb and Pc. Note that the method for inputting the reference point of the lamp area is not particularly limited, but for example, it can be realized by transmitting the coordinate values ​​of the planar coordinates assigned to each pixel of the image data from an input device such as a keyboard connected to the edge computer 3.

[0067] The lamp extraction unit 43 determines a pixel range (hereinafter referred to as an "extraction range") for extracting lamps included in the signal lamps 6 (red lamps, blue lamps, yellow lamps, and arrow lamps in the case of vehicle lamps) based on the reference point of the specified lamp area, and extracts each lamp within the determined extraction range. For example, when only one center point P1 is specified, the lamp extraction unit 43 determines a rectangular range of a predetermined length centered on the center point P1 as the extraction range, and when three center points P1 to P3 are specified, the lamp extraction unit 43 determines a circular range of a predetermined radius centered on each of the center points P1 to P3 as the extraction range.

[0068] Furthermore, when four vertices Pa to Pd are specified, the lamp unit extraction unit 43 sets the extraction range to a rectangular area surrounded by the four vertices Pa to Pd, and when two vertices Pa and Pc are specified, the extraction range to a rectangular area with the two vertices Pa and Pd as diagonals. Within the extraction range determined as above, the lamp unit extraction unit 43 extracts coordinate values ​​(hereinafter referred to as "lamp coordinates") of pixels corresponding to signal lights included in the signal lamp unit 6 based on features representing the signal lights, and outputs the extracted lamp coordinates to the lamp color determination unit 44.

[0069] The light extraction unit 43 considers the light to be the same light if the difference between the light coordinates in the previous frame and the light coordinates in the current frame is within a predetermined range. This allows the signal light 6 to be tracked even if it vibrates due to strong winds, for example. Therefore, it is possible to suppress an increase in processing load caused by repeated extraction of light coordinates.

[0070] The light color determination unit 44 determines the light color of the signal light unit 6 based on the light coordinates input from the light unit extraction unit 43. Specifically, the light color determination unit 44 determines the light that is on and its light color from the RGB values ​​included in the pixel data corresponding to the light coordinates. The light color determination unit 44 outputs the determined current light color to the light color check unit 45.

[0071] The sensor information input unit 47 is an input unit that receives sensor information output by an external traffic-related sensor. The external sensor is, for example, a millimeter wave band or quasi-millimeter wave band radar sensor or a LiDAR (Light Detection and Ranging) sensor installed near the intersection J. The external sensor may also be a wireless device that transfers probe information received from the vehicle 7 to the edge computer 3. The sensor information input unit 47 is an optional functional unit of the edge computer 3 and may be omitted.

[0072] The behavior determination unit 48 determines the behavior of the vehicle 7, pedestrian 8, etc. on the road based on at least one of the image data input from the data input unit 41 and the sensor information input from the sensor information input unit 47. Specifically, the behavior determination unit 48 determines whether the vehicle 7 is stopped near the stop line of the intersection J or is passing through, and outputs the determination result to the light color check unit 45.

[0073] The light color check unit 45 performs a predetermined inspection on the current light color (the light color of the signal light unit 6 at the current time) determined by the light color determination unit 44, and then outputs the light color that passes the inspection to the information generation unit 46. The processing content of the light color check unit 45 will be described later. The information generation unit 46 generates the above-mentioned signal information S1 using the input current light color, and outputs the generated signal information S1 to the communication unit 33. Furthermore, if the information generation unit 46 creates an estimated time limit table from the signal information S1, it stores the created estimated time limit table in the database 36.

[0074] [Processing Contents of Light Color Check Unit] As shown in Fig. 3, the processing executed by the light color check unit 45 includes a "temporal consistency check" and a "spatial consistency check." The temporal consistency check is a process for checking whether there is temporal consistency in the operation of the signal light device 6. The temporal consistency check includes a first check and a second check.

[0075] The spatial consistency check is a process for checking whether the operation of a signal light device 6 is spatially consistent with the operation of other signal light devices 6. The spatial consistency check includes checks 3 to 6. The contents of checks 1 to 6 will be explained below with reference to Figures 5 to 10.

[0076] (First check (temporal consistency check): Figure 5) The first check is a process of checking whether a light color change for one signal light unit 6 corresponds to a predetermined prohibited operation recorded in the prohibition database 35. As mentioned above, a prohibited operation is, for example, an operation of transitioning from blue to red (without passing through yellow). As shown in Figure 5, the first check performed by the light color check unit 45 includes the following steps S11 to S13, depending on whether the operation corresponds to a prohibited operation and whether a spatial consistency check is possible.

[0077] Step S11: If the light color change for one signal light 6 does not correspond to a prohibited operation and the one signal light 6 is a signal light 6 for which spatial consistency check with other signal lights, etc. is not possible, the current light color input from the light color determination unit 44 is output to the information generation unit 46.

[0078] Step S12: If the change in light color of one signal light 6 corresponds to a prohibited operation and the signal light 6 is a signal light 6 that cannot be checked for spatial consistency with other signal lights, etc., the current light color of the signal light 6 is determined to be indefinite, and the determination result is output to the information generation unit 46.

[0079] Step S13: If the light color change for one signal light 6 does not constitute a prohibited operation and the one signal light 6 is a signal light 6 that can be checked for spatial consistency with other signal lights, etc., the current light color input from the light color determination unit 44 is the subject of the spatial consistency check.

[0080] (Second check (time consistency check): Figure 6) The second check is a process of checking whether the light color change for one signal light device 6 is simultaneous with the estimated time limit table recorded in the time limit table database 36. "Simultaneity" means that the changes in the two comparison objects occur at substantially the same time (the time difference is less than a threshold value). As mentioned above, the estimated time limit table is data defined as a table that represents the order of light color transitions for each incoming route for one cycle. As shown in Figure 6, the second check performed by the light color check unit 45 includes the following steps S21 to S23, depending on whether there is synchronicity with the estimated time limit table and whether a spatial consistency check is possible.

[0081] Step S21: If the time difference between the light color change for one signal light 6 and the light color change in the estimated time limit table is less than a predetermined time threshold (e.g., 0.1 seconds) and the one signal light 6 is a signal light 6 that cannot be checked for spatial consistency with other signal lights, etc., the current light color input from the light color determination unit 44 is output to the information generation unit 46.

[0082] Step S22: If the time difference between the light color change for one signal light 6 and the light color change in the estimated time limit table exceeds a predetermined time threshold, and if one signal light 6 is a signal light 6 that cannot be checked for spatial consistency with other signal lights, etc., the current light color of the signal light 6 is determined to be indefinite, and the determination result is output to the information generation unit 46.

[0083] Step S23: If the time difference between the light color change for one signal light 6 and the light color change in the estimated time limit table is less than a predetermined time threshold, and if the one signal light 6 is a signal light 6 that can be checked for spatial consistency with other signal lights, etc., the current light color input from the light color determination unit 44 is subject to the spatial consistency check.

[0084] (Third Check (Spatial Consistency Check): Figure 7) The third check is a process for checking the simultaneity of light color changes for a pair of signal lights 6, 6 (hereinafter referred to as "opposing lights") facing in opposite directions (for example, northbound and southbound). As shown in Figure 7, the third check performed by the light color check unit 45 includes the following steps S31 to S34, depending on whether there is a time difference between the opposing lights.

[0085] Step S31: If the time difference between the light color changes of the opposing light units is less than a predetermined time threshold (e.g., 0.1 seconds), the current light color of the opposing light unit input from the light color determination unit 44 is output to the information generation unit 46. Step S32: If the time difference between the light color changes of the opposing light units exceeds the predetermined time threshold, the signal light unit 6 with the smaller error from the light color change time point in the estimated time limit table (hereinafter referred to as the "accurate light unit") is distinguished from the signal light unit 6 with the larger error (hereinafter referred to as the "inaccurate light unit").

[0086] Step S33: The current light color of the accurate light unit is output as is to the information generating unit 46. Step S34: The current light color of the inaccurate light unit is output to the information generating unit 46 after correcting the time difference with the current light color of the accurate light unit.

[0087] (Fourth Check (Spatial Consistency Check): Figure 8) The fourth check is a process of checking whether the light color change of a pair of signal lights 6, 6 (hereinafter referred to as "intersection lights") facing in mutually intersecting directions (for example, northbound and eastbound) corresponds to a predetermined interdiction relationship recorded in the interdiction database 35. As mentioned above, an interdiction relationship is, for example, a combination of light color states in which the intersection lights simultaneously show green lights. As shown in Figure 8, the fourth check performed by the light color check unit 45 includes the following steps S41 to S42, depending on whether the intersection lights correspond to an interdiction relationship.

[0088] Step S41: If the light color change of the crossing light does not correspond to a forbidden relationship, the current light color of the crossing light input from the light color determination unit 44 is output to the information generation unit 46. Step S42: If the light color change of the crossing light corresponds to a forbidden change, it is determined whether the crossing light is an accurate light or an inaccurate light based on the time point of the light color change in the estimated time limit table.

[0089] Step S43: The current light color of the accurate light unit is output as is to the information generating unit 46. Step S44: The current light color of the inaccurate light unit is output to the information generating unit 46 after correcting the time difference with the current light color of the accurate light unit.

[0090] (Fifth Check (Spatial Consistency Check): FIG. 9) The fifth check is a process of checking whether the light color changes of the vehicle lights and pedestrian lights (hereinafter collectively referred to as "pedestrian lights") violate the predetermined transition rules (e.g., the above-mentioned rules 1 and 2) recorded in the transition rule database 37. As shown in FIG. 9, the fifth check executed by the light color check unit 45 includes the following steps S51 to S54, depending on whether the light color change of the pedestrian lights violates the transition rules.

[0091] Step S51: If the change in light color of the pedestrian and vehicle lights does not violate the transition rules, the current light color of the pedestrian and vehicle lights input from the light color determination unit 44 is output to the information generation unit 46. Step S52: If the change in light color of the pedestrian and vehicle lights violates the transition rules, it is determined whether the pedestrian and vehicle lights are accurate or inaccurate based on the light color change time point in the estimated time limit table.

[0092] Step S53: The current light color of the accurate light unit is output as is to the information generating unit 46. Step S54: The current light color of the inaccurate light unit is output to the information generating unit 46 after correcting the time difference with the current light color of the accurate light unit.

[0093] (Sixth Check (Spatial Consistency Check): Figure 10) The sixth check is a process of checking whether the light color change for one signal lamp 6 is consistent with the behavior of the vehicle 7 (hereinafter referred to as the "leading vehicle") that is located upstream of and closest to the stop line in the direction in which that signal lamp 6 gives the right of way. In the sixth check, for example, if the leading vehicle is stopped even though it has the right of way, or if the leading vehicle is moving even though it does not have the right of way, it is determined to be inconsistent. As shown in Figure 10, the sixth check performed by the light color check unit 45 includes the following steps S61 to S62, depending on whether there is consistency with the behavior of the leading vehicle.

[0094] Step S61: If the change in light color of one signal light 6 matches the behavior of the leading vehicle input from the behavior determination unit 48, the current light color of the signal light 6 input from the light color determination unit 44 is output to the information generation unit 46.

[0095] Step S62: If the change in light color of one signal light 6 does not match the behavior of the leading vehicle input from the behavior determination unit 48, at least one of the above-mentioned first to fifth checks is performed again, and processing is performed according to the results of the re-executed check.

[0096] In the sixth check, the determination of whether the leading vehicle will stop or pass performed by the behavior determination unit 48 is not limited to a determination based on image data from the roadside camera 2. Specifically, when sensor information from an external sensor is acquired from the sensor information input unit 47, the behavior determination unit 48 may determine whether the leading vehicle will stop or pass based on the sensor information.

[0097] [Data Structure of Communication Packet] Fig. 11 is an explanatory diagram showing an example of the data structure of a communication packet for signal information S1. As shown in Fig. 11, the communication packet includes a header portion, an actual data portion, and a frame check sequence (FCS).

[0098] The header section contains address information of the sender and destination, and the type of information to be stored in the actual data section. The actual data section stores information of the type declared in the header section (signal information S1 in this embodiment). In the communication packet of this embodiment, a storage area for additional information related to the generation method of the signal information S1 is defined in the header section. The storage area for additional information includes, for example, a storage area F1 for an information source code and a storage area F2 for a correction code.

[0099] The information source code is a code that indicates the type of information source of the image data used in the generation process of the traffic light information S1, as one type of generation method of the traffic light information S1. For example, an information source code of 0 means "roadside camera," and an information source code of 1 means "in-vehicle camera." Since the edge computer 3 generates the traffic light information S1 based on image data from the roadside camera 2, "0" is written in the storage area F1. Therefore, a vehicle 7 or a pedestrian 8 that receives the communication packet can sense that the traffic light information S1 is information based on image data from the roadside camera 2.

[0100] The information source code is not limited to the above, and may be a code indicating, for example, what data was used to generate the traffic light information S1. In this case, the information source code may be defined as follows: Information source code = 0: Traffic light information generated from image data of a roadside camera Information source code = 1: Traffic light information generated from image data of an on-board camera mounted on one vehicle Information source code = 2: Traffic light information generated by an edge computer making a comprehensive judgment based on individual pieces of information generated from image data of on-board cameras mounted on multiple vehicles

[0101] Information source code = 3: Signal information generated by an edge computer through a comprehensive judgment based on individual information generated from image data from in-vehicle cameras mounted on multiple vehicles and information generated from image data from roadside cameras. Information source code = 4: Signal information based on output from an external traffic signal controller. Information source code = 5: Signal information based on the detection results of the signal light wire current by a CT sensor.

[0102] The source code may be expanded to include signal information distribution methods that do not rely on image data for information generation, as in the examples of source codes 4 and 5. For example, source code 4 refers to "signal information output from a traffic signal controller," while source code 5 refers to "signal information detected by a CT sensor," which detects the lighting status of each light color by grasping the light wire used to drive the signal lamps. The "signal information output from a traffic signal controller" may be, for example, a time table held by the traffic signal controller, or may be information obtained by deleting, updating, or adding specific information from the time table. The "signal information detected by a CT sensor" may be, for example, the current value of the light wire measured by the CT sensor, or may be information obtained by deleting, updating, or adding specific information from the current value.

[0103] This allows the receiver to know the source of the signal information even when the edge computer distributes signal information S1 to a vehicle 7 or a pedestrian 8 via the communication unit 33 based on current light color information acquired from an external source such as a traffic signal controller or CT sensor (external acquisition system not shown).

[0104] The correction code is a code that indicates whether or not the traffic light information S1 is corrected and the type of correction, as one type of generation method of the traffic light information S1. For example, a correction code of 0 means "no correction," a correction code of 1 means "correction made by the third check," a correction code of 2 means "correction made by the fourth check," and a correction code of 3 means "correction made by the fifth check." Therefore, upon receiving the communication packet, the vehicle 7 can determine whether the traffic light information S1 is information that requires correction of the light color determination, and, if correction is required, the type of correction.

[0105] 11, the storage areas F1 and F2 may be defined in the actual data portion. The source code and correction code may be defined by identification values ​​written in multiple storage areas.

[0106] [Other Modifications] The above-described embodiment is illustrative in all respects and is not restrictive. The scope of the present invention includes all modifications within the scope of equivalents to the configurations described in the claims. For example, while the above-described embodiment illustrates a case in which the roadside camera 2 and the edge computer (information processing device) 3 are separate devices, both devices may be housed in a single housing to form an integrated information processing device.

[0107] REFERENCE SIGNS LIST 1 Information provision system 2 Roadside camera 3 Edge computer (information processing device) 4 Terminal device (on-board terminal) 5 Terminal device (user terminal) 6 Traffic light 7 Vehicle 8 Pedestrian 9 Wireless base station 10 Public communication network 31 Control unit 32 Memory unit 33 Communication unit (acquisition unit, distribution unit) 34 Synchronization processing unit 35 Prohibition database 36 Time table database 37 Transition rule database 38 Computer program 41 Data input unit 42 Area setting unit 43 Traffic light extraction unit 44 Traffic light color determination unit 45 Traffic light color check unit 46 Information generation unit 47 Sensor information input unit 48 Behavior determination unit J Intersection S1 Traffic light information P1, P2, P3 Center point Pa, Pb, Pc, Pd Vertex

Claims

1. a communication unit that receives image data of a scene including a traffic light at an intersection; a control unit that generates signal information representing a lighting color state of the signal lamp device based on the image data and causes the signal information to be transmitted to the communication unit, The image data is An information processing device, in which the image data is generated by a roadside camera installed so as to include the traffic light in its angle of view.

2. The communication unit capable of transmitting a communication packet including the signaling information; The communication packet The information processing device according to claim 1 , further comprising additional information relating to a method for generating the signal information.

3. The control unit 2. The information processing device according to claim 1, wherein an extraction range of the signal light device is determined as a pixel range including a reference point set by a user in the image data, and the current light color of the signal light device is determined based on pixel data within the extraction range.

4. The control unit generating the signal information using a signal light color that has passed a predetermined inspection among the determined current light colors of the signal light device; The predetermined test is The information processing device according to claim 3 , further comprising a time consistency check for checking the time consistency of the operation of the signal lamp device.

5. The time consistency check 5. The information processing device according to claim 4, further comprising a first check for checking whether a change in the light color of said signal lamp corresponds to a prohibited operation.

6. The time consistency check 5. An information processing device according to claim 4, further comprising a second check for checking whether the change in light color of said signal lamp unit is simultaneous with a pre-calculated estimated time table.

7. The control unit generating the signal information using a light color that has passed a predetermined inspection among the determined current light colors of the signal lamp device; The predetermined test is The information processing device according to claim 3 , further comprising a spatial consistency check that checks the spatial consistency of the operation of the signal lamp device.

8. The spatial consistency check 8. The information processing device according to claim 7, further comprising a third check for checking the synchronicity of changes in light color of a pair of said signal lamp devices that are oriented in directions opposite to each other.

9. The spatial consistency check 8. The information processing device according to claim 7, further comprising a fourth check for checking whether or not the changes in light color of a pair of signal lamps oriented in directions intersecting each other are in a forbidden relationship.

10. The signal lamp is Includes vehicle and pedestrian lights, The spatial consistency check 8. The information processing apparatus according to claim 7, further comprising a fifth check for checking whether a relationship between a change in the light color of the vehicular lamp and a change in the light color of the pedestrian lamp violates a transition rule.

11. The spatial consistency check 8. The information processing device according to claim 7, further comprising a sixth check for checking whether the change in light color of the signal lamp is consistent with the behavior of a vehicle that is located upstream of and closest to the stop line in the direction in which the signal lamp gives the right of way.

12. The communication unit capable of transmitting a communication packet including the signaling information; The communication packet The information processing device according to claim 1 , further comprising identification information indicating whether or not the signal information has been corrected and the type of the correction.

13. The control unit 5. The information processing device according to claim 4, wherein if the time consistency check determines that there is no consistency and if a spatial consistency check that checks the spatial consistency of the operation of the signal light is impossible, the light color state of the signal light is determined to be indefinite.

14. a communication unit that receives data indicating a change in the color of a traffic light at an intersection; a control unit that generates signal information representing a lighting color state of the signal lamp device based on the data and transmits the signal information to the communication unit, the communication unit is capable of transmitting a communication packet including the signal information, The information processing device, wherein the communication packet includes additional information related to a generation method of the signal information.

15. a communication unit for receiving image data including a scene of a traffic light at an intersection; a control unit that generates signal information representing a lighting color state of the signal lamp device based on the image data and transmits the signal information to the communication unit, The image data is A computer program that is image data generated by a roadside camera installed so that the traffic light is included in its angle of view.

16. The steps executed by the information processing device include: receiving image data included in a scene by a traffic light at an intersection; generating signal information representing a lighting color state of the signal lamp device based on the image data; and transmitting the signal information to an external device, The image data is An information processing method, in which the image data is generated by a roadside camera installed so that the traffic light is included in its angle of view.