Information processing device, information processing system, and information processing program

Abstract

To provide a system which makes it possible to expect more accurate optimum route setting if information on an on-route and a switching ratio of each lamp of surrounding traffic lights can be used in shortest route search and route selection during automatic driving so as to solve the problem in which, in a system that acquires information on switching of the conventional traffic light, since a visible camera device must be installed at each traffic light, it is not easy to realize such a system from a cost perspective.SOLUTION: An information processing device comprises a control unit which executes the steps of: specifying a traffic light from a first image captured by a visible camera device; specifying a traffic light from a second image captured by an infrared camera device based on a position of the traffic light in the first image; and calculating a lighting time of each lamp of the traffic light during a predetermined period based on brightness of the second image of the traffic light.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing system, and an information processing program that measure the lighting time and lighting rate of a traffic signal. [Background technology]

[0002] With the development of map databases and the widespread use of vehicle positioning technology using GPS (Global Positioning System), car navigation systems are now commonly used to search for the shortest route. In addition, the spread of autonomous driving technology is expected to increase with the application of image recognition using AI (Artificial Intelligence).

[0003] In such shortest route searches and route selection during autonomous driving, if information on the switching ratio of each light of traffic lights on or around the route could be used, it would be possible to set an optimal route with higher accuracy. Here, information on the switching ratio of each light of traffic lights is information on the ratio of the time each light of a traffic light is turned on (red, green, yellow, right turn, etc.) in a given period, such as 10 minutes.

[0004] In addition, on major roads, the switching rate of traffic lights is beginning to be dynamically managed for the purpose of alleviating congestion, but the development of a framework for providing that information to vehicles traveling on the road has lagged behind, making it necessary to actually measure the switching rate of traffic lights.

[0005] Therefore, as a method for actually measuring information related to traffic light switching, for example, there is a system in which a visible light camera device is installed to observe traffic lights, and the information is collected from images captured by the visible light camera device. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2013-186507 A [Patent Document 2] Patent Publication No. 2022-062263 [Patent Document 3] US Patent Application Publication No. 2012 / 0146811 Summary of the Invention [Problem to be solved by the invention]

[0007] However, this system requires the installation of a visible camera device at each traffic light, and therefore it is not easy to realize this system in terms of cost.

[0008] According to one aspect, an object of the present invention is to provide an information processing device, an information processing system, and an information processing program that can more easily measure traffic light illumination information. [Means for solving the problem]

[0009] In one embodiment, the information processing device includes a control unit that executes a process of identifying a traffic light from a first image captured by a visible camera device, identifying the traffic light from a second image captured by an infrared camera device based on the position of the traffic light in the first image, and calculating the lighting time of each lamp of the traffic light for a predetermined period of time based on the luminance of the traffic light in the second image. Effect of the Invention

[0010] On the one hand, it makes it easier to measure traffic light illumination information. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an information processing system 1 according to the present embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of the information processing device 10 according to the present embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of capturing an image of a traffic signal by the visible light camera device 30 and the infrared camera device 40 according to this embodiment. [Figure 4] FIG. 4 is a diagram showing an example of the relationship between brightness and temperature in an infrared image according to this embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of the relationship between the temperature and the lighting ratio of a traffic signal according to the present embodiment. [Figure 6] FIG. 6 is a flowchart showing the flow of the lighting information measurement process according to the present embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a hardware configuration of the information processing device 10. As shown in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Below, examples of the information processing device, information processing system, and information processing program according to the present embodiment will be described in detail with reference to the drawings. Note that the present embodiment is not limited to these examples. In addition, each example can be appropriately combined within a range that does not cause inconsistency.

[0013] (Overall composition) First, an information processing system for carrying out this embodiment will be described. Fig. 1 is a diagram showing a configuration example of an information processing system 1 according to this embodiment. As shown in Fig. 1, the information processing system 1 is a system in which an information processing device 10, a collection device 100, and another information processing device 200 are connected via a network 50 so as to be able to communicate with each other.

[0014] The network 50 may be any of a variety of communication networks, such as the Internet, whether wired or wireless. The network 50 may not be a single network, but may be configured, for example, by combining the Internet and an intranet via a network device such as a gateway or other device (not shown).

[0015] The information processing device 10 is, for example, an information processing device installed in a moving body such as an automobile. The information processing device 10 includes, for example, a visible camera device 30 and an infrared camera device 40, or is communicably connected to the visible camera device 30 and the infrared camera device 40.

[0016] The visible camera device 30 is, for example, an RGB camera device for capturing color images. The infrared camera device 40 is, for example, an infrared camera device (also called a thermal infrared camera device) for capturing infrared images. The visible camera device 30 and the infrared camera device 40 are controlled, for example, to capture the same field of view at the same timing. The images captured by the visible camera device 30 and the infrared camera device 40 may be, for example, a series of frames of video captured by the visible camera device 30 and the infrared camera device 40.

[0017] For example, the information processing device 10 identifies a traffic light from an image captured by the visible camera device 30, and identifies the traffic light from an image captured by the infrared camera device 40 based on the position of the traffic light in the visible image. In order to distinguish between the image captured by the visible camera device 30 and the image captured by the infrared camera device 40, hereinafter, they may be referred to as a "visible image" and an "infrared image", respectively.

[0018] The information processing device 10 also measures the illumination information of a traffic light by calculating the illumination time or illumination ratio of each lamp of the traffic light for a predetermined period, such as 10 minutes, based on the luminance in an infrared image of the traffic light. Note that each lamp of the traffic light is, for example, a red, green, yellow, right turn, or other lamp of the traffic light. The information processing device 10 also transmits, for example, the measured position information and illumination information of the traffic light to the collection device 100 or another information processing device 200. Note that, although only one information processing device 10 is shown in FIG. 1 for convenience, the information processing device 10 may be installed in various mobile objects and may measure illumination information of traffic lights at various locations. Therefore, the information processing device 10 can also transmit, for example, the measured position information and illumination information of the traffic light to information processing devices 10 other than itself.

[0019] The collection device 100 is, for example, an information processing device such as a server computer managed by an administrator of the information processing system 1. Alternatively, the collection device 100 may be a cloud computer device managed by a service provider that provides a cloud computing service. In addition, although the collection device 100 is shown in FIG. 1 as a single computer, it may be a distributed computing system configured with a plurality of computers.

[0020] The collecting device 100 receives and collects, for example, traffic light position information and lighting information from the information processing device 10 installed in each mobile object. Then, the collecting device 100 transmits, for example, the collected traffic light position information and lighting information to the information processing device 10 or another information processing device 200 for use in optimal route setting for each mobile object.

[0021] The other information processing device 200 is also an information processing device installed in a moving body such as an automobile. However, unlike the information processing device 10, the other information processing device 200 does not measure the lighting information of traffic lights, but only receives the position information and lighting information of traffic lights measured by the information processing device 10 and uses it for optimum route setting, etc. However, the other information processing device 200 can also transmit the received position information and lighting information of traffic lights to the information processing device 10, the collection device 100, and other information processing devices 200 other than itself.

[0022] In this way, the information processing device 10 and other information processing devices 200 can receive the position information and lighting information of each traffic light via the collection device 100, or in a bucket brigade manner by the information processing device 10 and other information processing devices 200. This allows the information processing device 10 and other information processing devices 200 to use the position information and lighting information of each traffic light for optimal route setting and the like.

[0023] (Functional configuration of information processing device 10) Next, a description will be given of the functional configuration of the information processing device 10. Fig. 2 is a diagram showing an example of the configuration of the information processing device 10 according to the present embodiment. As shown in Fig. 2, the information processing device 10 has a communication unit 11, a storage unit 12, and a control unit 20.

[0024] The communication unit 11 is a processing unit that controls communication with other devices such as the collection device 100, and is, for example, a communication interface such as a network interface card.

[0025] The storage unit 12 has a function of storing various data and programs executed by the control unit 20, and is realized by a storage device such as a memory or a hard disk. The storage unit 12 stores an imaging DB 13, a luminance temperature DB 14, a lighting DB 15, a map DB 16, a position DB, etc. DB is an abbreviation for database.

[0026] The imaging DB 13 stores, for example, a visible image captured by the visible camera device 30, an infrared image captured by the infrared camera device 40, and information related to these images. Fig. 3 is a diagram showing an example of imaging of a traffic light by the visible camera device 30 and the infrared camera device 40 according to this embodiment. As shown in Fig. 3, the visible camera device 30 and the infrared camera device 40 are controlled, for example, to capture the same field of view at the same timing, a visible image 31 and an infrared image 41 are captured, and both images are stored in the imaging DB 13 in association with each other.

[0027] Returning to the description of FIG. 2, the brightness temperature DB 14 stores, for example, information on the brightness of the infrared image 41 and the temperature of each lamp of the traffic light. FIG. 4 is a diagram showing an example of the relationship between brightness and temperature in an infrared image according to this embodiment. The infrared image 41-1 shown in FIG. 4 is a partial image obtained by cutting out a part of the traffic light from the infrared image 41 captured by the infrared camera device 40. In the infrared image 41-1, bright parts are shown with high brightness. Also, as shown in the lower part of FIG. 4, for example, by storing the relationship between the brightness and temperature of each lamp of the traffic light in advance in the brightness temperature DB 14, the temperature can be estimated from the brightness of each lamp, and it is possible to estimate whether each lamp is on or not based on the brightness of each lamp of the traffic light. For example, in the example of FIG. 4, if the temperature of the blue lamp reaches 31 degrees based on the brightness of the infrared image 41-1, the information processing device 10 can determine that the blue lamp is on. Also, for example, if the luminance of the infrared image 41-1 indicates that the red lamp is 38 degrees and the right turn lamp is 27 degrees, the information processing device 10 can determine that the right turn lamp (red for going straight) is on. Note that the temperatures of the lamps shown in FIG. 4 are merely examples, and may be higher or lower than the values ​​shown in FIG.

[0028] However, in reality, the temperature of a traffic light is affected by heat generated when the lamps are turned on, as well as by radiation from the sun. Therefore, in this embodiment, whether or not each lamp is turned on is determined based on the temperature difference between the lamps, using the temperature when the yellow light is turned on as a reference. Details of the lighting determination based on the temperature difference between the lamps will be described later.

[0029] Returning to the explanation of Fig. 2, the lighting DB 15 stores, for example, lighting information related to the measured lighting time and lighting ratio of each lamp of a traffic signal. The lighting time and lighting ratio stored in the lighting DB 15 may be the lighting time or lighting ratio of each lamp of a traffic signal for a predetermined period of time, such as 10 minutes.

[0030] The map DB 16 stores, for example, information about a map including the positions of roads and traffic lights. The map DB 16 may also store, for example, information about the current position and orientation of a moving body such as a vehicle equipped with the information processing device 10.

[0031] The position DB 17 stores, for example, information on the position of a traffic light whose lighting information was measured. The position of the traffic light can be estimated using an existing algorithm based on, for example, the position and orientation of a vehicle equipped with the information processing device 10 that performed the measurement, the distance from the vehicle to the traffic light in the visible image 31 captured by the visible camera device 30, and the like.

[0032] The above information stored in the storage unit 12 is merely an example, and the storage unit 12 can store various information other than the above information.

[0033] The control unit 20 is a processing unit, such as a processor, that controls the entire information processing device 10. The control unit 20 includes an acquisition unit 21, a specification unit 22, a measurement unit 23, and an output unit 24. Each processing unit is an example of an electronic circuit included in a processor or an example of a process executed by a processor.

[0034] The acquisition unit 21 acquires, for example, a visible image 31 captured by the visible camera device 30 and an infrared image 41 captured by the infrared camera device 40 from the imaging DB 13.

[0035] The identification unit 22 identifies a traffic light from a first image, which is, for example, a visible image 31 captured by the visible camera device 30. More specifically, the identification unit 22 inputs the visible image 31 into a machine learning model that has been trained in advance to detect traffic lights from an image using, for example, an existing object detection algorithm, and identifies the traffic light. Note that identifying the traffic light from the visible image 31 includes identifying each lamp of the traffic light from the visible image 31.

[0036] Furthermore, the identification unit 22 identifies a traffic light from the second image, which is the infrared image 41 captured by the infrared camera device 40, based on the position of the traffic light in the first image, for example. More specifically, the identification unit 22 identifies a traffic light in the infrared image 41 from the position of the corresponding traffic light in the visible image 31, since the visible camera device 30 and the infrared camera device 40 are controlled to capture the same field of view at the same timing, for example.

[0037] Furthermore, the identification unit 22 identifies the position on the map of the traffic light whose lighting information was measured as position information of the traffic light, based on, for example, map information stored in the map DB 16 and the identified position of the traffic light in the first image. Note that the position of the traffic light on the map may be further identified based on, for example, the position or orientation of the vehicle equipped with the information processing device 10 that performed the measurement, the distance from the vehicle to the traffic light in the visible image 31 captured by the visible camera device 30, and the like.

[0038] Furthermore, the identification unit 22 identifies the type of traffic light from the first image, for example. More specifically, the identification unit 22 inputs the visible image 31 into a machine learning model that has been trained in advance to detect the type of traffic light from an image, for example, using an existing object detection algorithm, and identifies the type of traffic light. By identifying the type of traffic light, for example, when estimating the luminance and temperature of the traffic light, and even the lighting information, from the infrared image 41, the estimation accuracy can be improved by switching the algorithm for each identified type. Note that the type of traffic light may not be identified strictly, but may be identified by a rough classification, such as a light bulb type or an LED (Light Emitting Diode) type.

[0039] The measurement unit 23 measures the lighting information of the traffic light by calculating the lighting time or lighting ratio of each lamp of the traffic light in a predetermined period, such as 10 minutes, based on the luminance in the second image of the traffic light. The lighting time in the predetermined period is, for example, the total time for each lamp indicating how long each lamp was on in the predetermined period. The lighting ratio in the predetermined period is, for example, the ratio for each lamp indicating how long each lamp was on in the predetermined period. The process of calculating the lighting time or lighting ratio based on the luminance in the second image of the traffic light may include, for example, a process of measuring the temperature of each lamp based on the luminance and calculating the lighting time or lighting ratio of each lamp based on the temperature. The process of calculating the lighting time or lighting ratio of each lamp based on the luminance and temperature of each lamp can be estimated based on the relationship between the luminance and temperature stored in the luminance temperature DB 14, as described with reference to FIG. 4.

[0040] Furthermore, the measurement unit 23 measures by calculating the lighting time or lighting ratio of each lamp of the traffic light in a predetermined period based on, for example, the temperature difference between each lamp of the traffic light. This is because the temperature of the traffic light is affected by heat generated when the lamp is turned on as well as radiation from the sun, and therefore it is not possible to accurately estimate whether each traffic light is turned on or not based only on the temperature of each lamp of the traffic light. The measurement of the lighting time or lighting ratio based on the temperature difference between each lamp will be described in more detail with reference to FIG. 5.

[0041] FIG. 5 is a diagram showing an example of the relationship between the temperature of a traffic light and the lighting ratio according to the present embodiment. FIG. 5 shows an example of estimating whether each lamp is on or not based on the temperature difference between the lamps, with the temperature when the yellow light is on as a reference (temperature difference 0). As shown in the upper part of FIG. 5, for example, by calculating the relationship between the temperature difference of each lamp of a traffic light and the lighting ratio in advance and storing it in the storage unit 12, it is possible to estimate the lighting ratio and lighting time of each lamp from the measured temperature difference of each lamp. Note that, as shown in the lower part of FIG. 5, the measured lighting time and lighting ratio of each lamp of a traffic light in a predetermined period may be stored in the lighting DB 15 in association with the measured temperature and temperature difference.

[0042] Furthermore, the measurement unit 23 measures by calculating the lighting time or lighting ratio of each lamp of the traffic light in a predetermined period based on, for example, the type of traffic light identified by the identification unit 22 and the luminance in the second image of the traffic light. For example, the luminance of each lamp and the temperature when lit may differ depending on the type of traffic light and the lighting method such as bulb type or LED type. Therefore, for example, by identifying the type and lighting method of the traffic light, the accuracy of estimation can be improved by switching the algorithm for estimating the luminance and temperature of the traffic light, and thus the lighting information, for each identified type and lighting method.

[0043] The output unit 24 transmits, for example, the position information of the traffic light identified by the identification unit 22 and the lighting information of the traffic light measured by the measurement unit 23 to another information processing device. Note that the other information processing device that is the destination of the transmission may be, for example, the collection device 100, another information processing device 200, or an information processing device 10 other than itself.

[0044] (Processing flow) Next, a description will be given of the flow of lighting information measurement processing executed by the information processing device 10. Fig. 6 is a flowchart showing the flow of lighting information measurement processing according to this embodiment.

[0045] 6, the information processing device 10 acquires, for example, a visible image 31 captured by the visible camera device 30 and an infrared image 41 captured by the infrared camera device 40 from the imaging DB 13 (step S101). The visible image 31 and the infrared image 41 are transmitted from the visible camera device 30 and the infrared camera device 40 to the information processing device 10 at any time and stored in the imaging DB 13.

[0046] Next, the information processing device 10 inputs the visible image 31 acquired in step S101 into a machine learning model that has been trained in advance to detect a traffic light from an image, and identifies the traffic light from the visible image 31 (step S102). Note that identifying the traffic light in step S102 may include identifying each lamp of the traffic light, or the type or lighting method of the traffic light, for example.

[0047] Next, the information processing device 10 measures the temperature of each lamp of the traffic light from the infrared image 41 acquired in step S101 using, for example, the position of the traffic light identified from the visible image 31 in step S102 (step S103). More specifically, for example, the information processing device 10 controls the visible image 31 and the infrared image 41 to be captured in the same field of view and at the same timing, and therefore identifies the traffic light in the infrared image 41 from the position of the corresponding traffic light in the visible image 31. Then, the information processing device 10 measures the temperature of each lamp of the traffic light using the relationship between brightness and temperature stored in the brightness temperature DB 14, based on, for example, the brightness of the traffic light in the infrared image 41.

[0048] Next, the information processing device 10 measures the lighting information of each lamp of the traffic light for a predetermined period, such as 10 minutes, from the temperature of each lamp of the traffic light measured in step S103 (step S104). The lighting information may be measured, for example, by calculating the lighting time or lighting ratio of each lamp, which indicates how long each lamp was on during the predetermined period. Furthermore, measuring the lighting information of each lamp from the temperature of each lamp may include measuring the lighting information of the traffic light by calculating the lighting time or lighting ratio of each lamp from the temperature difference between the lamps, for example.

[0049] Next, the information processing device 10 identifies the position on the map of the traffic light where the temperature and lighting information were measured as traffic light position information, for example, from the map information and the position of the traffic light identified from the visible image 31 in step S102 (step S105). Note that the map information used in step S105 may be, for example, map information including the positions of roads and traffic lights that is stored in advance in the map DB 16. Also, identifying the position of the traffic light on the map may include identifying the position of the traffic light based on, for example, the position or orientation of a moving object such as a vehicle equipped with the information processing device 10 that performed the measurement, or the distance from the moving object to the traffic light in the visible image 31.

[0050] Next, the information processing device 10 outputs, for example, the position information of the traffic light identified in step S105 and the lighting information of each lamp of the traffic light measured in step S104 to another information processing device (step S106). Note that the other information processing device that is the output destination may be, for example, the collection device 100, another information processing device 200, or an information processing device 10 other than itself. After execution of step S106, the lighting information measurement process shown in Fig. 6 ends, but the other information processing device that is the output destination may use, for example, the position information of the traffic light and the lighting information of each lamp output in step S106 for optimal route setting or the like.

[0051] (effect) As described above, the information processing device 10 identifies a traffic light from a first image captured by the visible camera device 30, identifies the traffic light from a second image captured by the infrared camera device 40 based on the position of the traffic light in the first image, and calculates the lighting time of each lamp of the traffic light for a specified period of time based on the luminance of the traffic light in the second image.

[0052] In this way, the information processing device 10 calculates the lighting time of each lamp of the traffic light in a predetermined period based on the luminance of each lamp of the traffic light in the infrared image 41 identified based on the position of the traffic light in the visible image 31. This enables the information processing device 10 to more easily measure the lighting information of the traffic light.

[0053] Furthermore, the information processing device 10 calculates the lighting ratio of each lamp of the traffic light in a predetermined period based on the lighting time.

[0054] This allows the information processing device 10 to measure the lighting information of the traffic lights more easily.

[0055] In addition, the process of calculating the lighting time based on the luminance executed by the information processing device 10 includes a process of measuring the temperature of each lamp based on the luminance, and calculating the lighting time of each lamp in a predetermined period based on the temperature.

[0056] This allows the information processing device 10 to measure the lighting information of the traffic lights more easily.

[0057] Furthermore, the process of calculating the lighting time based on the temperature, which is executed by the information processing device 10, includes a process of calculating the lighting time based on the temperature difference between the lamps.

[0058] This enables the information processing device 10 to measure the traffic light illumination information more easily and accurately.

[0059] In addition, the information processing device 10 identifies the position of the traffic light on the map based on the map information and the position of the traffic light in the first image, and transmits the position of the traffic light on the map and the lighting time or the lighting ratio to another information processing device.

[0060] This allows other information processing devices to use the traffic light position information and lighting information for optimal route planning, etc.

[0061] In addition, the information processing device 10 identifies the type of traffic light from the first image, and the process of calculating the lighting time based on the luminance executed by the information processing device 10 includes a process of calculating the lighting time based on the type and the luminance.

[0062] This enables the information processing device 10 to measure the traffic light illumination information more easily and accurately.

[0063] (system) The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings may be changed as desired unless otherwise specified. In addition, the specific examples, distributions, numerical values, etc. described in the embodiments are merely examples and may be changed as desired.

[0064] Furthermore, the specific form of distribution or integration of the components of each device is not limited to that shown in the figures. In other words, all or part of the components may be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions of each device may be realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0065] (Hardware) Fig. 7 is a diagram for explaining a hardware configuration example of the information processing device 10. The collection device 100 and other information processing devices 200 can also have a configuration similar to the hardware configuration example of the information processing device 10 shown in Fig. 7. As shown in Fig. 7, the information processing device 10 has a communication device 10a, a HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. Moreover, each unit shown in Fig. 7 is connected to each other by a bus or the like.

[0066] The communication device 10a is a network interface card or the like, and communicates with other information processing devices. The HDD 10b stores programs and DBs that operate the functions shown in FIG.

[0067] The processor 10d is a hardware circuit that reads out a program that executes the same processes as the processing units shown in FIG. 2 from the HDD 10b or the like and loads it in the memory 10c, thereby operating a process that executes each function described in FIG. 2 or the like. That is, this process executes a function similar to that of each processing unit possessed by the information processing device 10. Specifically, the processor 10d reads out a program having functions similar to those of the acquisition unit 21, the identification unit 22, the measurement unit 23, the output unit 24, or the like from the HDD 10b or the like. Then, the processor 10d executes a process that executes the same processes as those of the acquisition unit 21, the identification unit 22, the measurement unit 23, the output unit 24, or the like.

[0068] In this way, the information processing device 10 operates as an information processing device that executes operation control processing by reading and executing a program that executes the same processing as each processing unit shown in Fig. 2. The information processing device 10 can also realize functions similar to those of the above-mentioned embodiment by reading a program from a recording medium using a medium reading device and executing the read program. Note that the program in this other embodiment is not limited to being executed by the information processing device 10. For example, this embodiment may also be applied in the same way to a case where another information processing device executes a program, or a case where the information processing device 10 and the other information processing device cooperate to execute a program.

[0069] A program that executes the same processes as those of each processing unit shown in Fig. 2 can be distributed via a network such as the Internet. This program can be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and can be executed by being read out from the recording medium by a computer.

[0070] The following supplementary notes are further disclosed regarding the embodiments including the above examples.

[0071] (Appendix 1) Identifying a traffic light from a first image captured by the visible light camera device; Identifying the traffic light from a second image captured by an infrared camera device based on a position of the traffic light in the first image; Calculating the lighting time of each lamp of the traffic light in a predetermined period based on the luminance of the second image of the traffic light 1. An information processing device comprising: a control unit that executes processing.

[0072] (Supplementary Note 2) The information processing device according to Supplementary Note 1, characterized in that the control unit executes a process of calculating a lighting ratio of each lamp of the traffic light during the specified period based on the lighting time.

[0073] (Additional Note 3) The process of calculating the lighting time based on the luminance includes: measuring a temperature of each of the lamps based on the luminance; Calculating the lighting time of each of the lamps in the predetermined period based on the temperature. 2. The information processing device according to claim 1, further comprising:

[0074] (Additional Note 4) The process of calculating the lighting time based on the temperature includes: The lighting time is calculated based on the temperature difference between the lamps. 4. The information processing device according to claim 3, further comprising:

[0075] (Appendix 5) Identifying a position of the traffic light on a map based on map information and a position of the traffic light in the first image; The position of the traffic signal on the map and the lighting time or the lighting ratio are transmitted to another information processing device. 3. The information processing device according to claim 2, wherein the control unit executes the processing.

[0076] (Appendix 6) Identifying the type of the traffic light from the first image The control unit executes the process, The process of calculating the lighting time based on the luminance includes: The lighting time is calculated based on the model and the luminance. 2. The information processing device according to claim 1, further comprising:

[0077] (Appendix 7) Identifying a traffic light from a first image captured by the visible light camera device; Identifying the traffic light from a second image captured by an infrared camera device based on a position of the traffic light in the first image; calculating a lighting time of each lamp of the traffic light during a predetermined period based on a luminance of the second image of the traffic light; Transmitting the position of the traffic light on the map and the lighting time or the lighting ratio of each lamp of the traffic light during the predetermined period to another information processing device. A first information processing device that executes processing; receiving the position of the traffic signal on a map and the lighting time or the predetermined period transmitted from the first information processing device; A second information processing device that executes processing; An information processing system comprising:

[0078] (Supplementary Note 8) The information processing system according to Supplementary Note 7, wherein the first information processing device executes a process of calculating the lighting ratio based on the lighting time.

[0079] (Supplementary Note 9) The process of calculating the lighting time based on the luminance includes: measuring a temperature of each of the lamps based on the luminance; Calculating the lighting time of each of the lamps in the predetermined period based on the temperature. 8. The information processing system according to claim 7, further comprising:

[0080] (Supplementary Note 10) The process of calculating the lighting time based on the temperature includes: The lighting time is calculated based on the temperature difference between the lamps. 10. The information processing system according to claim 9, further comprising:

[0081] (Appendix 11) Identifying the position of the traffic light on the map based on map information and the position of the traffic light in the first image. 8. The information processing system according to claim 7, wherein the processing is executed by the first information processing device.

[0082] (Appendix 12) Identifying the type of the traffic light from the first image The first information processing device executes the process, The process of calculating the lighting time based on the luminance includes: The lighting time is calculated based on the model and the luminance. 8. The information processing system according to claim 7, further comprising: An information processing system.

[0083] (Appendix 13) Identifying a traffic light from a first image captured by the visible light camera device; Identifying the traffic light from a second image captured by an infrared camera device based on a position of the traffic light in the first image; Calculating the lighting time of each lamp of the traffic light in a predetermined period based on the luminance of the second image of the traffic light An information processing program that causes a computer to execute a process.

[0084] (Supplementary Note 14) The information processing program according to Supplementary Note 13, characterized in that the information processing program causes the computer to execute a process of calculating a lighting ratio of each lamp of the traffic light during the specified period based on the lighting time.

[0085] (Appendix 15) The process of measuring the lighting time or the lighting ratio based on the luminance includes: measuring a temperature of each of the lamps based on the luminance; Calculating the lighting time of each of the lamps in the predetermined period based on the temperature. 14. The information processing program according to claim 13, comprising:

[0086] (Appendix 16) The process of calculating the lighting time based on the temperature includes: The lighting time is calculated based on the temperature difference between the lamps. 16. The information processing program according to claim 15, further comprising:

[0087] (Appendix 17) Identifying a position of the traffic light on a map based on map information and a position of the traffic light in the first image; The position of the traffic signal on the map and the lighting time or the lighting ratio are transmitted to another information processing device. 15. The information processing program according to claim 14, which causes the computer to execute a process.

[0088] (Appendix 18) Identifying the type of the traffic light from the first image causing the computer to execute a process; The process of calculating the lighting time based on the luminance includes: The lighting time is calculated based on the model and the luminance. 14. The information processing program according to claim 13, comprising:

[0089] (Appendix 19) A processor; a memory operatively connected to the processor; An information processing device comprising: Identifying a traffic light from a first image captured by the visible light camera device; Identifying the traffic light from a second image captured by an infrared camera device based on a position of the traffic light in the first image; Calculating the lighting time of each lamp of the traffic light in a predetermined period based on the luminance of the second image of the traffic light 2. An information processing device that executes processing. [Explanation of symbols]

[0090] 1. Information Processing Systems 10. Information processing device 10a Communication equipment 10b HDD 10c Memory 10d Processor 11 Communications Department 12 Storage section 13 Imaging DB 14 Brightness Temperature DB 15 Lighting DB 16 Map DB 17 Location DB 20 Control section 21 Acquisition Department 22 Specific section 23 Measurement section 24 Output section 30 Visible Camera Equipment 31 Visible Images 40 Infrared camera equipment 41 Infrared Imaging 50 Network 100 Collector 200 Other information processing devices

Claims

1. Identifying a traffic light from a first image captured by the visible light camera device; Identifying the traffic light from a second image captured by an infrared camera device based on a position of the traffic light in the first image; Calculating the lighting time of each lamp of the traffic light in a predetermined period based on the luminance of the second image of the traffic light 1. An information processing device comprising: a control unit that executes processing.

2. 2 . The information processing device according to claim 1 , wherein the control unit executes a process of calculating a lighting rate of each lamp of the traffic light during the predetermined period based on the lighting time.

3. The process of calculating the lighting time based on the luminance includes: measuring a temperature of each of the lamps based on the luminance; Calculating the lighting time of each of the lamps in the predetermined period based on the temperature.

2. The information processing apparatus according to claim 1, further comprising a process for:

4. The process of calculating the lighting time based on the temperature includes: The lighting time is calculated based on the temperature difference between the lamps.

4. The information processing apparatus according to claim 3, further comprising a process for:

5. Identifying a position of the traffic light on a map based on map information and a position of the traffic light in the first image; The position on the map and the lighting time or the lighting ratio are transmitted to another information processing device.

3. The information processing apparatus according to claim 2, wherein the processing is executed by the control unit.

6. Identifying the type of the traffic light from the first image The control unit executes the process, The process of calculating the lighting time based on the luminance includes: The lighting time is calculated based on the model and the luminance.

2. The information processing apparatus according to claim 1, further comprising a process for:

7. Identifying a traffic light from a first image captured by the visible light camera device; Identifying the traffic light from a second image captured by an infrared camera device based on a position of the traffic light in the first image; calculating a lighting time of each lamp of the traffic light in a predetermined period based on a luminance of the second image of the traffic light; Transmitting the position of the traffic light on the map and the lighting time or the lighting ratio of each lamp of the traffic light during the predetermined period to another information processing device. A first information processing device that executes processing; receiving the position of the traffic signal on a map and the lighting time or the predetermined period transmitted from the first information processing device; A second information processing device that executes processing; An information processing system comprising:

8. Identifying a traffic light from a first image captured by the visible light camera device; Identifying the traffic light from a second image captured by an infrared camera device based on a position of the traffic light in the first image; Calculating the lighting time of each lamp of the traffic light in a predetermined period based on the luminance of the second image of the traffic light An information processing program that causes a computer to execute a process.

Images