Detection system, detection device, and detection device installation method

Abstract

This detection system comprises: a first sensor that is installed above a first area adjacent to a pedestrian crossing such that a first detection area in which the first sensor detects objects includes at least part of the first area; and a second sensor that has detection properties differing from those of the first sensor and is installed above the first area and such that a second detection area in which the second sensor detects objects includes at least part of pedestrian crossing.

Description

Detection system, detection device, and detection device installation method

[0001] This application claims priority to Japanese Patent Application No. 2022-064297, filed April 8, 2022, and incorporates by reference all of the contents of that application.

[0002] To grasp road traffic conditions, sensors that detect objects on the road (vehicles, pedestrians, etc.) are installed on the roadside. Conventionally, various types of sensors, such as image sensors and millimeter-wave sensors, have been known as roadside sensors. Recently, it has been proposed to use composite sensors, each consisting of multiple sensors with different detection characteristics, as roadside sensors. The composite sensor includes, for example, an image sensor and a millimeter-wave sensor. This composite sensor enhances the detection accuracy of objects on the road by integrating sensor data from the image sensor and the millimeter-wave sensor.

[0003] A road sensing system using such a composite sensor is disclosed in Patent Document 1 (hereinafter referred to as Patent Document 1). The road sensing system of Patent Document 1 detects objects on a road using multiple sensors. The multiple sensors include a radar sensor having a first field of view and a camera (image sensor) having a second field of view. The multiple sensors have a common field of view in which the first field of view and the second field of view at least partially overlap. In the common field of view, sensor data from the radar sensor and sensor data from the camera are integrated.

[0004] The road sensing system in Patent Document 1 is configured to be able to detect pedestrians crossing a crosswalk. Each sensor is installed in a position overlooking the target crosswalk diagonally downward (at a distance from the target crosswalk), and is set so that the common field of view, which is the overlapping portion of the first field of view and the second field of view, is located at the crosswalk. When the road sensing system detects a pedestrian crossing the crosswalk in the common field of view, it notifies vehicles approaching the crosswalk of the detection result.

[0005] US Patent Application Publication No. 2018 / 0350231

[0006] A detection system according to an aspect of the present disclosure includes a first sensor installed above a first area adjacent to a crosswalk, where a first detection target area in which the first sensor detects an object includes at least a portion of the first area, and a second sensor installed above the first area, where a second detection target area in which the second sensor detects an object includes at least a portion of the crosswalk, and where the second sensor has detection characteristics different from those of the first sensor.

[0007] A detection device according to another aspect of the present disclosure includes a first sensor, a second sensor having detection characteristics different from those of the first sensor, and a housing that unitizes the first sensor and the second sensor, the housing being mounted on a support member provided in a first area adjacent to a crosswalk, and with the housing mounted on the support member, a first detection target area in which the first sensor detects an object includes at least a portion of the first area, and a second detection target area in which the second sensor detects an object includes at least a portion of the crosswalk.

[0008] a detection device installation method according to yet another aspect of the present disclosure, comprising: installing a detection device on a support member provided above a first area adjacent to a crosswalk, the detection device including: an image sensor having a detection target area that is at least a portion of the first area; a millimeter-wave sensor having the crosswalk as its detection target area; a housing that unites the image sensor and the millimeter-wave sensor; an installation angle adjustment device that adjusts the installation angle of the housing with respect to the support member; and an imaging direction adjustment device that adjusts the imaging direction of the image sensor, the installation method including the steps of: attaching the housing to the support member; aligning the imaging direction of the image sensor with the detection direction of the millimeter-wave sensor by having an worker operate the installation angle adjustment device; adjusting the installation angle of the housing by having the worker operate the installation angle adjustment device based on image information from the image sensor so that the detection target area of ​​the millimeter-wave sensor includes at least a portion of the crosswalk; and adjusting the imaging direction of the image sensor by having the worker operate the imaging direction adjustment device based on image information from the image sensor so that the detection target area of ​​the image sensor includes at least a portion of the first area.

[0009] FIG. 1 is a diagram illustrating the overall configuration of a detection system according to a first embodiment. FIG. 2 is a plan view illustrating the detection system shown in FIG. 1. FIG. 3 is a diagram illustrating the detection system shown in FIG. 1. FIG. 4 is a diagram illustrating the detection system shown in FIG. 1. FIG. 5 is a block diagram illustrating an example configuration of the detection system shown in FIG. 1. FIG. 6 is a block diagram illustrating an example configuration of an integrated processing unit. FIG. 7 is a flowchart illustrating an example control structure of a program executed in the detection system shown in FIG. 1. FIG. 8 is a detailed flow diagram of step S1000 in FIG. 7. FIG. 9 is a detailed flow diagram of step S1020 in FIG. 7. FIG. 10 is a diagram illustrating an example operation of the detection system shown in FIG. 1. FIG. 11 is a diagram illustrating an example operation of the detection system shown in FIG. 1. FIG. 12 is a diagram illustrating a detection system of a comparative example. FIG. 13 is a plan view illustrating a detection system according to a first modification. FIG. 14 is a plan view illustrating a detection system according to a second modification. FIG. 15 is a block diagram illustrating an example configuration of a detection system according to a second embodiment. FIG. 16 is a diagram illustrating the detection system shown in FIG. 15. 17 is a flowchart showing an example of a control structure of a program executed in the detection system shown in FIG. 15 . FIG. 18 is a perspective view showing an example of the configuration of a detection system (detection device) according to a third embodiment. FIG. 19 is a side view showing an example of the configuration of the detection system (detection device) shown in FIG. 18 . FIG. 20 is a block diagram showing an example of the configuration of the detection system (detection device) shown in FIG. 18 . FIG. 21 is a block diagram showing an example of the configuration of a detection device according to Modification 3. FIG. 22 is a block diagram showing an example of the configuration of a detection device according to Modification 4. FIG. 23 is a perspective view showing an example of the configuration of a detection system (detection device) according to a fourth embodiment. FIG. 24 is a side view showing an example of the configuration of the detection system (detection device) shown in FIG. 23 . FIG. 25 is a block diagram showing an example of the configuration of the detection system (detection device) shown in FIG. 23 . FIG. 26 is a block diagram showing an example of the configuration of an imaging direction adjustment device. FIG. 27 is a block diagram showing an example of the configuration of an installation angle adjustment device. FIG. 28 is a diagram showing an example of a terminal device communicating with the detection system (detection device) shown in FIG. 23 . FIG. 29 is a sequence diagram showing the steps of installing and adjusting the detection device shown in FIG. 23 .Fig. 30 is a diagram illustrating a detection system according to a fifth embodiment. Fig. 31 is a diagram illustrating the detection system shown in Fig. 30. Fig. 32 is a flowchart illustrating an example of a control structure of a program executed in the detection system shown in Fig. 30. Fig. 33 is a diagram illustrating another example configuration of the detection system. Fig. 34 is a diagram illustrating yet another example configuration of the detection system.

[0010] [Problem to be Solved by the Present Disclosure] In the road sensing system of Patent Document 1, as described above, each sensor is installed in a position overlooking the target crosswalk diagonally downward. However, there is not necessarily an existing support member (e.g., a pole) for supporting the sensor in such a position. Furthermore, if the detection area is to be expanded to include the pedestrian waiting area around the crosswalk (hereinafter simply referred to as the "waiting area"), the installation position of the sensor must be changed accordingly. In such a case, it becomes increasingly difficult to install each sensor on the existing support member. In such a case, new support members must be installed to install the sensors. Installing new support members incurs additional construction costs. This results in a large cost burden.

[0011] Pedestrian signal light poles are usually installed near waiting areas. If these signal light poles could be used to install sensors, the cost burden could be reduced. However, the road sensing system of Patent Document 1 restricts the installation location of each sensor because it detects pedestrians on a crosswalk within a common field of view. This makes it difficult to use existing signal light poles to install each sensor.

[0012] The present disclosure has been made to solve the above-mentioned problems, and one objective of the present disclosure is to provide a detection system and a detection device that have a high degree of freedom in sensor installation and are capable of detecting a wider area.

[0013] Another object of the present disclosure is to provide a method for installing a detection device that has a high degree of freedom in installation and is capable of detecting a wider area.

[0014] [Advantages of the Present Disclosure] According to the present disclosure, it is possible to provide a detection system and a detection device that have a high degree of freedom in sensor installation and are capable of detecting a wider area. Furthermore, according to the present disclosure, it is possible to provide an installation method for a detection device that has a high degree of freedom in installation and is capable of detecting a wider area.

[0015] [Summary of the embodiments of the present disclosure] Preferred embodiments of the present disclosure will be listed and described below. At least some of the embodiments described below may be combined in any combination.

[0016] (1) A detection system according to a first aspect of the present disclosure includes a first sensor installed above a first area adjacent to a crosswalk, where a first detection target area in which the first sensor detects an object includes at least a part of the first area, and a second sensor installed above the first area, where a second detection target area in which the second sensor detects an object includes at least a part of the crosswalk, where the second sensor has detection characteristics different from those of the first sensor.

[0017] The first sensor's detection target area is an area including at least a portion of the first area adjacent to the crosswalk. The second sensor's detection target area is at least a portion of the crosswalk. This allows the first sensor to be installed near the crosswalk, and the second sensor to be installed near the first sensor. Therefore, the first sensor and the second sensor can be installed near the crosswalk, which was previously difficult to install. In other words, the flexibility in installing each sensor can be increased. By increasing the flexibility in installing each sensor, existing support members can be easily used to install each sensor, thereby suppressing the increase in costs associated with installing new support members.

[0018] In addition, by setting different detection target areas for the first sensor and the second sensor, it is possible to detect objects in a wider area. Furthermore, by using the first sensor to detect objects in the first area and the second sensor with detection characteristics different from the first sensor to detect objects on the crosswalk, it is possible to use sensors suitable for detecting objects in each area as the first sensor and the second sensor. This prevents a decrease in object detection accuracy.

[0019] (2) In the above (1), at least a portion of the first detection target area may overlap with the second detection target area in the first area or the crosswalk. This makes it easier to provide information about an object detected by the first sensor to the second sensor. By receiving object information from the first sensor, the second sensor can recognize an object that has moved from the first area to the crosswalk and track the object at the crosswalk.

[0020] (3) In the above (2), the overlapping area where the first detection target area and the second detection target area overlap may be smaller than both the first detection target area excluding the overlapping area and the second detection target area excluding the overlapping area. This allows the area in which objects are detected to be easily expanded. Furthermore, the detection target area of ​​the second sensor may be larger than the detection target area of ​​the first sensor.

[0021] (4) In any of (1) to (3) above, the first sensor may include an image sensor that detects an object present in the first area, and the second sensor may include a millimeter-wave sensor that detects an object present in the crosswalk. This effectively prevents a decrease in object detection accuracy when the sensors detect objects in each area.

[0022] (5) In any one of (1) to (4) above, the detection system may further include a housing that unitizes the first sensor and the second sensor, thereby facilitating installation of multiple sensors.

[0023] (6) In the above (5), the housing may have a first surface and a second surface intersecting the first surface, the first sensor may be provided on the second surface of the housing, and the second sensor may be provided inside the housing at a position corresponding to the first surface. This allows the first and second sensors to be easily installed so that their detection target areas are the first area and the crosswalk, respectively.

[0024] (7) In the above (6), the millimeter wave sensor includes an antenna that irradiates a detection signal in the millimeter wave band and receives a wave of the irradiated detection signal reflected by an object, and the antenna may be configured to be arranged inside the housing and parallel to the first surface.

[0025] (8) In any one of (4) to (7) above, the detection system may further include an imaging direction adjustment device that adjusts the imaging direction of the image sensor. This makes it possible to easily adjust the imaging direction of the image sensor.

[0026] (9) In the above (1) to (8), the detection system may further include an illumination device that illuminates the first area. This can prevent a decrease in recognition accuracy of the image sensor due to a decrease in illuminance.

[0027] (10) In any of the above (1) to (9), the first sensor may be installed on a support member provided in the first area, and the second sensor may be installed on the support member on which the first sensor is installed. This makes it easier to use the existing support member by installing each sensor.

[0028] (11) In any of (5) to (9) above, the housing may further include an installation angle adjustment device that is installed on a support member provided in the first area and that adjusts the installation angle of the housing. This makes it easy to adjust the installation angle of the housing and, ultimately, the installation angles of the sensors.

[0029] (12) In any of (1) to (11) above, the detection system may further include a processing unit that identifies the type and orientation of an object present in the first area based on sensor data from the first sensor, and, depending on the identification result, causes the second sensor to track the identified object at the crosswalk. This enables tracking of the object in conjunction with object recognition in the first area.

[0030] (13) In the above (12), the processing unit may be configured to further execute a predetermined process depending on the result of identifying the object in the first area or the tracking status of the object at the crosswalk. This makes it possible to provide support for traffic safety by effectively utilizing information on the tracking status.

[0031] (14) In the above (13), the predetermined processing may include at least one of notifying a vehicle entering a crosswalk of the presence of an object, recording the trajectory of the object's movement on the crosswalk, and controlling pedestrian traffic lights installed on the crosswalk. This makes it easier to provide support for traffic safety.

[0032] (15) A detection device according to a second aspect of the present disclosure includes a first sensor, a second sensor having detection characteristics different from those of the first sensor, and a housing that unites the first sensor and the second sensor, the housing being installed on a support member provided in a first area adjacent to a crosswalk, and when the housing is installed on the support member, a first detection target area in which the first sensor detects an object includes at least a portion of the first area, and a second detection target area in which the second sensor detects an object includes at least a portion of the crosswalk.

[0033] By unitizing multiple sensors in a housing, the first sensor can be installed near the detection target area, and the second sensor can be installed near the first sensor. Therefore, the first sensor and the second sensor can be installed in locations close to crosswalks, where installation was previously difficult. In other words, the degree of freedom in installing each sensor can be increased. By increasing the degree of freedom in installing each sensor, it becomes easier to use existing support members for installing each sensor. As a result, the increase in costs associated with installing new support members can be suppressed.

[0034] The first sensor covers at least a portion of a first area adjacent to the crosswalk as its detection target area, and the second sensor covers at least a portion of the crosswalk as its detection target area. By using different areas as detection target areas for the first sensor and the second sensor, it becomes possible to detect objects in a wider area. Furthermore, by using the first sensor to detect objects in the first area and the second sensor, which has detection characteristics different from those of the first sensor, to detect objects on the crosswalk, it is possible to use sensors suitable for detecting objects in each area as the first sensor and the second sensor. This prevents a decrease in object detection accuracy.

[0035] (16) In the above (15), the detection device may further include an illumination device provided in the housing and configured to illuminate the first area, thereby suppressing a decrease in recognition accuracy of the first sensor due to a decrease in illuminance.

[0036] (17) In the above (15) or (16), the first sensor may include an image sensor that detects an object present in the first area, and the second sensor may include a millimeter wave sensor that detects an object present in the second area. This effectively prevents a decrease in object detection accuracy when the sensors detect objects in each area.

[0037] (18) A detection device installation method according to a third aspect of the present disclosure is an installation method for installing a detection device on a support member provided above a first area adjacent to a crosswalk, the detection device including an image sensor whose detection target area is at least a portion of the first area, a millimeter-wave sensor whose detection target area is the crosswalk, a housing that unites the image sensor and the millimeter-wave sensor, an installation angle adjustment device that adjusts the installation angle of the housing with respect to the support member, and an imaging direction adjustment device that adjusts the imaging direction of the image sensor, and the installation method includes the steps of attaching the housing to the support member, aligning the imaging direction of the image sensor with the detection direction of the millimeter-wave sensor by having an operator operate the installation angle adjustment device based on image information from the image sensor, adjusting the installation angle of the housing by having the operator operate the installation angle adjustment device based on image information from the image sensor so that the detection target area of ​​the millimeter-wave sensor includes at least a portion of the crosswalk, and adjusting the imaging direction of the image sensor so that the detection target area of ​​the image sensor includes at least a portion of the first area by having the operator operate the imaging direction adjustment device. This allows the detection area of ​​the millimeter wave sensor to be easily set to the crosswalk, and the detection area of ​​the image sensor to be easily set to include at least a part of the first area adjacent to the crosswalk, which allows for greater freedom in installation and makes it easier to install a detection device that can detect a wider area.

[0038] The present disclosure can be realized not only as a detection system, a detection device, and a detection device installation method including such characteristic configurations, but also as a program for causing a computer to execute the characteristic steps executed by the detection system or the detection device, or the characteristic steps executed in the detection device installation method, and a recording medium on which the program is recorded. Furthermore, the present disclosure can be realized as other systems or devices including the detection system or the detection device.

[0039] [Details of the embodiments of the present disclosure] Specific examples of a detection system, a detection device, and a method for installing a detection device according to embodiments of the present disclosure will be described below with reference to the drawings. In the following embodiments, an example of detecting an object at a crosswalk and a waiting area around the crosswalk will be described. Note that in the following embodiments, identical components are assigned the same reference numerals. Their functions and names are also identical. Therefore, detailed description thereof will not be repeated.

[0040] (First Embodiment) [Overall Configuration] Referring to FIG. 1 , a detection system 50 according to this embodiment is a system installed on a roadside. This detection system 50 detects objects (vehicles, pedestrians, etc.) on a road using roadside sensors. The detection system 50 includes a plurality of sensors as roadside sensors. The plurality of sensors include, for example, an image sensor 100 and a millimeter-wave sensor 200 that is a sensor having detection characteristics different from those of the image sensor 100. The image sensor 100 is typically a camera. The image sensor 100 may be either a monocular camera or a stereo camera. The millimeter-wave sensor 200 is a radar sensor that detects the distance, direction, and speed of an object using radio waves in the millimeter wave band.

[0041] In this embodiment, the image sensor 100 mainly detects objects in the waiting area 10 (e.g., people such as waiting person P). The millimeter-wave sensor 200 mainly detects objects in the crosswalk 20 (e.g., people such as pedestrians). The millimeter-wave sensor 200 also has a function of tracking detected objects. In this way, the image sensor 100 and the millimeter-wave sensor 200 detect objects in different areas. The image sensor 100 also has a function of recognizing objects in the waiting area 10. Here, the recognition of an object by the image sensor 100 refers to the image sensor 100 obtaining data indicating the type of object, as will be described later. However, the object recognized by the image sensor 100 is not limited to the type of object, and may also include the direction of a person, the presence of an object that has entered the crosswalk from the waiting area, or the presence of a pedestrian who has completely crossed the crosswalk. For example, the image sensor 100 uses a Kalman filter to track the movement of the person before stopping in the waiting area 10 in time series, thereby identifying the direction of movement of the person, and recognizes the identified direction of movement as the orientation of the person who has stopped in the waiting area 10. Furthermore, the image sensor 100 uses image analysis to identify the direction of the line of sight of the person who has stopped in the waiting area 10, or the direction that passes through the person's back and chest, and recognizes the identified direction as the orientation of the person who has stopped in the waiting area 10.

[0042] A pedestrian signal light pole 30 is provided near the waiting area 10. The image sensor 100 and the millimeter-wave sensor 200 are installed on the signal light pole 30 via an arm member 32. The arm member 32 is attached to the signal light pole 30 so as to be positioned above the waiting area 10. In other words, the image sensor 100 and the millimeter-wave sensor 200 are attached to the signal light pole 30, which is an existing support member.

[0043] Referring to FIG. 2 , a roadway extends in a direction intersecting the direction of travel of the crosswalk, and a sidewalk is located between the roadway and the crosswalk. The roadway includes the crosswalk, and the sidewalk includes a waiting area. Two straight lines extending along the roadway and curving along the way are the boundary lines between the roadway and the sidewalk. A waiting area 10 is located directly below the image sensor 100, and the image sensor 100 defines an area including the waiting area 10 (first area) as its detection target area 60. The waiting area is an area near the crosswalk where people wait until it is possible to cross the crosswalk. The detection target area 60 of the image sensor 100 is preferably set to include the entire waiting area 10. However, because the waiting area does not have a clear boundary, the detection target area 60 of the image sensor 100 only needs to be set to be able to detect the target waiting person. Therefore, the detection target area 60 of the image sensor 100 only needs to be set to include the waiting area 10. The size of the waiting area varies depending on factors such as the size of the crosswalk, and the detection target area 60 of the image sensor 100 is therefore changed as appropriate depending on factors such as the size of the crosswalk.

[0044] The area (second area) corresponding to the crosswalk 20 is adjacent to the waiting area 10. The millimeter wave sensor 200 sets the second area (crosswalk 20) ​​adjacent to the waiting area 10 as the detection target area 70. The detection target area 70 of the millimeter wave sensor 200 is preferably set to cover the entire crosswalk 20. However, the detection target area 70 of the millimeter wave sensor 200 may also be set to cover only a portion of the crosswalk 20. In this case, the remaining area of ​​the crosswalk 20 may be included in the detection target area of ​​a sensor other than the millimeter wave sensor 200 (for example, the image sensor 100).

[0045] An image sensor 100a similar to image sensor 100 is provided in the waiting area 12 of the crosswalk 20. A pedestrian signal light pole 40 is also provided near the waiting area 12. This image sensor 100a is attached to the signal light pole 40 via an arm member 42. The waiting area 12 is located directly below the installation position of image sensor 100a. Like image sensor 100, image sensor 100a detects objects in the waiting area 12 by setting an area including the waiting area 12 as the detection target area 62.

[0046] 3 and 4 , the image sensor 100 is attached so that its imaging direction faces the waiting area 10, i.e., so that the area substantially directly below it becomes the image capture target. The image sensor 100a provided above the waiting area 12 is attached in the same manner. Meanwhile, the millimeter-wave sensor 200 is attached so that it looks diagonally downward at the crosswalk 20. As shown in FIG. 4 , a signal lamp 34 is provided on the signal lamp pole 30 on which the image sensor 100 and the millimeter-wave sensor 200 are attached, and a signal lamp 44 is provided on the signal lamp pole 40 on which the image sensor 100a is attached.

[0047] 2 , at least a portion of the detection target area 60 of the image sensor 100 overlaps with the detection target area 70 of the millimeter-wave sensor 200 in the crosswalk 20 (second area). From the perspective of the millimeter-wave sensor 200, at least a portion of the detection target area 70 of the millimeter-wave sensor 200 overlaps with the detection target area 60 of the image sensor 100 in the waiting area 10 (first area). As a result, an overlapping area 64 is formed between the detection target area 60 of the image sensor 100 and the detection target area 70 of the millimeter-wave sensor 200. Similarly, in the image sensor 100a provided above the waiting area 12, an overlapping area 66 is formed between the detection target area 62 of the image sensor 100a and the detection target area 70 of the millimeter-wave sensor 200.

[0048] 2 to 4 , the detection target area 70 of the millimeter wave sensor 200 is larger than the detection target area 60 of the image sensor 100. An overlapping area 64 where the detection target area 70 of the millimeter wave sensor 200 and the detection target area 60 of the image sensor 100 overlap is smaller than both the detection target area 60 of the image sensor 100 excluding the overlapping area 64 and the detection target area 70 of the millimeter wave sensor 200 excluding the overlapping area 64. The detection target area 70 of the millimeter wave sensor 200 is larger than the detection target area 62 of the image sensor 100a. An overlapping area 66 where the detection target area 70 of the millimeter wave sensor 200 and the detection target area 62 of the image sensor 100a overlap is smaller than both the detection target area 62 of the image sensor 100a excluding the overlapping area 66 and the detection target area 70 of the millimeter wave sensor 200 excluding the overlapping area 66.

[0049] [Hardware Configuration] Referring to FIG. 5 , the detection system 50 includes an image sensor 100, a millimeter wave sensor 200, and an integrated processing unit 300. The image sensor 100 includes an image capturing unit 110 and an image processing unit 120. The image capturing unit 110 includes a lens, an image sensor, etc. The image processing unit 120 processes information captured by the image capturing unit 110. The millimeter wave sensor 200 includes an antenna 210 that transmits and receives radio waves, a transmitter 220 that irradiates a target object with millimeter wave band radio waves (detection signals) via the antenna 210, a receiver 230 that receives, via the antenna 210, reflected waves of the irradiated radio waves from the target object, and a processing circuit unit 240. The processing circuit unit 240 detects the distance, direction, and speed of an object by processing information transmitted and received by the transmitter 220 and the receiver 230. The image sensor 100 and the millimeter wave sensor 200 perform time synchronization by acquiring time information from, for example, an NTP (Network Timing Protocol) server (not shown).

[0050] The integration processing unit 300 executes various processes, such as integrating the sensor data from the image sensor 100 and the sensor data from the millimeter-wave sensor 200, causing the millimeter-wave sensor 200 to track an object, and controlling a traffic light device according to the object tracking status, etc. The integration processing unit 300 also has a function of communicating with the image sensor 100a provided above the waiting area 12. The image sensor 100a also obtains time information from, for example, an NTP server (not shown), thereby performing time synchronization with the image sensor 100 and the millimeter-wave sensor 200. Note that the detection system 50 may be configured to include the image sensor 100a provided above the waiting area 12.

[0051] 6 , the integrated processing unit 300 includes a calculation unit 302, a memory 304, and a communication unit 306. The calculation unit 302 includes a calculation element (processor) such as a CPU (Central Processing Unit) or an MPU (Microprocessing Unit). The memory 304 is, for example, a non-volatile storage device. Software (computer programs) executed by the calculation unit 302 is pre-stored in the memory 304. The communication unit 306 communicates information with each sensor, including the image sensor 100 and the millimeter wave sensor 200, and with devices such as traffic lights, via wired or wireless communication.

[0052] 7 to 9, the control structure of the computer program executed by the detection system 50 to detect pedestrians in the waiting area or at the crosswalk and assist them in crossing the crosswalk will be described. This program starts in response to a start command from, for example, an administrator.

[0053] 7 , this program includes step S1000 in which an image sensor recognizes an object in a waiting area, step S1010, which is executed after step S1000, in which the integrated processing unit transmits a switching instruction to a signal light to switch the signal light to green, step S1020, which is executed after step S1010, in which the integrated processing unit causes the millimeter wave sensor to track a pedestrian crossing the pedestrian crossing the crosswalk, step S1030, which is executed after step S1020, in which the integrated processing unit transmits a switching instruction to the signal light to switch the signal light to red, and step S1040, which is executed after step S1030, in which the program waits until a predetermined time has elapsed. When the predetermined time has elapsed in step S1040, control returns to step S1000.

[0054] Fig. 8 is a detailed flow of step S1000 in Fig. 7. Referring to Fig. 8, this routine includes step S1100, which detects an object in the waiting area by processing sensor data from the image sensor, and step S1110, which is executed after step S1100, which determines whether or not a person is present in the waiting area and branches the control flow depending on the determination result. If it is determined in step S1110 that no person is present in the waiting area, control returns to step S1100.

[0055] This routine further includes step S1120, which is executed if it is determined in step S1110 that a person is present in the waiting area, and which processes sensor data from the image sensor (image processing) to recognize the orientation of the detected person, and step S1130, which is executed after step S1120, and which determines whether the detected person has been facing the crosswalk for t seconds or more and branches the control flow depending on the determination result. In step S1130, it is determined whether the detected person has been facing the crosswalk for, for example, 5 to 15 seconds or more. If it is determined in step S1130 that the detected person has not been facing the crosswalk for t seconds or more, control returns to step S1120.

[0056] This routine further includes step S1140, which is executed when it is determined in step S1130 that the detected person has been facing the crosswalk for t seconds or more, and determines that there is a person wishing to cross the street, and ends this routine.

[0057] Fig. 9 is a detailed flow of step S1020 in Fig. 7. Referring to Fig. 9, this routine includes step S1200 in which the image sensor provides (sends) information about people detected in the waiting area (crossing person information) to the millimeter wave sensor, step S1210, which is executed after step S1200, in which the millimeter wave sensor is controlled to track pedestrians at the crosswalk, and step S1220 in which the tracked pedestrian information is notified to a vehicle entering the crosswalk. This routine further includes step S1230, which is executed after step S1220, in which the image sensor detects pedestrians who have completely crossed the crosswalk, and step S1240, which is executed after step S1230, in which it is determined whether all detected people have completely crossed the crosswalk and the control flow branches depending on the determination result. This routine further includes step S1250, which is executed if it is determined in step S1240 that all detected pedestrians have not yet crossed the crosswalk, to determine whether a predetermined time has elapsed since the light turned green and branch the control flow depending on the determination result. If it is determined in step S1250 that the predetermined time has not elapsed since the light turned green, control returns to step S1210.

[0058] This routine further includes step S1260, which is executed when it is determined in step S1240 that all detected pedestrians have crossed the crosswalk, or when it is determined in step S1250 that a predetermined time has passed since the light turned green, and which ends tracking of pedestrians by the millimeter wave sensor and ends this routine.

[0059] [Operation] The detection system 50 according to this embodiment operates as follows.

[0060] Referring to FIG. 10 , the image sensor 100 of the detection system 50 recognizes objects in the waiting area 10. Assume that person A and person B are present in the waiting area 10. The image sensor 100 detects person A and person B present in the waiting area 10 (YES in steps S1100 and S1110 in FIG. 8 ) and assigns labels (object IDs) to the detected people. The image sensor 100 recognizes the orientation of the detected people (step S1120) and, based on the recognition result, determines whether any person has been facing the crosswalk 20 for t seconds or more. Person B is facing away from the crosswalk 20, while person A is facing the crosswalk 20. If person A continues in this state for t seconds, the image sensor 100 determines that there is someone facing the crosswalk 20 for t seconds or more (YES in step S1130) and determines that there is a person wishing to cross (step S1140).

[0061] In response to the image sensor 100's determination that a person wishes to cross, the integrated processing unit 300 transmits an instruction to the signal lights 34 and 44 to switch to green (step S1010 in FIG. 7 ). When the signal lights 34 and 44 switch to green, person A begins crossing the crosswalk 20.

[0062] 11 , the image sensor 100 sends information about a pedestrian entering the crosswalk 20 (crossing person information: for example, a label assigned to a detected object, the point where the pedestrian enters the crosswalk, the direction, etc.) to the millimeter-wave sensor 200. Specifically, the integration processing unit 300 (see FIG. 5 ) integrates the sensor data from the image sensor 100 and the sensor data from the millimeter-wave sensor 200 in the overlap area 64, thereby providing the millimeter-wave sensor 200 with information about the pedestrian entering the crosswalk 20 (here, information about person A) (step S1200 in FIG. 9 ).

[0063] The millimeter-wave sensor 200 recognizes person A who has entered the crosswalk 20 based on the pedestrian information from the image sensor 100 and starts tracking the pedestrian (person A) (step S1210). The integration processing unit 300 notifies a vehicle entering the crosswalk 20 that a pedestrian is crossing the crosswalk 20 (step S1220). The image sensor 100a on the outgoing side of the crosswalk 20 detects an object in the waiting area 12. The image sensor 100a on the outgoing side receives information about the pedestrian exiting the crosswalk 20 (crosser information: for example, a label assigned to the detected object, the point where the object enters the crosswalk, the direction, etc.) from the millimeter-wave sensor 200. Specifically, the integration processing unit 300 integrates the sensor data from the image sensor 100a and the sensor data from the millimeter-wave sensor 200 in the overlap area 64 to provide information about the pedestrian exiting the crosswalk 20 (here, information about person A) to the image sensor 100a.

[0064] The image sensor 100a on the outgoing side recognizes pedestrians who have completed crossing the crosswalk 20 and transmits this information to the integrated processing unit 300. The integrated processing unit 300 detects pedestrians who have completed crossing the crosswalk 20 based on the information from the image sensor 100a on the outgoing side (step S1230) and determines whether all detected pedestrians have completed crossing the crosswalk 20. In the example shown in FIG. 11 , person A is the only pedestrian crossing the crosswalk 20, so when person A completes crossing the crosswalk 20, the integrated processing unit 300 determines that all detected pedestrians have completed crossing the crosswalk 20 (YES in step S1240). The millimeter wave sensor 200 ends tracking (step S1260), and the integrated processing unit 300 transmits an instruction to switch the light to red to the signal lamps 34 and 44 (step S1030 in FIG. 7 ).

[0065] On the other hand, if there are many people crossing the crosswalk 20, the green light state will be maintained for a long time if all detected people wait until they have finished crossing the crosswalk 20. Therefore, when a predetermined time has elapsed since the integrated processing unit 300 sent a command to switch to a green light to the signal lamps 34 and 44, the integrated processing unit 300 sends a command to switch to a red light to the signal lamps 34 and 44, regardless of whether all detected people have finished crossing the crosswalk 20 (step S1030 in FIG. 7 ).

[0066] Detection system 50 waits for a predetermined time while the light is red, and when the predetermined time has elapsed (YES in step S1040), repeats the above-described operations.

[0067] If the pedestrian crosses in the opposite direction to that described above, the roles of the image sensor 100 and the image sensor 100a are reversed.

[0068] [Advantages of the Present Embodiment] As is clear from the above description, the detection system 50 according to the present embodiment has the following advantages.

[0069] Referring to FIG. 12 , a composite sensor 1000 including an image sensor and a millimeter-wave sensor integrates sensor data from the image sensor and the millimeter-wave sensor to improve detection accuracy when detecting objects on the road. Therefore, the image sensor and the millimeter-wave sensor have the same detection target area. Because it is difficult for both the image sensor and the millimeter-wave sensor to achieve a wide vertical field of view, the composite sensor 1000 is installed so as to overlook the crosswalk 20 diagonally downward. If the composite sensor 1000 were installed on a signal light pole 30 near the waiting area 10, it would be difficult to detect objects in the waiting area 10. Therefore, it is necessary to install a support member 1010, such as a pole, behind the waiting area 10 and install the composite sensor 1000 there. In this case, additional construction costs for installing the support member 1010 are incurred, resulting in a significant financial burden.

[0070] Referring to FIG. 4 , the detection system 50 according to this embodiment allows the image sensor 100 and the millimeter-wave sensor 200 to detect different areas, thereby increasing the flexibility in sensor installation and enabling detection over a wider area. Specifically, the image sensor 100 sets the detection target area 60 to an area including the waiting area 10 adjacent to the crosswalk 20. The millimeter-wave sensor 200 sets at least a portion of the crosswalk 20 as the detection target area 70. This allows the image sensor 100 to be installed near the detection target area 60, and the millimeter-wave sensor 200 to be installed near the image sensor 100. Because this increases the flexibility in sensor installation, the existing support member, the signal light pole 30, can be used to install the sensors. This reduces the cost associated with installing new support members.

[0071] With each sensor installed on the signal light pole 30, the image sensor 100 covers the waiting area 10 adjacent to the crosswalk 20 as a detection target area 60, and the millimeter-wave sensor 200 covers the crosswalk 20 as a detection target area 70. By using different areas as detection target areas for the image sensor 100 and the millimeter-wave sensor 200, it becomes possible to detect objects in a wider area. Furthermore, by using the image sensor 100 to detect objects in the waiting area 10 and the millimeter-wave sensor 200, which has detection characteristics different from those of the image sensor 100, to detect objects on the crosswalk 20, it is possible to use sensors that are suitable for detecting objects in each area. This makes it possible to prevent a decrease in object detection accuracy.

[0072] In the detection system 50, at least a portion of the detection target area 60 of the image sensor 100 overlaps with the detection target area 70 of the millimeter wave sensor 200 in the crosswalk 20, and at least a portion of the detection target area 70 of the millimeter wave sensor 200 overlaps with the detection target area 60 of the image sensor 100 in the waiting area 10. This makes it easier to provide information about objects detected by the image sensor 100 to the millimeter wave sensor 200. By receiving object information from the image sensor 100, the millimeter wave sensor 200 can recognize an object that has moved from the waiting area 10 to the crosswalk 20, and can track the object on the crosswalk 20. Note that the above configuration also makes it easier to provide information about objects detected by the millimeter wave sensor 200 to the image sensor 100.

[0073] The area in which objects are detected can be easily expanded by making the overlapping area 64, where the detection target area 60 of the image sensor 100 and the detection target area 70 of the millimeter wave sensor 200 overlap, smaller than both the detection target area 60 of the image sensor 100 excluding the overlapping area 64 and the detection target area 70 of the millimeter wave sensor 200 excluding the overlapping area 64.

[0074] The image sensor 100 is installed facing downward to detect objects in the waiting area 10, allowing the image sensor 100 to focus on detecting and recognizing the area directly below. This allows a camera with low sensitivity to be used as the image sensor 100. Furthermore, since the image sensor 100 is installed with its lens facing downward, it is possible to prevent the lens from becoming dirty. With the lens of the image sensor 100 facing downward, it becomes easier to clean the lens using, for example, a long-handled mop.

[0075] The image sensor 100a provided in the waiting area 12 also has the same effect.

[0076] 13 , depending on the position of the signal light pole 30 or the surrounding conditions of the crosswalk 20, the waiting area 10 may not be located directly below the image sensor 100. In the first modification, the image sensor 100 is installed so that the area 80 directly below the image sensor 100 includes at least a portion of the waiting area 10. That is, the image sensor 100 sets the detection target area 82 to an area including at least a portion of the waiting area 10, for the waiting area 10 including at least a portion of the area 80 directly below the image sensor 100. The area 80 directly below the image sensor 100 can be an area that can be detected by the image sensor 100 when the image sensor 100 is pointed directly downward, for example. The detection target area 82 can be set to include at least a portion of the waiting area 10.

[0077] 14 , the detection system of Modification 2 differs from the above-described embodiment in that the detection target area 72 of the millimeter wave sensor 200 is wider. Specifically, the detection target area 72 of the millimeter wave sensor 200 includes not only the crosswalk 20 but also at least a portion of the waiting area 12. That is, the millimeter wave sensor 200 detects objects in the waiting area 12 in addition to objects in the crosswalk 20. This makes it possible to omit an image sensor that detects objects in the waiting area 12. Note that FIG. 14 shows an example in which the detection target area 84 of the image sensor 100 is set to include a portion of the waiting area 10.

[0078] 15, a detection system 50A according to the present embodiment differs from the first embodiment in that it includes an illumination device 320 that illuminates the waiting area, and in that it includes an image sensor 150 instead of image sensor 100 (see FIG. 5). The other configurations are the same as those of the first embodiment.

[0079] The lighting device 320 includes a white LED (Light Emitting Diode) that emits visible light. The image sensor 150 also functions as an illuminance sensor. The image sensor 150 detects objects in the waiting area 10 and measures the illuminance of the waiting area 10 based on the captured image data. The image sensor 150 outputs an ON signal to the lighting device 320 when the illuminance of the waiting area 10 falls below a reference value, and outputs an OFF signal to the lighting device 320 when the illuminance of the waiting area 10 exceeds the reference value. The reference value can be, for example, an illuminance at which the detection performance of the image sensor 150 deteriorates due to insufficient light.

[0080] 16 , an image sensor 150a is installed on a signal lamp pole 40 provided in the waiting area 12, and a lighting device that illuminates the waiting area 12 may be provided near the image sensor 150a. In this case, the image sensor 150a measures the illuminance of the waiting area 12. Like the image sensor 150, the image sensor 150a outputs an ON signal to the lighting device when the illuminance of the waiting area 12 falls below a reference value, and outputs an OFF signal to the lighting device when the illuminance of the waiting area 12 exceeds the reference value.

[0081] Other configurations of the second embodiment are similar to those of the first embodiment.

[0082] [Software Configuration] The control structure of a computer program executed in the image sensor to control the lighting device that illuminates the waiting area will be described with reference to Fig. 17. This program starts in response to power-on of the image sensor.

[0083] This program includes step S2000, which measures the illuminance in the waiting area, and step S2010, which is executed after step S2000, to determine whether the illuminance in the waiting area is equal to or less than a reference value and to branch the control flow depending on the determination result. If it is determined in step S2010 that the illuminance in the waiting area is not equal to or less than the reference value, control returns to step S2000.

[0084] This program further includes step S2020, which is executed if it is determined in step S2010 that the illuminance in the waiting area is equal to or lower than the reference value, and which turns on lighting device 320 (see FIG. 15 ), step S2030, which is executed after step S2020, and which measures the illuminance in the waiting area, and step S2040, which is executed after step S2030, and which determines whether the illuminance in the waiting area has exceeded the reference value and branches the control flow depending on the determination result. If it is determined in step S2040 that the illuminance in the waiting area does not exceed the reference value, control returns to step S2030.

[0085] This program further includes step S2050, which is executed when it is determined in step S2040 that the illuminance in the waiting area exceeds the reference value, for turning off illumination device 320 and returning control to step S2000.

[0086] [Operation] The detection system 50A according to this embodiment operates as follows: Operations other than the operation of controlling the lighting device 320 are the same as those in the first embodiment.

[0087] 16, image sensor 150 measures the illuminance of waiting area 10 based on image information of waiting area 10. When the illuminance of waiting area 10 decreases to a reference value or less, such as at night (YES in step S2010 in FIG. 17), image sensor 150 outputs an ON signal to lighting device 320 (see FIG. 15) (step S2020). In response to the ON signal from image sensor 150, lighting device 320 supplies power to a white LED to illuminate waiting area 10.

[0088] Image sensor 150 measures the illuminance in waiting area 10 (step S2030), and when the illuminance in waiting area 10 exceeds the reference value (YES in step S2040), outputs an OFF signal to lighting device 320 (step S2050). In response to the OFF signal from image sensor 150, lighting device 320 stops supplying power to the white LEDs and ends irradiation of waiting area 10 with visible light.

[0089] An image sensor 150a provided above the waiting area 12 similarly measures the illuminance of the waiting area 12, and outputs an ON signal to the lighting device when the illuminance of the waiting area 12 falls below a reference value. In response to the ON signal from the image sensor 150a, the lighting device supplies power to a white LED to illuminate the waiting area 12. When the illuminance of the waiting area 12 exceeds the reference value, the image sensor 150a outputs an OFF signal to the lighting device. In response to the OFF signal from the image sensor 150a, the lighting device stops supplying power to the white LED and stops irradiating the waiting area 12 with visible light.

[0090] [Effect] The detection system 50A according to the present embodiment includes the lighting device 320 that illuminates the waiting area, thereby suppressing a decrease in the recognition accuracy of the image sensor 150 due to a decrease in illuminance. The same effect is achieved for the image sensor 150a.

[0091] Other effects of the second embodiment are the same as those of the first embodiment.

[0092] In the second embodiment, an example in which the image sensor has the function of an illuminance sensor is shown, but the present disclosure is not limited to such an embodiment. For example, a configuration in which an illuminance sensor is separately provided may be used. A configuration in which the lighting device includes an illuminance sensor so that the lighting device turns on and off autonomously according to the illuminance of the waiting area may be used. The lighting device may be separate from the image sensor or may be integrated with the image sensor.

[0093] 18 , a detection system 50B according to the present embodiment differs from the first and second embodiments in that the image sensor 100 and the millimeter-wave sensor 200 are unitized by a housing 330. That is, the detection system 50B according to the present embodiment integrates the image sensor 100 and the millimeter-wave sensor 200 by providing them in a single housing 330.

[0094] The detection device 400 is configured by unitizing the image sensor 100 and the millimeter-wave sensor 200 using a housing 330. That is, the detection system 50B includes the detection device 400 in which the image sensor 100 and the millimeter-wave sensor 200 are unitized using the housing 330. A first surface 332 and a second surface 334 disposed below the first surface 332 are provided on the front surface of the housing 330. The millimeter-wave sensor 200 is provided inside the housing 330 at a position corresponding to the first surface 332. The image sensor 100 is provided on the second surface 334 of the housing 330. Specifically, a radome 336, which is a cover for protecting the antenna 210 (see FIG. 15 ) of the millimeter-wave sensor 200, is provided on the first surface 332 of the housing 330. The millimeter-wave sensor 200 is provided inside the housing 330 at a position corresponding to the first surface 332. The antenna 210 is disposed inside the housing 330 so as to face the radome 336. That is, the antenna 210 is provided parallel to the first surface 332. However, the antenna 210 may be provided at a slight inclination with respect to the first surface 332. The image sensor 100 is attached to the second surface 334 of the housing 330.

[0095] 19 , the second surface 334 of the housing 330 intersects with the first surface 332 at a predetermined angle Y. The predetermined angle Y is an angle at which the detection area of ​​the image sensor 100 includes at least a part of the waiting area when the housing 330 is installed on a roadside support member (for example, a traffic light pole) so that the detection area of ​​the millimeter wave sensor 200 corresponds to a crosswalk.

[0096] 20 , the detection device 400 has a configuration in which the image sensor 100 and the millimeter wave sensor 200 are provided in a housing 330. The integrated processing unit is configured as a separate entity from the detection device 400, and the detection device 400 executes predetermined processing by communicating with the integrated processing unit.

[0097] In this way, the installation of multiple sensors can be facilitated by unitizing the image sensor 100 and the millimeter-wave sensor 200 using the housing 330. Furthermore, by providing the millimeter-wave sensor 200 at a position corresponding to the first surface 332 inside the housing 330 and providing the image sensor 100 on the second surface 334 that intersects with the first surface 332 at a predetermined angle Y, the installation of the image sensor 100 and the millimeter-wave sensor 200 can be facilitated.

[0098] Furthermore, by unitizing the image sensor 100 and the millimeter wave sensor 200, it becomes easier to synchronize the time between the image sensor 100 and the millimeter wave sensor 200 and to share information such as pedestrian information.

[0099] The other configurations and effects are the same as those of the first or second embodiment.

[0100] (Modification 3) Referring to FIG. 21, a detection device 400A according to Modification 3 differs from the third embodiment in that an integrated processing unit 300 is further included.

[0101] 22 , a detection device 400B according to a fourth modification differs from the third embodiment in that it further includes an illumination device 320 that illuminates the waiting area. The detection device 400B may further include an integrated processing unit, as in the third modification.

[0102] 23 and 24 , a detection system 50C according to the present embodiment differs from the third embodiment in that the imaging direction of the image sensor and the installation angle of the housing are adjustable. The other configurations are the same as those of the third embodiment.

[0103] The detection system 50C includes a detection device 500. The detection device 500 includes an image sensor 100, a millimeter-wave sensor 200, and a housing 340 that houses them. The housing 340 has a shape similar to that of the housing 330 (see FIGS. 18 and 19 ) described in the third embodiment. A first surface 332 and a second surface 334 disposed below the first surface 332 are provided on the front surface of the housing 340. A millimeter-wave sensor 200 is provided inside the housing 340 at a position corresponding to the first surface 332. An imaging direction adjustment device 520 for adjusting the imaging direction of the image sensor 100 is provided on the second surface 334 of the housing 340, and the image sensor 100 is connected to the imaging direction adjustment device 520. The second surface 334 of the housing 340 intersects with the first surface 332 at a predetermined angle Y (see FIG. 19 ).

[0104] The detection device 500 further includes an installation angle adjustment device 510 for adjusting the installation angle of the housing 340. The housing 340 is fixed to the arm member 32 by a fixing device 350. The installation angle adjustment device 510 is provided between the rear surface of the housing 340 and the fixing device 350.

[0105] 25 , the detection device 500 further includes a control unit 530 that controls the imaging direction adjustment device 520 and the installation angle adjustment device 510. The control unit 530 includes a communication unit 532 that communicates with the terminal device 600 wirelessly or via a wired connection (wireless in this case). The control unit 530 drives the imaging direction adjustment device 520 and the installation angle adjustment device 510 in response to a control command from the terminal device 600. Note that the detection device 500 may be configured to include an integrated processing unit, as shown in Modification 3. The detection device 500 may further include a lighting device that illuminates a waiting area, as shown in Modification 4.

[0106] 26 , the imaging direction adjustment device 520 includes a movable connecting part 512 for attaching the image sensor 100 to the housing 340, and a motor 514 for driving the movable connecting part 512. The motor 514 is, for example, a stepping motor, and drives the movable connecting part 512 in response to an instruction from the control part 530 (see FIG. 25 ). The imaging direction of the image sensor 100 changes as the movable connecting part 512 is driven by the motor 514.

[0107] 27 , installation angle adjustment device 510 includes a movable connecting part 522 provided between housing 340 and fixture 350 (see FIGS. 23 and 24 ), and a motor 524 for driving movable connecting part 522. Motor 524 is, for example, a stepping motor, and drives movable connecting part 522 in response to instructions from control unit 530 (see FIG. 25 ). The installation angle of housing 340 changes as motor 524 drives movable connecting part 522.

[0108] 28 , terminal device 600 is, for example, a smartphone. Terminal device 600 includes a touch panel display 610 that displays information and accepts user input via touch. Terminal device 600 is configured to allow adjustment of the installation angle of housing 340 and the imaging direction of image sensor 100. Touch panel display 610 includes an image display area 612 that displays an image captured by image sensor 100, and an operation area 614 that accepts operations to change the installation angle of housing 340 and the imaging direction of image sensor 100.

[0109] The detection device 500 may be configured to further include a device, such as a gyro sensor, that measures the orientation of the image sensor 100 and the orientation of the housing 340. In this case, the detection device 500 may transmit the orientation of the image sensor 100 and the orientation of the housing 340 measured by the device to the terminal device 600 via the control unit 530. This allows the user of the terminal device 600 to know the orientation of the image sensor 100 and the orientation of the housing 340 via the terminal device 600.

[0110] [Method of Installing and Adjusting the Detection Device 500] Fig. 29 is a sequence diagram showing steps by which an operator installs and adjusts the detection device 500. The method of installing and adjusting the detection device 500 will be described with reference to Fig. 29 .

[0111] An operator attaches the sensor housing (detection device 500) to the arm member 32 of the signal lamp pole (step S3000). The detection device 500 transmits the image (image information) captured by the image sensor 100 to the terminal device 600 (step S3010). The terminal device 600 displays the image captured by the image sensor 100 in the image display area 612 (see FIG. 28 ) (step S3020).

[0112] The worker operates the terminal device 600 to set the millimeter wave sensor 200 and the image sensor 100 in the same orientation (step S3030). The terminal device 600 accepts the worker's operation and transmits a corresponding control command to the detection device 500 (step S3040). The detection device 500 drives the imaging direction adjustment device 520 in response to the control command from the terminal device 600, and aligns the imaging direction of the image sensor 100 with the orientation (radiation direction of radio waves) of the millimeter wave sensor 200 (step S3050).

[0113] The worker operates the terminal device 600 to adjust the installation angle of the housing 340 while viewing the image displayed in the image display area 612 of the terminal device 600 (step S3060). The terminal device 600 accepts the worker's operation and transmits a corresponding control command to the detection device 500 (step S3070). The detection device 500 drives the installation angle adjustment device 510 in response to the control command from the terminal device 600 (step S3080) and changes the installation angle of the housing 340. This changes the radio wave irradiation area of ​​the millimeter wave sensor 200. The area imaged by the image sensor 100 generally coincides with the radio wave irradiation area of ​​the millimeter wave sensor 200. In other words, the image display area 612 displays the area in which the millimeter wave sensor 200 can detect objects. The worker, for example, walks around the perimeter of a crosswalk and adjusts the angle of the millimeter wave sensor 200 while checking that his or her image is displayed in the image display area 612 of the terminal device 600. If there are multiple workers, the other workers may walk on the outer periphery of the crosswalk, which allows the workers to visually align the detection area (radio wave irradiation area) of the millimeter wave sensor 200 with the crosswalk.

[0114] After completing the angle adjustment of the millimeter wave sensor 200, the worker operates the terminal device 600 to adjust the imaging direction (camera angle) of the image sensor 100 so that the target waiting area is displayed in the image display area 612 (step S3090). The terminal device 600 accepts the worker's operation and transmits a corresponding control command to the detection device 500 (step S3100). The detection device 500 drives the imaging direction adjustment device 520 in response to the control command from the terminal device 600 (step S3110) and changes the imaging direction (camera angle) of the image sensor 100. The worker determines the position where the target waiting area is displayed in the image display area 612 as the imaging direction of the image sensor 100. This completes the installation adjustment work of the detection device 500.

[0115] The terminal device 600 is not limited to a smartphone, but may be, for example, a tablet terminal, a personal computer (laptop PC), or the like.

[0116] [Effects] With the above-described configuration, the detection system 50C according to this embodiment can remotely adjust the detection target areas of the image sensor 100 and the millimeter wave sensor 200. The settings and adjustments for each detection target area can be performed on the ground. This allows the settings and adjustments for each detection target area to be easily performed. In addition, manual work at high altitudes can be reduced, which contributes to the safety of workers.

[0117] Furthermore, since the installation angle of the housing 340 and the imaging direction of the image sensor 100 can be adjusted, the detection system 50C can be applied to crosswalks of various sizes.

[0118] Other effects are the same as those of the first to third embodiments.

[0119] (Fifth embodiment) The detection system according to this embodiment differs from the above-described embodiments in the method for supporting pedestrians safely crossing a pedestrian crossing. The detection system according to this embodiment also differs from the above-described embodiments in that the image sensor and millimeter-wave sensor are installed on a signal light pole without an arm member. The other configurations are the same as those of the above-described embodiments.

[0120] With reference to FIG. 1 , the embodiment described above illustrates an example in which the image sensor 100 and the millimeter-wave sensor 200 are installed on the signal light pole 30 via the arm member 32. However, the present disclosure is not limited to such an embodiment. The image sensor 100 and the millimeter-wave sensor 200 may be installed on the signal light pole 30 without the arm member 32. With reference to FIG. 30 , in a detection system 50D, the image sensor 100 and the millimeter-wave sensor 200 are installed on a support arm 34a that supports, for example, a signal light 34. The image sensor 100a (see FIG. 4 ) may also be installed on a signal light pole in the same manner as the image sensor 100. With reference to FIG. 31 , the detection system 50D further performs a process of extending the green light depending on the crossing status of a pedestrian crossing the crosswalk 20. The detection system 50D, for example, uses the image sensors 100 and 100a to recognize (identify) the traffic-restricted person A1 waiting in the waiting area 10 and the waiting area 12, and tracks the traffic-restricted person A1 using the millimeter-wave sensor 200 when the traffic-restricted person A1 starts crossing the crosswalk 20. A traffic-restricted person refers to a person who is restricted in their travel behavior, such as someone who requires human assistance or equipment, or who has difficulty moving safely. Specific examples include people who require wheelchairs and visually impaired people who use white canes. The traffic-restricted person may be identified by recognizing a wheelchair, white cane, or the like. The detection system 50D determines whether the traffic-restricted person A1 can cross the crosswalk 20 within the green light, based on information such as the remaining time of the green light and the position and speed of the traffic-restricted person A1. If the detection system 50D determines that the traffic-restricted person A1 cannot cross the crosswalk 20 within the green light, it transmits an extension instruction to the signal lights 34 and 44 to extend the green light time.

[0121] The detection system 50D may be configured to calculate the time required for the traffic-restricted person A1 to cross the crosswalk 20 based on information such as the position and speed of the traffic-restricted person A1, and to send an instruction to extend the green light, specifying the extension time, to the signal lights 34 and 44.

[0122] The objects to be recognized in the waiting area 10 and the waiting area 12 are not limited to traffic restricted persons, but may also be people (pedestrians), bicycles, etc.

[0123] Referring to FIG. 32, the control structure of a computer program executed by detection system 50D will be described.

[0124] This program includes step S4000, in which the image sensors 100 and 100a recognize objects in the waiting areas 10 and 12, and step S4010, which is executed after step S4000, for communicating with the signal lamps 34 and 44 to determine whether the signal lamps 34 and 44 are showing green signals and branching the control flow depending on the determination result. If it is determined in step S4010 that the signal is not green, control returns to step S4000. In step S4000, objects (people, bicycles, traffic restriction persons, etc.) present in the waiting areas 10 and 12 are recognized.

[0125] This program further includes step S4020, which is executed when it is determined in step S4010 that the signal lights 34 and 44 are green, and which tracks the pedestrian using the millimeter wave sensor 200. This program also includes step S4030, which is executed after step S4020, and which determines whether the green light needs to be extended based on information such as the remaining time of the green light and the position and speed of the pedestrian, and which branches the control flow depending on the determination result. This program further includes step S4040, which is executed when it is determined in step S4030 that the green light needs to be extended, and which sends an instruction to the signal lights 34 and 44 to extend the green light and returns control to step S4020, and step S4050, which is executed when it is determined in step S4030 that the green light does not need to be extended, and which ends tracking of the pedestrian and returns control to step S4000.

[0126] The signal lamp units 34 and 44 that have received the extension instruction extend the green signal for a predetermined time. If an extension time is specified in the extension instruction, the signal lamp units 34 and 44 extend the green signal for the specified time.

[0127] The detection system 50D has the above-described configuration, and can effectively provide safety support for pedestrians, particularly for those subject to traffic restrictions.

[0128] Other safety support measures for pedestrians may also be implemented, such as notifying vehicles approaching a pedestrian crossing of the type of pedestrian and the presence of the pedestrian.

[0129] Such a detection system 50E will be described with reference to FIG. 33 . In the detection system 50E, the image sensor 100 and the image sensor 100a recognize objects in the waiting area 10 and the waiting area 12, respectively. The object recognition includes recognition of the object's type (pedestrian, bicycle, traffic-restricted person, etc.). The millimeter-wave sensor 200 tracks a pedestrian A2 crossing the crosswalk 20. Here, the pedestrian A2 is assumed to be a cyclist. The detection system 50E determines whether the pedestrian A2 is in danger based on the tracking information (type, location, speed, etc.) of the pedestrian A2 obtained by the millimeter-wave sensor 200. For example, the detection system 50E determines whether the pedestrian A2 is in danger based on whether the pedestrian A2's current position is in a blind spot of a right-turning vehicle V1 or a left-turning vehicle V2. The detection system 50E notifies right- or left-turning vehicles of the presence of the pedestrian A2 along with the type of the pedestrian A2 (bicycle).

[0130] Furthermore, in preparation for an accident occurring at a crosswalk, information detected by the image sensor and millimeter wave sensor may be stored in a storage device to help understand the situation of the accident.

[0131] Such a detection system 50F will be described with reference to Fig. 34. In the detection system 50F, the image sensor 100 and the image sensor 100a recognize objects in the waiting area 10 and the waiting area 12, respectively. The object recognition includes recognition of the object type (pedestrian, bicycle, traffic-restricted person, etc.). The millimeter-wave sensor 200 tracks a pedestrian A3 on the crosswalk 20. The detection data (recognition results, etc.) of the image sensor 100 and the image sensor 100a and the detection data (tracking results, etc.) of the millimeter-wave sensor 200 are stored as history information in, for example, the memory of the integrated processing unit.

[0132] When an accident occurs at the crosswalk 20, the detection system 50F accepts an operation from the administrator and calculates the trajectory T of the movement of the victim (crossing pedestrian A2) based on the recognition results of the image sensors 100 and 100a and the tracking results of the millimeter wave sensor 200, which are stored as historical information. When an accident occurs, the trajectory T of the movement of the victim can be used to understand the situation of the accident.

[0133] In the above embodiment, an example in which an image sensor and a millimeter wave sensor are combined has been described, but the present disclosure is not limited to such an embodiment. For example, a LiDAR (Light Detection and Ranging) sensor may be used instead of or together with the image sensor. Other combinations of sensors may also be used. When an image sensor is used, a function for automatically detecting lens dirt based on the detection results of the image sensor (camera) may be provided. In this case, it is preferable that the lens dirt be automatically removed using a wiper, air duster, or the like.

[0134] In the above embodiment, an example in which both the installation angle adjustment device and the imaging direction adjustment device are provided in the detection device has been shown, but the present disclosure is not limited to such an embodiment. The detection device may be configured to include either the installation angle adjustment device or the imaging direction adjustment device.

[0135] In the above embodiment, an example has been shown in which the installation angle of the housing of the detection device and the imaging direction of the image sensor can be remotely adjusted by an installation angle adjustment device and an imaging direction adjustment device, but the present disclosure is not limited to such an embodiment. The installation angle adjustment device and the imaging direction adjustment device may be manually adjustable devices such as ball joints.

[0136] In the above embodiment, an example has been shown in which the integrated processing unit is provided separately from the sensor (image sensor, millimeter wave sensor), but the present disclosure is not limited to such an embodiment. The integrated processing unit may be incorporated as a function of the sensor (image sensor, millimeter wave sensor, etc.).

[0137] In the above embodiment, an example in which multiple sensors (image sensor, millimeter-wave sensor) are mounted on the same support member has been described, but the present disclosure is not limited to such an embodiment. For example, the image sensor and the millimeter-wave sensor may be mounted on separate support members.

[0138] In the above embodiment, an example in which an image sensor performs object recognition processing has been described, but the present disclosure is not limited to such an embodiment. The object recognition processing may be performed by a device other than an image sensor, such as an integrated processing unit.

[0139] Each process (function) in the above-described embodiments may be implemented by a processing circuit including one or more processors. The processing circuit may be configured by an integrated circuit or the like that combines one or more memories, various analog circuits, and various digital circuits in addition to the one or more processors. The one or more memories store programs (instructions) that cause the one or more processors to execute the processes. The one or more processors may execute the processes according to the programs read from the one or more memories, or may execute the processes using logic circuits pre-designed to execute the processes. The processor may be a CPU, a graphics processing unit (GPU), a digital signal processor (DSP), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or any other processor suitable for computer control. The physically separated processors may cooperate with each other to execute the processes. For example, the processors installed in each of a plurality of physically separated computers may cooperate with each other via a network such as a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet to execute the above processes.

[0140] It should be noted that embodiments obtained by appropriately combining the techniques disclosed in the above embodiments are also included in the technical scope of the present disclosure.

[0141] The embodiments disclosed herein are merely examples, and the present disclosure is not limited to the above-described embodiments. The scope of the present disclosure is defined by the claims in the appended claims, taking into consideration the description of the detailed description of the invention, and includes all modifications within the meaning and scope equivalent to the wordings described therein.

[0142] DESCRIPTION OF SYMBOLS 10, 12 Waiting area 20 Pedestrian crossing 30, 40 Signal lamp pole 32, 42 Arm member 34, 44 Signal lamp 34a Support arm 50, 50A, 50B, 50C, 50D, 50E, 50F Detection system 60, 62, 70, 72, 82, 84 Detection target area 64, 66 Overlapping area 80 Area located directly below 100, 100a, 150, 150a Image sensor 110 Imaging unit 120 Image processing unit 200 Millimeter wave sensor 210 Antenna 220 Transmitting unit 230 Receiving unit 240 Processing circuit unit 300 Integrated processing unit 302 Arithmetic unit 304 Memory 306, 532 Communication unit 320 Lighting device 330, 340 Housing 332 First surface 334 Second surface 336 Radome 350 Fixture 400, 400A, 400B, 500 Detection device 510 Installation angle adjustment device 512, 522 Movable connecting portion 514, 524 Motor 520 Imaging direction adjustment device 530 Control unit 600 Terminal device 610 Touch panel display 612 Image display area 614 Operation area 1000 Composite sensor 1010 Support member A, B Person A1 Traffic restricted person A2, A3 Crossing person P Waiting person V1 Right-turning vehicle V2 Left-turning vehicle T Locus

Claims

1. A first sensor is installed above a first area adjacent to a crosswalk, and a first detection target area in which the first sensor detects an object includes at least a part of the first area; A second sensor is installed above the first area, a second detection target area in which the second sensor detects an object includes at least a part of the crosswalk, and the second sensor has detection characteristics different from those of the first sensor; A detection system comprising:

2. The detection system of claim 1 , wherein at least a portion of the first detection target area overlaps with the second detection target area in the first area or the crosswalk.

3. The detection system according to claim 2 , wherein an overlapping area where the first detection target area and the second detection target area overlap is smaller than both the first detection target area excluding the overlapping area and the second detection target area excluding the overlapping area.

4. the first sensor includes an image sensor that detects an object present in the first area; The detection system according to claim 1 , wherein the second sensor includes a millimeter wave sensor or a LiDAR sensor that detects an object present at the crosswalk.

5. 2. The thief detection system according to claim 1, further comprising a housing unitizing said first sensor and said second sensor.

6. The housing has a first surface and a second surface intersecting the first surface, the first sensor is provided on the second surface of the housing, The detection system according to claim 5 , wherein the second sensor is provided at a position inside the housing corresponding to the first surface.

7. The second sensor is a millimeter wave sensor that detects an object present on the crosswalk, The millimeter wave sensor includes an antenna that irradiates a detection signal in a millimeter wave band and receives a wave of the irradiated detection signal reflected by an object; The detection system according to claim 6 , wherein the antenna is provided inside the housing and parallel to the first surface.

8. The detection system according to claim 4 , further comprising an imaging direction adjustment device for adjusting an imaging direction of the image sensor.

9. The detection system of claim 1 , further comprising an illumination device for illuminating the first area.

10. The first sensor is installed on a support member provided in the first area, The detection system according to claim 1 , wherein the second sensor is mounted on the support member on which the first sensor is mounted.

11. The housing is installed on a support member provided in the first area, The detection system according to claim 5 , further comprising an installation angle adjustment device that adjusts an installation angle of the housing.

12. 4. The detection system according to claim 1, further comprising a processing unit that identifies a type and an orientation of an object present in the first area based on the sensor data from the first sensor, and, depending on a result of the identification, executes a process of causing the second sensor to track the identified object at the crosswalk.

13. The detection system according to claim 12 , wherein the processing unit further executes a predetermined process depending on a result of identifying the object in the first area or a tracking status of the object at the crosswalk.

14. 14. The detection system of claim 13, wherein the predetermined processing includes at least one of notifying a vehicle entering the crosswalk of the presence of the object, recording a trajectory of the object's movement across the crosswalk, and controlling a pedestrian traffic light installed at the crosswalk.

15. A first sensor; a second sensor having detection characteristics different from those of the first sensor; a housing that unitizes the first sensor and the second sensor, The housing is installed on a support member provided in a first area adjacent to a crosswalk, a detection device in which, when the housing is installed on the support member, a first detection target area in which the first sensor detects an object includes at least a portion of the first area, and a second detection target area in which the second sensor detects an object includes at least a portion of the crosswalk.

16. The detection device of claim 15 , further comprising an illumination device mounted on the housing to illuminate the first area.

17. the first sensor includes an image sensor that detects an object present in the first area; The detection device according to claim 15 or 16, wherein the second sensor includes a millimeter wave sensor that detects an object present at the crosswalk.

18. A method for installing a detection device on a support member provided above a first area adjacent to a crosswalk, comprising: the detection device includes an image sensor having at least a part of the first area as a detection target area, a millimeter wave sensor having the crosswalk as a detection target area, a housing that unitizes the image sensor and the millimeter wave sensor, an installation angle adjustment device that adjusts an installation angle of the housing with respect to the support member, and an imaging direction adjustment device that adjusts an imaging direction of the image sensor, The installation method includes: attaching the housing to the support member; an imaging direction adjustment device that is operated by an operator to adjust the imaging direction of the image sensor to a detection direction of the millimeter wave sensor; an installation angle of the housing is adjusted by an operator operating the installation angle adjustment device based on image information from the image sensor so that a detection area of ​​the millimeter wave sensor includes at least a portion of the crosswalk; and adjusting the imaging direction of the image sensor so that the detection area of ​​the image sensor includes at least a portion of the first area by an operator operating the imaging direction adjustment device based on image information from the image sensor.