Roadside device

Abstract

To improve a technique for detecting an object such as a pedestrian.SOLUTION: A roadside device 10 includes a photographing section 16, an output section 15, and a control section 11. The control section 11 performs: acquiring a reference image photographed by the photographing section 16 when the roadside device 10 is arranged in a reference posture; acquiring an image photographed by the photographing section 16 after acquiring the reference image; calculating a deviation amount between a posture of the roadside device 10 and the reference posture based on comparison between the reference image and the image; and outputting information which indicates a work procedure to allow the posture of the roadside device 10 to be close to the reference posture to a pedestrian via the output section 15 when the deviation amount is equal to or more than a threshold.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to roadside devices. [Background technology]

[0002] Conventionally, there are technologies for detecting objects such as pedestrians. For example, Patent Document 1 discloses that a roadside sensor equipped with a camera unit 11 detects bicycles or pedestrians on the side of the road or on the sidewalk. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-050629 Summary of the Invention [Problem to be solved by the invention]

[0004] There was room for improvement in the technology for detecting objects such as pedestrians. [Means for solving the problem]

[0005] The roadside device according to the present disclosure is a roadside device comprising an imaging unit, an output unit, and a control unit, wherein the control unit acquires a reference image captured by the imaging unit when the roadside device is installed in a reference posture, and after acquiring the reference image, further acquires an image captured by the imaging unit, and calculates the amount of deviation between the posture of the roadside device and the reference posture based on a comparison between the reference image and the image, and if the amount of deviation is greater than or equal to a threshold value, outputs to pedestrians via the output unit information indicating a procedure for bringing the posture of the roadside device closer to the reference posture. [Effects of the Invention]

[0006] According to the present disclosure, it is possible to improve the technology for detecting objects such as pedestrians. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 2 is a diagram illustrating a schematic configuration of a roadside device according to the present embodiment. [Figure 2] 4 is a flowchart showing the operation of the roadside device according to the present embodiment. [Figure 3] FIG. 2 is a diagram for explaining the positional relationship between a pedestrian and a roadside device. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or corresponding parts are designated by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0009] The configuration of the roadside device 10 according to this embodiment will be described with reference to FIG. 3. The roadside device 10 notifies a pedestrian P attempting to cross a roadway R of the presence of a vehicle, such as an autonomous vehicle, approaching the roadway R. The roadside device 10 may also notify the vehicle of the presence of the pedestrian P. The roadside device 10 is installed, for example, at a position where the pedestrian P will cross the roadway R. Specifically, this position includes a position within a predetermined range from the start position of the pedestrian crossing. The roadside device 10 may be installed near the boundary between the roadway R and the sidewalk. The roadside device 10 detects the pedestrian P based on video captured by a camera.

[0010] First, an outline of this embodiment will be described, and details will be described later. The roadside device 10 acquires a reference image taken when the roadside device 10 is installed in a reference attitude, acquires an image taken after the reference image is acquired, calculates the amount of deviation between the attitude of the roadside device 10 and the reference attitude based on a comparison between the reference image and the image, and if the amount of deviation is equal to or greater than a threshold, outputs to the pedestrian P information indicating a procedure for bringing the attitude of the roadside device 10 closer to the reference attitude.

[0011] For example, in object recognition technology based on images captured by a surveillance camera or the like, the camera's angle of view is often known. In FIG. 3 , the roadside device 10 detects a pedestrian P from the captured image using a reference posture, which is the posture when capturing the image in the direction indicated by the arrow. For example, an algorithm is used to detect a pedestrian P moving from left to right in the image as a pedestrian P attempting to cross the roadway R. If the posture of the roadside device 10 changes from the reference posture for some reason—for example, if the posture of the roadside device 10 changes from the direction indicated by the arrow in FIG. 3 to a posture capturing an image in a direction tilted 90 degrees horizontally relative to the ground—the algorithm cannot correctly detect the pedestrian P attempting to cross the roadway R. In contrast, according to this embodiment, the roadside device 10 automatically detects a change in posture based on a comparison between the reference image and the image, and can prompt the pedestrian P to correct the posture of the roadside device 10. This makes it easier for the roadside device 10 to maintain a desired posture, thereby improving the technology for detecting objects such as pedestrians.

[0012] The configuration of a roadside device 10 according to this embodiment will be described with reference to Fig. 1. The roadside device 10 includes a control unit 11, a storage unit 22, a communication unit 13, an input unit 14, an output unit 15, and an image capture unit 16.

[0013] The control unit 11 includes at least one processor, at least one programmable circuit, at least one dedicated circuit, or any combination thereof. The processor is a general-purpose processor such as a CPU or GPU, or a dedicated processor specialized for specific processing. "CPU" is an abbreviation for central processing unit. "GPU" is an abbreviation for graphics processing unit. An example of the programmable circuit is an FPGA. "FPGA" is an abbreviation for field-programmable gate array. An example of the dedicated circuit is an ASIC. "ASIC" is an abbreviation for application specific integrated circuit. The control unit 11 controls each part of the roadside device 10 and executes processing related to the operation of the roadside device 10.

[0014] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or any combination thereof. The semiconductor memory is, for example, a RAM, a ROM, or a flash memory. "RAM" is an abbreviation for random access memory. "ROM" is an abbreviation for read only memory. RAM is, for example, an SRAM or a DRAM. "SRAM" is an abbreviation for static random access memory. "DRAM" is an abbreviation for dynamic random access memory. ROM is, for example, an EEPROM. "EEPROM" is an abbreviation for electrically erasable programmable read only memory. Flash memory is, for example, an SSD. "SSD" is an abbreviation for solid-state drive. Magnetic memory is, for example, an HDD. "HDD" is an abbreviation for hard disk drive. The storage unit 12 functions, for example, as a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores information used in the operation of the roadside device 10 and information obtained by the operation of the roadside device 10.

[0015] The communication unit 13 includes at least one communication interface. The communication interface is, for example, an interface compatible with a mobile communication standard such as LTE, 4G standard, or 5G standard, an interface compatible with a short-range wireless communication standard such as Bluetooth (registered trademark), or a LAN interface. "LTE" is an abbreviation for Long Term Evolution. "4G" is an abbreviation for 4th generation. "5G" is an abbreviation for 5th generation. The communication unit 13 receives information used for the operation of the roadside device 10 via a network, and transmits information obtained by the operation of the roadside device 10.

[0016] The network may include the Internet, at least one WAN, at least one MAN, or any combination thereof. "WAN" is an abbreviation for wide area network. "MAN" is an abbreviation for metropolitan area network. The network may include at least one wireless network, at least one optical network, or any combination thereof. The wireless network may be, for example, an ad hoc network, a cellular network, a wireless LAN, a satellite communication network, or a terrestrial microwave network. "LAN" is an abbreviation for local area network.

[0017] The input unit 14 includes at least one input interface. The input interface is, for example, a physical key, a capacitance key, a pointing device, a touch screen integrated with a display, or a microphone. The input unit 14 accepts an operation to input data used for the operation of the roadside device 10. The input unit 14 may be connected to the roadside device 10 as an external input device instead of being provided in the roadside device 10. Any connection method can be used, for example, USB, HDMI (registered trademark), or Bluetooth (registered trademark). "USB" is an abbreviation for Universal Serial Bus. "HDMI (registered trademark)" is an abbreviation for High-Definition Multimedia Interface.

[0018] The output unit 15 includes at least one output interface. The output interface is, for example, a display or a speaker. The display is, for example, an LCD or an organic EL display. "LCD" is an abbreviation for liquid crystal display. "EL" is an abbreviation for electroluminescence. The display may be an LED display. "LED" is an abbreviation for light emitting diode. The output unit 15 outputs data obtained by the operation of the roadside device 10. The output unit 15 may be connected to the roadside device 10 as an external output device instead of being provided in the roadside device 10. Any connection method can be used, for example, USB, HDMI (registered trademark), or Bluetooth (registered trademark). The output unit 15 may include a light emitting element such as an LED light.

[0019] The photographing unit 16 is a camera that photographs the surroundings of the roadside device 10. The photographing unit 16 can output the photographed images and videos to the control unit 11.

[0020] The functions of the roadside device 10 are realized by executing a program according to this embodiment on a processor serving as the control unit 11. That is, the functions of the roadside device 10 are realized by software. The program causes a computer to execute the operations of the roadside device 10, thereby causing the computer to function as the roadside device 10. That is, the computer functions as the roadside device 10 by executing the operations of the roadside device 10 in accordance with the program.

[0021] The program can be stored on a non-transitory computer-readable medium. Examples of non-transitory computer-readable media include flash memory, magnetic recording devices, optical disks, magneto-optical recording media, and ROMs. The program can be distributed by selling, transferring, or lending portable media such as SD cards, DVDs, or CD-ROMs that store the program. "SD" is an abbreviation for Secure Digital. "DVD" is an abbreviation for digital versatile disc. "CD-ROM" is an abbreviation for compact disc read only memory. The program can also be distributed by storing it in the storage of a server and transferring it from the server to another computer. The program can also be provided as a program product.

[0022] A computer temporarily stores a program stored on a portable medium or transferred from a server in its main storage device. The computer then reads the program stored in the main storage device using a processor and executes processing in accordance with the read program. The computer may also read the program directly from a portable medium and execute processing in accordance with the program. The computer may also execute processing in accordance with the received program each time a program is transferred from a server to the computer. Processing may also be executed through a so-called ASP-type service that achieves its functions by issuing execution instructions and obtaining results without transferring the program from the server to the computer. "ASP" is an abbreviation for application service provider. A program is information used for processing by a computer and includes something equivalent to a program. For example, data that is not a direct instruction to a computer but has properties that specify computer processing falls under the category of "something equivalent to a program."

[0023] Some or all of the functions of the roadside device 10 may be realized by a programmable circuit or a dedicated circuit as the control unit 11. In other words, some or all of the functions of the roadside device 10 may be realized by hardware.

[0024] The operation of the roadside device 10 according to this embodiment will be described with reference to Figures 2 and 3. In the following, it is assumed that communication between the roadside device 10 and an external device is performed via the communication unit 13 and a network.

[0025] In the following, it is assumed that the control unit 11 of the roadside device 10 is constantly detecting pedestrian P based on the video captured by the image capturing unit 16. Specifically, the control unit 11 acquires the video captured by the image capturing unit 16 and analyzes the video using any image analysis technology. When the video captures pedestrian P and a vehicle, such as an autonomous vehicle, approaching the roadside device 10, the control unit 11 issues a message via the output unit 15, such as "An autonomous vehicle is approaching 20 meters to the right."

[0026] In S1 of FIG. 2, the control unit 11 of the roadside device 10 acquires a reference image captured by the image capture unit 16 when the roadside device 10 is installed in a reference orientation. The control unit 11 stores the acquired reference image in the memory unit 12. When the roadside device 10 is installed by a worker, the control unit 11 may acquire the reference image by receiving instructions from the worker via the input unit 14. The control unit 11 may acquire the reference image by communicating with, for example, an external server device that manages the roadside device 10 and receiving instructions from the server device. The reference orientation is, for example, the orientation when the image capture unit 16 of the roadside device 10 installed near a crosswalk faces a pedestrian P at the start position of the crosswalk in a direction perpendicular to the direction of the crosswalk. FIG. 3 is a diagram illustrating the positional relationship between the roadside device 10 installed near a crosswalk on the roadway R and the pedestrian P. The roadside device 10 faces the pedestrian P in the direction indicated by the arrow in FIG. 3, which is perpendicular to the direction of the crosswalk.

[0027] In S2 of FIG. 2, the control unit 11 acquires an image captured by the image capturing unit 16. The image may be one of multiple frames constituting a video captured by the image capturing unit 16 for detecting a pedestrian P. The control unit 11 may acquire an image periodically, such as every hour. The control unit 11 may acquire an image when a predetermined condition is met. The predetermined condition may include, for example, detecting the presence of a pedestrian P near the roadside device 10, receiving an instruction to acquire an image from an external server device, etc. Information indicating the predetermined condition may be stored in the storage unit 12 in advance.

[0028] In S3, the control unit 11 compares the reference image acquired in S1 with the image acquired in S2, and calculates the amount of deviation between the current attitude of the roadside device 10 and the reference attitude. Specifically, the control unit 11 first uses any image analysis technology to extract feature points from each of the reference image acquired in S1 and the image acquired in S2. The control unit 11 compares the positions of the feature points in each image, and calculates the distance of deviation between the positions as the amount of deviation between the current attitude of the roadside device 10 and the reference attitude.

[0029] In S4, the control unit 11 determines whether the amount of deviation calculated in S3 is equal to or greater than a threshold value. Information indicating the threshold value may be set in advance and stored in the storage unit 12. If the amount of deviation is equal to or greater than the threshold value, the process of the control unit 11 proceeds to S5. If the amount of deviation is less than the threshold value, the process of the control unit 11 returns to S2.

[0030] In S5, the control unit 11 outputs, to the pedestrian P, via the output unit 15, information indicating a procedure for adjusting the attitude of the roadside device 10 to approach the reference attitude. Specifically, the control unit 11 acquires the information by reading it from the storage unit 12. For example, if alignment marks are provided in advance on both the ground position where the roadside device 10 is installed and on the surface of the roadside device 10, the control unit 11 outputs a message such as "Please adjust your attitude so that the marks align" via the output unit 15. The control unit 11 may output the message by voice from a speaker serving as the output unit 15, or may output the message by displaying it on a display serving as the output unit 15. The control unit 11 may output information indicating the message by directly transmitting it to the terminal device of the pedestrian P. Thereafter, the processing of the control unit 11 returns to S2.

[0031] The control unit 11 may acquire information indicating the work procedure by generating it using any method. For example, the control unit 11 calculates a movement amount for correcting the posture of the roadside device 10 so that the positions of each feature point detected from the image acquired in S1 and the image acquired in S2 approximately match. The movement amount includes the movement amount of the roadside device 10 in the forward / backward direction, the movement amount in the left / right direction, the rotation amount around an axis, etc. The control unit 11 may calculate the movement amount by referring to information previously stored in the storage unit 12 that associates the distance between feature points in the images with the movement amount of the roadside device 10 in each direction. The control unit 11 outputs the information indicating the work procedure, including the calculated movement amount, via the output unit 15. For example, the control unit 11 may output a message such as "Please move 10 cm to the right and 10 cm forward to correct your posture" by displaying it on a display serving as the output unit 15.

[0032] The control unit 11 may superimpose the real-time image captured by the image capture unit 16 for detecting the pedestrian P on the reference image acquired in S1 and display the superimposed image on the display serving as the output unit 15 as information indicating the operation procedure. In this case, the control unit 11 may superimpose a semi-transparent reference image on the real-time image and display a message such as "Please move the roadside device so that the reference image and the real-time image match." This allows the pedestrian P attempting to correct the posture of the roadside device 10 to perform the operation while checking the display serving as the output unit 15. The control unit 11 may output the superimposed image by directly transmitting it to the terminal device of the pedestrian P. The control unit 11 may further extract feature points from the real-time image in real time using any image analysis technology and display the extracted feature points on the display serving as the output unit 15 together with the feature points of the reference image obtained by superimposing the feature points on the semi-transparent image. In this case, the control unit 11 may also display a message such as "Please move the roadside device so that the two feature points match." In this case, the control unit 11 may also display a message such as "The posture has been corrected" when the feature points match.

[0033] The present disclosure is not limited to the above-described embodiments. For example, two or more blocks shown in the block diagrams may be integrated, or one block may be divided. Two or more steps shown in the flowcharts may be executed in parallel or in a different order, instead of being executed in chronological order as described, depending on the processing capabilities of the device executing each step, or as needed. Other modifications are possible within the scope of the present disclosure.

[0034] As a modified example of the present disclosure, after determining in S4 that the amount of deviation is less than the threshold, the control unit 11 may further determine whether the amount of deviation is 0. If it is determined that the amount of deviation is 0, the operation of the control unit 11 may return to S2 described above. If it is determined that the amount of deviation is not 0, that is, if it is determined that the amount of deviation is greater than 0 and less than the threshold, the control unit 11 determines, based on the image acquired in S2, to correct the video subsequently captured by the image capturing unit 16 and use the corrected video for detecting the pedestrian P. The processing of the control unit 11 then returns to S2 described above.

[0035] For example, suppose that the feature point of the image acquired in S2 is shifted a predetermined distance to the right from the feature point of the reference image, i.e., the attitude of the roadside device 10 is shifted so that the image capturing unit 16 of the roadside device 10 faces leftward. In this case, the control unit 11 may determine to correct the image subsequently captured by the image capturing unit 16 by trimming the left edge by an amount corresponding to the predetermined distance. The correction method is not limited to this. For example, the control unit 11 may normally detect a pedestrian P using only the central region of the image captured by the image capturing unit 16, and when it determines that the amount of shift is greater than 0 and less than a threshold, it may determine to use an image including an area in a predetermined direction from the central region for subsequent detection of the pedestrian P. For example, when the control unit 11 determines that the feature point of the image acquired in S2 is shifted a predetermined distance to the right from the feature point of the reference image, it determines to use an image including an area in a rightward direction from the central region for subsequent detection of the pedestrian P. This allows the image capturing unit 16 to continue capturing images in which the positions of the feature points match those of the reference image without having to return the roadside device 10 to its original position.

[0036] As a modified example of the present disclosure, after outputting information indicating the work procedure in S5 described above, the control unit 11 may determine whether the attitude of the roadside device 10 has been corrected, and if it determines that the attitude has been corrected, store the image acquired in S2 in the storage unit 12 as a new reference image. If it determines that the attitude has not been corrected, the operation of the control unit 11 returns to S5 described above. Any method may be used to determine whether the attitude of the roadside device 10 has been corrected. For example, the control unit 11 may determine that the attitude has been corrected when the feature points of the reference image and the image acquired in S2 match, as described in S5 above.

[0037] As a modified example of the present disclosure, the roadside device 10 may include legs and wheels on its bottom surface to support the roadside device 10. In this modified example, the legs include motors and are configured to extend and retract so that the wheels are lifted off the ground or placed on the ground in response to instructions from the control unit 11. That is, the wheels are lifted off the ground when the legs are extended and are placed on the ground when the legs are retracted. In this modified example, when outputting the work procedure in S5 described above, the control unit 11 sends instructions to the motors of the legs to retract the legs and place the wheels on the ground. This allows the roadside device 10 to roll on the ground using the wheels, making it easier for the pedestrian P to correct the position of the roadside device 10. After the pedestrian P has completed the work, the control unit 11 may again send instructions to the motors to extend the legs and lift the wheels off the ground again. [Explanation of symbols]

[0038] 10 Roadside equipment 11 Control section 12 Storage section 13 Communications Department 14 Input section 15 Output section 16 Photography Department

Claims

[Claim 1] A roadside device comprising an imaging unit, an output unit, and a control unit, The control unit acquiring a reference image captured by the image capturing unit when the roadside device is installed in a reference posture; After acquiring the reference image, an image captured by the imaging unit is further acquired; calculating a deviation amount between the attitude of the roadside device and the reference attitude based on a comparison between the reference image and the image; When the deviation amount is equal to or greater than a threshold value, the roadside device outputs, via the output unit, to the pedestrian, information indicating a procedure for bringing the attitude of the roadside device closer to the reference attitude.

Images