Information processing device, roadside unit, and information processing method

Abstract

This information processing device comprises an image acquisition part and a processor. The image acquisition part acquires a captured image from an imaging part. The processor detects a detection target in a target area that is included in the captured image. The processor performs: detecting a road surface area on the basis of at least one captured image acquired from the imaging part; extracting the movement locus of the detection target on the captured image on the basis of the captured images in a time series that are acquired from the imaging part; and setting, as the target area, an area obtained by adding the area based on the movement locus to the road surface area.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to Japanese Patent Application No. 2022-062081, filed April 1, 2022, the entire disclosure of which is incorporated herein by reference. [Technical Field]

[0002] The present disclosure relates to an information processing device, a roadside device, and an information processing method. [Background technology]

[0003] A technology for detecting the road surface based on an image captured by an imaging device has been developed. For example, in Patent Document 1, an image processing device mounted on a vehicle detects a boundary line between the road surface area and other areas, and determines the area on the vehicle's side from the detected boundary line as the road surface area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-41972 Summary of the Invention

[0005] The information processing device according to the present disclosure includes an image acquisition unit and a processor. The image acquisition unit acquires captured images from an imaging unit. The processor detects a detection target in a target area included in the captured images. The processor detects a road surface area based on at least one captured image acquired from the imaging unit, extracts a movement trajectory of the detection target on the captured images based on time-series captured images acquired from the imaging unit, and sets an area obtained by adding an area based on the movement trajectory to the road surface area as the target area.

[0006] The roadside device according to the present disclosure includes an imaging unit and the information processing device.

[0007] The information processing method disclosed herein acquires an image from an imaging unit, detects a road surface area based on at least one image acquired from the imaging unit, extracts a movement trajectory of a detection target on the image based on time-series image images acquired from the imaging unit, and sets an area obtained by adding an area based on the movement trajectory to the road surface area as a target area for detecting the detection target. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of a situation in which a roadside unit installed at an intersection is used. [Figure 2] FIG. 2 is a block diagram illustrating an example of a system including a roadside device according to an embodiment. [Figure 3] FIG. 3 is a functional block diagram of the information processing device of FIG. [Figure 4] FIG. 4 is a flowchart illustrating an example of processing executed by the control unit. [Figure 5] FIG. 5 is a diagram showing an example of a first road surface area acquired as a result of the road surface recognition processing. [Figure 6] FIG. 6 is a flowchart showing an example of a process for identifying the second road surface area based on the movement trajectory. [Figure 7] FIG. 7 is a diagram illustrating an example of a method for extracting a movement trajectory. [Figure 8] FIG. 8 is a diagram showing a target area in which the first road surface area and the second road surface area are superimposed. DETAILED DESCRIPTION OF THE INVENTION

[0009] Roadside devices are known that are installed at intersections with poor visibility, detect targets such as vehicles and / or pedestrians approaching from one road, and notify vehicles approaching from the other road to prevent head-on collisions and other accidents. Roadside devices preferably have road surface position information in order to accurately recognize targets moving on the road surface. Setting the road surface position in an image captured by the roadside device manually creates a workload, so it would be more convenient if the roadside device could automatically detect the road surface from an image captured by an imaging device. However, it is expected that the roadside device will not be able to accurately grasp the road surface from an image captured by the imaging device due to the influence of structures surrounding the road, such as telephone poles and / or chain-link fences. The information processing device, roadside device, and information processing method described below can recognize road surface areas of surrounding roads and improve the accuracy of detecting targets.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The drawings used in the following description are schematic. The dimensional ratios and the like in the drawings do not necessarily correspond to the actual ones.

[0011] (Scenes where the system can be applied) In the present disclosure, a roadside device 10 is a device that is placed near an intersection of a road 5 as shown in Fig. 1 and is used for traffic control and / or information provision. The roadside device 10 includes an imaging unit 11 and an information processing device 12. In addition to the imaging unit 11, the roadside device 10 may also include sensors such as millimeter-wave radar and LiDAR.

[0012] The imaging unit 11 is placed on a structure such as a utility pole, a traffic light pole, a dedicated support pole, or a road gantry located near a road intersection so as to overlook the intersection. The imaging unit 11 continuously captures images of the road 5 including the intersection.

[0013] The information processing device 12 can provide information to a vehicle 20 traveling on a road. The information processing device 12 analyzes an image captured by the imaging unit 11 to detect a detection target 21. The detection target 21 is an object and / or a person that may affect the passage of the vehicle 20 traveling on the road at an intersection. The detection target 21 includes multiple types of objects, such as a vehicle, a motorcycle, a bicycle, and a pedestrian. In FIG. 1, the detection target 21 is a vehicle. When a detection target 21 is approaching an intersection, the information processing device 12 can notify a vehicle 20 approaching the intersection from another direction that the detection target 21 is approaching the intersection. Furthermore, when the vehicle 20 is an autonomous vehicle capable of autonomous driving, the information processing device 12 may be configured to transmit control information instructing the vehicle 20 to decelerate and / or stop, etc., to prevent a collision between the detection target 21 and the vehicle 20 at the intersection.

[0014] In order to correctly recognize the detection target 21 moving on the road, it is advantageous for the information processing device 12 to accurately grasp the position of the road surface near the intersection. If the information processing device 12 does not correctly grasp the road surface, there is a risk that vehicles moving in a parking lot other than the road 5, pedestrians walking in a park, etc. may be mistakenly recognized as detection targets 21 on the road. Furthermore, there is a risk that unnecessary information processing will be performed to recognize these vehicles, people, etc., and the performance of the device will be reduced.

[0015] One method for registering the road surface of the road 5 in the information processing device 12 is for the worker installing the roadside device 10 to display an image captured by the imaging unit 11 on the screen of the worker's terminal and manually register the road position. However, this method has problems such as the burden of registration being placed on the worker and the possibility of variations in the accuracy of road surface registration depending on the worker.

[0016] Therefore, it is conceivable to apply road surface recognition processing using AI (artificial intelligence) to the image of the intersection captured by the imaging unit 11 to automatically determine the road surface. However, in an actual road environment, structures such as telephone poles, streetlights, and fences exist around the intersection. When these structures are located within the road 5 in the captured image and / or across the road 5, the road surface may not be accurately identified using only road surface recognition processing using AI. As will be described below, the information processing device 12 of the present disclosure can improve the accuracy of road surface identification.

[0017] (Roadside unit configuration) 2, the system including the roadside unit 10 includes, in addition to the roadside unit 10, an operator terminal 17 and a management center 18 that can communicate with the roadside unit 10. Furthermore, the roadside unit 10 can communicate with vehicles 20 on the road.

[0018] The imaging unit 11 included in the roadside unit 10 is an imaging device (camera) that captures images of the area near an intersection. In this embodiment, the imaging unit 11 is an imaging device that detects visible light. Alternatively, the imaging unit 11 may be an imaging device that detects infrared light. The imaging unit 11 includes an imaging element and an optical system. The imaging element may include either a charge-coupled device image sensor (CCD image sensor) or a complementary metal-oxide semiconductor (CMOS) image sensor. The optical system may include one or more lenses. To capture images of roads extending in multiple directions from an intersection, the imaging unit 11 may include an optical system capable of obtaining images with a wide angle of view, such as a wide-angle lens. The imaging unit 11 can capture images at a predetermined frame rate. The frame rate may be, for example, 30 frames per second (fps) or 60 fps, but is not limited to these. The frame rate may be adjustable.

[0019] The information processing device 12 processes the images captured by the imaging unit 11 and transmits information to the vehicle 20 based on the image processing results. The information processing device 12 may be located away from the imaging unit 11. Alternatively, the information processing device 12 may be configured integrally with the imaging unit 11 rather than being separate from the imaging unit 11. The information processing device 12 includes an image acquisition unit 13, a control unit 14, a storage unit 15, and a communication unit 16.

[0020] The image acquisition unit 13 acquires the image captured by the imaging unit 11. The image acquisition unit 13 includes an input interface for receiving the image from the imaging unit 11. The image acquisition unit 13 may include an electrical connector compatible with transmission of an electrical signal from the imaging unit 11, or a wireless communication device compatible with wireless communication.

[0021] The control unit 14 controls the entire information processing device 12 and executes processing based on the image acquired by the image acquisition unit 13. The control unit 14 includes one or more processors. Processors include general-purpose processors that load specific programs to execute specific functions, and dedicated processors specialized for specific processing. Processors include central processing units (CPUs), graphics processing units (GPUs), application specific integrated circuits (ASICs), and programmable logic devices (PLDs). PLDs include field-programmable gate arrays (FPGAs). The control unit 14 may include either a system-on-a-chip (SoC) or a system in a package (SiP) in which one or more processors work together.

[0022] The control unit 14 can execute a program that defines a control procedure. The control unit 14 may be configured to load a program recorded on a non-transitory computer-readable medium into a memory built into the processor and implement the program.

[0023] The storage unit 15 can store various information required for processing by the control unit 14, information generated by the control unit 14, programs for operating the control unit 14, etc. For example, the storage unit 15 may store information indicating the position on an image captured by the imaging unit 11 of a road surface area around an intersection. The storage unit 15 may also store external parameters such as the depression angle and height from the ground of the imaging unit 11. Such parameters may be set by an operator from the operator terminal 17 when installing the roadside unit 10. The storage unit 15 includes a primary storage device and a secondary storage device. The storage unit 15 includes a semiconductor memory. The semiconductor memory may include a volatile memory and a non-volatile memory. The storage unit 15 may further include a magnetic memory and / or an optical memory.

[0024] The communication unit 16 transmits and receives information to and from devices external to the roadside unit 10 by wired and wireless communication means. The communication unit 16 may communicate with external devices via a dedicated line or a network such as the Internet.

[0025] For example, the communication unit 16 can be connected to the worker terminal 17 via a cable or wireless device to communicate directly with the worker terminal 17. The worker terminal 17 is a terminal that allows a worker to configure various settings for the roadside unit 10 when installing the roadside unit 10. The worker terminal 17 may be a personal computer (PC), a portable information terminal, or a dedicated information processing device. The worker terminal 17 may be equipped with a dedicated application for configuring the roadside unit 10.

[0026] Furthermore, for example, the communication unit 16 can communicate with information devices in a management center 18 that manages multiple roadside units 10. The management center 18 can send various instructions to the roadside units 10 and acquire information such as images from the roadside units 10. For example, when the road environment around an intersection changes, the management center 18 can instruct the roadside units 10 to reset road surface information.

[0027] Furthermore, the communication unit 16 can communicate with a communication device mounted on the vehicle 20. The communication unit 16 can perform communication compatible with a short-range wireless communication system in order to communicate with the vehicle 20. Short-range wireless communication systems include, but are not limited to, DSRC (Dedicated Short Range Communications), LTE-V2X (Long-Term Evolution V2X), and 5G (5th Generation).

[0028] (Operation of control unit) The functions of the control unit 14 will be further described with reference to FIG. 3. The control unit 14 includes functional blocks: a road surface extraction unit 14a, a first road surface area identification unit 14b, an object tracking unit 14c, a second road surface area identification unit 14d, a target area setting unit 14e, and a recognition processing unit 14f. The road surface extraction unit 14a, the first road surface area identification unit 14b, the object tracking unit 14c, the second road surface area identification unit 14d, and the target area setting unit 14e perform a setting stage process to set a target area for recognizing the detection target 21 when, for example, installing the roadside device 10. The recognition processing unit 14f performs an operation stage process to monitor the detection target 21 around an intersection and provide information to the vehicle 20. Generally, the control unit 14 transitions to the operation stage after performing the setting stage process. However, after transitioning to the operation stage process, the control unit 14 may perform a setting stage process to re-set the target area in response to changes in the road environment, in response to instructions from the management center 18 or the like.

[0029] Each functional block of the control unit 14 may be a hardware module or a software module. The control unit 14 may execute all operations performed by each functional block. The operations performed by each functional block may be rephrased as operations performed by the control unit 14. Some of the processing performed by each unit of the control unit 14 may be executed using a computer located in a remote location, such as a computer in the management center 18.

[0030] In the setting stage, the road surface extraction unit 14a extracts the road surface based on the image captured by the imaging unit 11 and acquired from the image acquisition unit 13. The road surface extraction can be performed using, for example, AI. For example, the road surface extraction unit 14a can input an image of the intersection from which road surface extraction is to be performed into a trained model trained using training data that takes images of a large number of intersections as input data and outputs the road surface area of ​​each image, and cause the trained model to output the road surface area.

[0031] The first road surface area identification unit 14b identifies a first road surface area based on the road surface area identified by the road surface extraction unit 14a. The first road surface area identification unit 14b may directly use the road surface area identified by the road surface extraction unit 14a as the first road surface area. Alternatively, the first road surface area identification unit 14b may display the road surface area identified by the road surface extraction unit 14a on the worker terminal 17 and wait for instructions from the worker. The worker may input an instruction to correct a part of the displayed image from the worker terminal 17. For example, if a parking lot that is not a road is recognized as a road surface, the worker may use an input device such as a mouse to remove that area from the road surface area.

[0032] The object tracking unit 14c tracks a plurality of detection targets 21 on the captured images based on the time-series captured images acquired from the imaging unit 11 via the image acquisition unit 13. The detection targets 21 can be tracked by sequentially searching for each detection target 21 in each frame of the images acquired in time series. As a method for tracking the detection targets 21, a region-based object tracking method including template matching and active search methods, or a feature-point-based object tracking method including the Kanade-Lucas-Tomasi (KLT) method, can be used.

[0033] The object tracking unit 14c sequentially extracts the movement trajectory of the detection target 21 included in the captured image. The object tracking unit 14c executes the extraction of the movement trajectory, for example, for a specific period of time or until a specific number of movement trajectories are extracted. The object tracking unit 14c identifies a change in a specific position of the detection target 21 as a movement trajectory in the image space of the captured image. The specific position can be, for example, the center of the part of the detection target 21 that contacts the road surface of the road 5.

[0034] The second road surface area identifying unit 14d identifies an area based on the movement trajectory extracted by the object tracking unit 14c as the second road surface area. The second road surface area identifying unit 14d can identify an area that is a portion through which the movement trajectory extracted by the object tracking unit 14c passes and its vicinity and that intersects with the first road surface area as the second road surface area. The second road surface area identifying unit 14d can be a collection of areas having a predetermined width centered on a line indicating the movement trajectory. The predetermined width can be the width when converted into real space.

[0035] The target area setting unit 14e sets the area obtained by adding the second road surface area to the first road surface area as the target area for detecting the detection target 21 in the operation stage. The target area setting unit 14e stores the target area in the storage unit 15.

[0036] During the operation of the roadside device 10, the recognition processing unit 14f acquires position information of the target area from the storage unit 15. The recognition processing unit 14f sequentially acquires images captured by the imaging unit 11 from the image acquisition unit 13. The recognition processing unit 14f performs image recognition on the target area, and can detect the detection target 21.

[0037] The recognition processing unit 14f can further communicate with the vehicle 20 approaching the intersection via the communication unit 16. When the detection target 21 is approaching the intersection, the recognition processing unit 14f may transmit information indicating that the detection target 21 is approaching the intersection to the vehicle 20 approaching the intersection. The information transmitted by the recognition processing unit 14f may be transmitted to the driver of the vehicle 20 through a monitor or speaker in the vehicle 20 as a message such as "A vehicle is approaching from the right." Furthermore, if the vehicle 20 is an autonomous vehicle, the recognition processing unit 14f may transmit control information to the vehicle 20, such as adjusting the speed, so that the detection target 21 and the vehicle 20 do not collide or get too close to each other at the intersection.

[0038] The recognition processing unit 14f performs image recognition processing only on the target area, which is the road surface area, thereby reducing the load on image recognition. Furthermore, since the detection target 21 located in a place other than the road 5 is not detected, the detection accuracy of the detection target 21 is improved.

[0039] (Processing flow at the setting stage by the control unit) The flow of the process in the setting stage executed by the control unit 14 will be described with reference to Fig. 4. In the following, the process executed by each functional block of the control unit 14 will be described as the process executed by the control unit 14.

[0040] The information processing device 12 may be configured to read and implement a program recorded on a non-transitory computer-readable medium to perform the processing performed by the control unit 14 described below. Non-transitory computer-readable media include, but are not limited to, magnetic storage media, optical storage media, magneto-optical storage media, and semiconductor storage media. Magnetic storage media include magnetic disks, hard disks, and magnetic tapes. Optical storage media include optical disks such as CDs (Compact Discs), DVDs, and Blu-ray (registered trademark) Discs. Semiconductor storage media include ROMs (Read Only Memory), EEPROMs (Electrically Erasable Programmable Read-Only Memory), and flash memories.

[0041] First, an operator installs the roadside unit 10 at an appropriate position near an intersection. After completing the installation, the operator activates the roadside unit 10 using an activation switch provided on the roadside unit 10 or the operator terminal 17. When the roadside unit 10 is activated, the control unit 14 acquires an image captured by the imaging unit 11 and displays it on the operator terminal 17 (step S101). The control unit 14 may display the image together with a message prompting the operator to start road surface recognition processing.

[0042] When the control unit 14 receives an instruction to start the road surface recognition process from the operator terminal 17, it starts the road surface recognition process (step S102). The control unit 14 extracts the road surface using the trained model stored in the storage unit 15. The road surface may be extracted by synthesizing an image in which there are no objects and / or people moving on the road from multiple images acquired from the image acquisition unit 13, and using the image as an input to the trained model.

[0043] As a result of step S102, the road surface area is identified as a first road surface area (step S103). An example of the identified first road surface area 32 will be described with reference to FIG. 5. An image processing area 31 indicated by a rectangular frame in FIG. 5 is an area for which the control unit 14 performs image processing. The control unit 14 does not perform image processing on areas outside the image processing area 31. In the example of FIG. 5, a utility pole 33 divides the road surface on the captured image, and therefore a portion of the road surface area that is connected to the first road surface area 32 is actually a non-recognized road surface area 34 that is not recognized as a road surface. The non-recognized road surface area 34 is an area that should normally be recognized as a target area for detecting the detection target 21.

[0044] The captured image also includes a wire mesh fence 35. The wire mesh fence 35 is made up of mesh-like metal wires, and the other side can be seen through the gaps between the metal wires. However, the control unit 14 may not be able to recognize that the road surface portion overlapping the wire mesh fence 35 is a road surface. The imaging unit 11 can capture an image of the detection target 21 even in the area beyond the wire mesh fence 35. Therefore, it is desirable that the road surface area visible beyond the wire mesh fence 35 be included in the target area.

[0045] In step S102, there is a case where a parking lot 36, which is not actually a road, is recognized as a road surface. In step S103, the control unit 14 may perform a process of displaying the road surface area extracted in step S102 on the worker terminal 17 and prompting the worker to correct the error. For example, if the parking lot 36 is recognized as a road surface area and the worker issues an instruction via the worker terminal 17 to exclude the parking lot 36 from the road surface area, the control unit 14 may exclude the parking lot 36 from the first road surface area 32.

[0046] After step S103, the control unit 14 performs image recognition processing on the images sequentially acquired from the image acquisition unit 13, and starts processing to extract a movement trajectory of the detection target 21 (step S104). After step S103, the control unit 14 may wait for an instruction to start extracting a movement trajectory from the worker terminal 17.

[0047] When the process of extracting a movement trajectory is started in step S104, the control unit 14 executes a series of processes (step S105) of extracting a movement trajectory and identifying a second road surface area, which is a road surface area based on the extracted movement trajectory. The process performed in step S105 starting from step S104, that is, the process from extracting a movement trajectory to identifying the second road surface area, will be described with reference to the flowchart in FIG.

[0048] First, the control unit 14 performs image recognition processing on the captured image acquired from the image acquisition unit 13 to recognize the detection target 21 (step 201). Methods for recognizing the detection target 21 include various known object recognition methods. For example, methods for recognizing the detection target 21 include a method using shape recognition, a method using template matching, a method of calculating feature amounts from an image and using them for matching, etc. This allows the control unit 14 to determine the type of the detection target 21 in the captured image. Types of the detection target 21 include vehicles, motorcycles, bicycles, pedestrians, etc.

[0049] The control unit 14 identifies the type of the detection target 21 on the captured image and also identifies the position of the detection target 21 (step S202). The position of the detection target 21 is recognized as a predetermined point on the captured image corresponding to the detection target 21. An example of a method for identifying the position of the detection target 21 will be described with reference to FIG. 7. In FIG. 7, the detection target 21 is a pedestrian. In FIG. 7, the detection targets 21 sequentially captured by the imaging unit 11 are displayed on a single diagram.

[0050] When the control unit 14 recognizes the detection target 21 on each captured image, it sets a rectangular frame 42 (bounding box) surrounding the detection target 21. The control unit 14 determines the position of the detection target 21 to be a center point 43 located in the center of the bottom of the frame 42. The center point 43 corresponds to the position of the detection target 21 closest to the road-side unit 10 among the positions where the detection target 21 contacts the road surface of the road 5.

[0051] The control unit 14 tracks the detection target 21 in the images that are successively acquired, and extracts a movement trajectory 44 of the detection target 21, as exemplified in FIG. 7 (step S203).

[0052] The control unit 14 stores the extracted movement trajectory 44 in the storage unit 15 (step S204). The control unit 14 may store the movement trajectory 44 and the type of the detection target 21 corresponding to the movement trajectory 44 in the storage unit 15.

[0053] Following step S204, the control unit 14 determines whether a predetermined condition for ending extraction of the movement trajectories 44 is satisfied (step S205). The predetermined condition is set, for example, as the time for extracting the movement trajectories 44 and / or the number of movement trajectories 44 to be extracted. For example, if extraction of the movement trajectories 44 is to be performed for 30 minutes, the predetermined condition may be set as 30 minutes having passed since the start of extraction of the movement trajectories 44.

[0054] In step S205, if the process of extracting the movement trajectories 44 is not to be ended (step S205: No), the control unit 14 returns to the process of step S201 and repeats the processes of steps S201 to S204. This allows a large number of movement trajectories 44 to be extracted and stored in the storage unit 15.

[0055] In step S205, if the process of extracting the movement trajectory 44 is to be ended (step S205: Yes), the control unit 14 proceeds to the process of step S206 and subsequent steps.

[0056] In step S206, the control unit 14 reads out each of the movement trajectories 44 stored in the storage unit 15, and performs processing to widen the width of each of the movement trajectories 44. The control unit 14 may widen the width of the movement trajectory 44 on the captured image to a predetermined width. The predetermined width may differ depending on the type of the detection target 21 corresponding to the movement trajectory 44. When the detection target 21 is a vehicle, the predetermined width may be set wider than when the detection target 21 is a person.

[0057] Alternatively, the control unit 14 may widen the width of the movement trajectory 44 so that it corresponds to a predetermined width in real space. The predetermined width may differ depending on the type of the detection target 21. For example, if the detection target 21 is a person, the predetermined width may be 40 cm. Alternatively, if the detection target 21 is a vehicle, the predetermined width may be 1.8 m. The control unit 14 can calculate the distance to the detection target 21 using external parameters such as the depression angle and height from the ground of the imaging unit 11 stored in the storage unit 15, and calculate the width of the movement trajectory 44 on the captured image that corresponds to the predetermined width in real space.

[0058] Next, the control unit 14 excludes movement trajectories 44 that satisfy a predetermined condition (step S207). For example, the control unit 14 performs statistical processing on the movement trajectories 44 and deletes all or part of the movement trajectories 44 that have few overlapping or adjacent portions with other movement trajectories 44. In this way, the control unit 14 excludes areas based only on movement trajectories 44 that occur at a predetermined frequency or a predetermined number or less. For example, the control unit 14 can exclude movement trajectories 44 that pass through places other than the road 5, such as movement trajectories 44 of a person passing through the grounds of a house adjacent to the road 5. Step S207 is not essential. Also, step S207 can be executed before step S206.

[0059] The control unit 14 identifies the area through which the movement trajectory 44 obtained as a result of steps S206 and S207 passes as the second road surface area 37 (step S208). An example of the second road surface area 37 is shown as an area shaded with coarse dots in FIG. 8. As shown in FIG. 8, the unrecognized road surface area 34, which was not included in the first road surface area 32 in FIG. 5 due to the utility pole 33, is included in the second road surface area 37. In addition, the area not recognized as the first road surface area 32 due to the chain link fence 35 is also included in the second road surface area 37.

[0060] In step S208, the control unit 14 can determine as the second road surface area 37 only the area through which the movement trajectory 44 passes that overlaps with the first road surface area 32. For example, even if the movement trajectory 44 of the detection object 21, such as a vehicle or a person moving through the parking lot 36, is extracted, the movement trajectory 44 does not intersect with the first road surface area 32, and therefore is determined not to be the second road surface area 37.

[0061] Furthermore, the control unit 14 adds an area surrounded by a plurality of movement trajectories 44 to the second road surface area 37 (step S209). This makes it possible to prevent many gaps from occurring in the second road surface area 37 even when the number of movement trajectories 44 is small. Note that step S209 is not essential.

[0062] After step S209, the control unit 14 proceeds to step S106 in the flowchart of Fig. 4. The control unit 14 sets the area obtained by adding the first road surface area 32 specified in step S103 to the second road surface area 37 specified in step S105 as a target area 38 for detecting the detection target 21. The area obtained by adding the second road surface area 37 to the first road surface area 32 means the area that is the union of the first road surface area 32 and the second road surface area 37. The control unit 14 stores information indicating the position of the target area 38 on the captured image in the memory unit 15.

[0063] Through the processing in the setting stage as described above, the roadside device 10 can detect the detection target 21 by using the target area 38 stored in the storage unit 15 in the operation stage as an area for performing image processing.

[0064] As described above, by using the roadside device 10 according to this embodiment, the setting of the target area 38 can be automated by the information processing device 12 at the stage of setting the roadside device 10. Furthermore, by using the roadside device 10 according to the present disclosure, it is possible to set areas separated by structures such as utility poles, which were difficult to extract as road surfaces using conventional systems, and areas visible through a chain-link fence, as the target area 38 for detecting the detection target 21. This makes it possible to set the target area 38 with high accuracy, without individual differences between operators, operational errors, omissions in settings, etc.

[0065] Furthermore, when the roadside device 10 is in operation, it is possible to perform a highly accurate recognition process of the detection target 21 in the target area 38. This makes it possible to prevent errors, omissions, and / or delays in transmitting an approaching notification when transmitting an approaching notification to a vehicle 20 attempting to pass through an intersection, the approaching notification.

[0066] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art would easily be able to make various modifications or alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of the present disclosure. For example, the functions of each component or step can be rearranged so as not to cause logical inconsistencies, and multiple components or steps can be combined or divided into one. Although the embodiments of the present disclosure have been described primarily in terms of an apparatus, the embodiments of the present disclosure can also be realized as a method including steps executed by each component of the apparatus. The embodiments of the present disclosure can also be realized as a method, a program, or a storage medium having a program recorded thereon, executed by a processor provided in the apparatus. It should be understood that these are also encompassed within the scope of the present disclosure. [Explanation of symbols]

[0067] 5 road 10 Roadside unit 11 Imaging unit 12 Information processing equipment 13 Image acquisition unit 14 Control unit (processor) 14a Road surface extraction part 14b 1st road surface area identification part (road surface area) 14c Object Tracking Unit 14d Second road surface area identification unit (area based on movement trajectory) 14e Target area setting section 14f Recognition processing section 15 Storage section 16 Communications Department 17 Worker terminal 18 Management Center 20 vehicles 21 Detection target 31 Image Processing Area 32 1st road surface area 33 Electric pole 34 Unrecognized road surface area 35 Chainlink Fence 36 Parking 37 Second road surface area 38 Object Recognition Target Area 42 slots 43 Center point 44 Movement trajectory

Claims

1. an image acquisition unit that acquires a captured image from the imaging unit; A process for detecting targets including people and vehicles in a target area included in the captured image. processor and Equipped with The processor detects a road surface based on at least one captured image acquired from the imaging unit. and detecting a region on the captured image based on the time-series captured images acquired from the imaging unit. A movement trajectory of the detection object is extracted, and an area based on the movement trajectory is added to the road surface area. setting a region as the region of interest; The movement trajectory obtained by tracking the detection target in time series also includes a movement trajectory that overlaps with an obstacle between the imaging unit and the road surface area. Information processing device.

2. and setting the target area based on the movement trajectory that intersects with the road surface area.

2. The information processing device according to claim 1.

3. The area based on the movement trajectory has a predetermined width when the line indicating the movement trajectory is converted into real space. The information processing device according to claim 1 , further comprising an area enlarged so as to

4. The predetermined width differs depending on the type of the detection target related to the movement trajectory. The information processing device described.

5. The processor excludes regions based only on the movement trajectories having a predetermined frequency or a predetermined number or less. The information processing apparatus according to claim 1 , wherein the area based on the movement trajectory is set so as to exclude the movement trajectory.

6. The processor divides an area surrounded by a plurality of the trajectories into an area based on the trajectories. The information processing apparatus according to claim 1 .

7. An imaging unit; The information processing device according to any one of claims 1 to 6. A roadside unit comprising:

8. The control unit Acquire a captured image from the imaging unit; detecting a road surface area based on at least one captured image acquired from the imaging unit; Based on the time-series captured images acquired from the imaging unit, people and vehicles on the captured images are detected. Extracting the movement trajectory of the detection object including both, The area obtained by adding the area based on the movement trajectory to the road surface area is used as the detection target. Set it as the target area, The movement trajectory obtained by tracking the detection target in time series also includes a movement trajectory that overlaps with an obstacle between the imaging unit and the road surface area. Information processing methods.

Images