Entry inference system, entry inference method, and control device

Abstract

Provided is an entry inference system that comprises: a detection unit that sets, as a detection range, a first area that is a prescribed target area and a second area through which a user is to pass before entering the first area; and a control device that performs pre-processing for inferring whether the user has entered the first area. At the pre-processing, the control device sets an exit area that is an area of the first area through which the user is to pass first when entering the first area from the second area on the basis of detection results from the detection unit and sets a reference coordinate system for the exit area on the basis of the position and shape of the exit area.

Description

Approach estimation system, approach estimation method, and control device

[0001] This application claims priority to Japanese Patent Application No. 2023-201100, filed November 28, 2023, and incorporates by reference all of the contents of that application.

[0002] Patent Document 1 discloses a technology for estimating a pedestrian's movement route by detecting the trajectory and body orientation of the pedestrian with a sensor.

[0003] Japanese Patent Application Laid-Open No. 2020-173600

[0004] An entry estimation system according to one embodiment of the present disclosure includes a detection unit having a detection range that includes a first area, which is a predetermined destination area, and a second area that a user passes through before entering the first area, and a control unit that performs pre-processing related to estimating whether a user has entered the first area, and in the pre-processing, the control unit sets an exit area, which is the first area in the first area that a user passing through when entering the first area from the second area, based on the detection result by the detection unit.

[0005] FIG. 1 is a diagram illustrating an overview of the entry estimation system. FIG. 2 is a diagram illustrating issues with the entry estimation system. FIG. 3 is a diagram illustrating issues with the entry estimation system. FIG. 4 is a diagram illustrating features of the entry estimation system according to this embodiment. FIG. 5 is a configuration diagram of the entry estimation system according to this embodiment. FIG. 6 is a block diagram illustrating an example of the hardware configuration of the estimation device. FIG. 7 is a functional block diagram of the estimation device. FIG. 8 is a diagram illustrating an example of setting an exit area on a GUI. FIG. 9 is a diagram illustrating an example of correcting an exit area on a GUI. FIG. 10 is a diagram illustrating an example of setting an exit area and a reference coordinate system. FIG. 11 is a diagram illustrating another example of setting an exit area and a reference coordinate system. FIG. 12 is a diagram illustrating another example of setting an exit area and a reference coordinate system. FIG. 13 is a flowchart illustrating processing for setting an exit area and a reference coordinate system. FIG. 14 is a flowchart illustrating processing for entry estimation. FIG. 15 is a flowchart illustrating processing for setting an exit area and a reference coordinate system according to a modified example. FIG. 16 is a diagram illustrating walking detection processing for setting an exit area. 17 and 18 are diagrams illustrating approach estimation according to a modified example.

[0006] [Problem to be Solved by the Present Disclosure] Here, the detected trajectory of a pedestrian changes (appears different) depending on the sensor installation conditions, road shape conditions, etc. Therefore, when estimating a pedestrian's movement path from a trajectory, etc., it is necessary to prepare estimation logic for each sensor installation condition, road conditions, etc. This increases the burden of preparing estimation logic when estimating whether a user is entering a predetermined destination area.

[0007] Effect of the Present Disclosure According to the present disclosure, in an approach estimation system and an approach estimation method, estimation logic can be prepared more easily.

[0008] [Description of Embodiments of the Present Disclosure] First, the contents of the embodiments of the present disclosure will be listed and described. [1] An entry estimation system according to one embodiment includes a detection unit having a detection range that includes a first area that is a predetermined destination area and a second area that a user passes through before entering the first area, and a control unit that performs pre-processing related to estimation of whether the user has entered the first area, and in the pre-processing, the control unit sets an exit area that is the first area in the first area that a user passing through when entering the first area from the second area based on the detection result by the detection unit, and sets a reference coordinate system related to the exit area based on the position and shape of the exit area.

[0009] In the approach estimation system described in [1] above, in pre-processing, an exit area is set in the first area, which is a predetermined destination area, based on the detection results by the detection unit, and a reference coordinate system for the exit area is set taking into account the position and shape of the exit area. By setting the reference coordinate system in advance taking into account the position and shape of the exit area, the user's trajectory can be derived under the same conditions regardless of differences between each exit area. For example, since the trajectory appears different depending on the installation conditions of the detection unit and the road shape conditions of the first area, it is usually necessary to prepare estimation logic (estimated results corresponding to the trajectory) for each installation condition of the detection unit and each road shape condition, which increases the preparation burden. In this regard, by setting a reference coordinate system for each exit area as described above, it is possible to derive the user's trajectory under the same conditions as long as the setting rules are standardized. This eliminates the need to prepare estimation logic (estimated results corresponding to the trajectory) for each installation condition of the detection unit and each road shape condition, making it easier to prepare the estimation logic.

[0010] [2] In the entry estimation system described in [1] above, the control device may further execute an estimation process after completing the pre-processing, and in the estimation process, derive a trajectory of the user's movement in a reference coordinate system from multiple detection results of the user in the second area, and estimate whether the detected user will enter the first area based on the trajectory. In this way, by deriving the user's trajectory based on the reference coordinate system, it is possible to derive the user's trajectory under the same conditions regardless of differences in each exit area.

[0011] [3] In the entry estimation system described in [1] or [2] above, the control device may set a reference coordinate system in a pre-processing step, in which the direction of travel of the user in the exit area is the y-coordinate system and the direction perpendicular to the direction of travel is the x-coordinate system. This configuration allows the reference coordinate system to be set simply and appropriately using a common rule that takes into account the shape of the exit area.

[0012] [4] The entry estimation system described in any one of [1] to [3] above may further include a display unit that displays the detection results by the detection unit and can receive screen operation instructions from an operator related to the detection results, and the control device may set the position and shape of the exit area in the pre-processing based on the screen operation instructions from the operator received by the display unit. With this configuration, the exit area can be easily and appropriately set based on the screen operation by the operator.

[0013] [5] In the entry estimation system described in [4] above, the control device may set a tentative exit area by analyzing the detection results in pre-processing, and then set the position and shape of the exit area after the setting based on screen operation instructions related to modification of the tentative exit area by the operator, which are received by the display unit after the setting. In such a configuration, the tentative exit area is set automatically, for example, by AI or image processing, etc., which simplifies the screen operation instructions (instructions related to modification) by the operator, making it possible to set the exit area more easily.

[0014] [6] In the approach estimation system described in any one of [1] to [5] above, the first area may be an area including a pedestrian crossing, and the second area may be a waiting area on a sidewalk through which a user passes before entering the pedestrian crossing. With this configuration, it is possible to estimate with high accuracy whether a user on a sidewalk adjacent to the pedestrian crossing is a user entering the pedestrian crossing.

[0015] [7] An entry estimation method according to one embodiment is an entry estimation method executed by an entry estimation system, and includes the steps of: setting an exit area, which is the first area in the first area that a user entering the first area from the second area passes through, based on detection results relating to a first area, which is a predetermined destination area, and a second area that the user passes through before entering the first area; and setting a reference coordinate system relating to the exit area based on the position and shape of the exit area.

[0016] [8] The entry estimation method described in [7] above may further include a step of deriving a trajectory of the user's movement in a reference coordinate system from multiple detection results of the user in the second area, and estimating whether the detected user will enter the first area based on the trajectory.

[0017] [9] A control device according to one embodiment is a control device that performs pre-processing related to estimating whether a user has entered a first area, which is a predetermined destination area, and obtains detection results from a detection unit whose detection range is the first area and a second area that the user passes through before entering the first area. In the pre-processing, based on the detection results from the detection unit, an exit area is set, which is the first area in the first area that a user passing through when entering the first area from the second area, and a reference coordinate system related to the exit area is set based on the position and shape of the exit area.

[0018] [Details of the embodiments of the present disclosure] Specific examples of the embodiments of the present disclosure will be described below with reference to the drawings. The present invention is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. In the description of the drawings, the same elements are given the same reference numerals, and duplicate explanations will be omitted.

[0019] First, an overview of the approach estimation system will be described with reference to Figures 1 to 4. Figure 1 is a diagram illustrating the overview of the approach estimation system. Figures 2 and 3 are diagrams illustrating issues with the approach estimation system. Figure 4 is a diagram illustrating features of the approach estimation system according to this embodiment.

[0020] The entry estimation system is a system that estimates whether a user will enter a first area, which is a predetermined destination area, based on a detection result by a detection unit such as a camera. More specifically, the entry estimation system estimates whether a user will enter the first area based on a detection result of a user in a second area, which is an area adjacent to the first area and through which the user passing just before entering the first area passes.

[0021] In the example shown in FIG. 1 , an area including a crosswalk E1 is exemplified as an example of the first area, and two crosswalks E1A and E1B are exemplified as the crosswalk E1. Furthermore, a waiting area E2 on a sidewalk adjacent to both crosswalks E1A and E1B is exemplified as an example of the second area. The waiting area E2 is an area through which users entering the crosswalks E1A and E1B pass immediately before entering the crosswalks. The waiting area E2 is an area where a user Y1 who plans to enter the crosswalk E1A waits when a vehicle is passing over the crosswalk E1A, and an area where a user Y2 who plans to enter the crosswalk E1B waits when a vehicle is passing over the crosswalk E1B. For example, after waiting for a traffic light in the waiting area E2, the user Y1 (or user Y2) enters the crosswalk E1A (or crosswalk E1B), crosses the crosswalk E1A (or crosswalk E1B), and reaches the sidewalk on the opposite bank. Examples of users Y1 and Y2 include not only people walking, but also people traveling by vehicle (bicycle, wheelchair, kick scooter, stroller, etc.). Examples of user Y also include people using smartphones while walking (smartphone zombies), people carrying white canes, etc. Users Y1 and Y2 may also include moving objects other than people (animals, robots, etc.).

[0022] As shown in Figures 2(a) and 2(b), if the installation conditions of a detection unit such as a camera are different, a user's trajectory may appear different even at the same intersection. In Figure 2(a), the camera detects waiting area E2 from diagonally above, while in Figure 2(b), the camera detects waiting area E2 from vertically above. Due to the different camera installation positions and angles, as shown in Figures 2(a) and 2(b), the trajectories A11 and A12 of user Y1, which are actually the same trajectory, appear to be different from each other. Furthermore, the trajectories A21 and A22 of user Y2, which are actually the same trajectory, also appear to be different from each other. Therefore, the logic for estimating whether or not a user will enter the crosswalk E1 from the trajectory requires different crossing possibility determination criteria depending on the installation conditions of the detection unit such as a camera. For example, the estimation results for the trajectory need to be learned for each installation condition of the detection unit.

[0023] 3(a) and 3(b), the shapes of the crosswalk E1A into which user Y1 enters are different. In such a case, even if the trajectories A23 and A24 of user Y2 heading toward crosswalk E1B are the same, if the different trajectories A13 and A14 of user Y1 heading toward E1A are used as references, the trajectories A23 and A24 of user Y2 heading toward crosswalk E1B will appear different from each other. Therefore, the logic for estimating whether or not a user will enter the crosswalk E1 from the trajectory requires different crossing possibility determination criteria for each crosswalk shape. For example, the estimation results for the trajectory must be learned for each crosswalk shape.

[0024] Thus, preparing estimation logic for each detection unit installation condition and crosswalk shape is a heavy burden and poses a challenge for entry estimation systems. Therefore, the entry estimation system according to this embodiment sets a reference coordinate system for each crosswalk exit area (crosswalk entrance) and standardizes the rules for setting the reference coordinate system, making it possible to derive a user's trajectory under the same conditions even when the detection unit installation conditions and crosswalk shapes are different.

[0025] In the example shown in FIG. 4 , an exit area E11 is set at the entrance of user Y1 at the crosswalk E1A. An exit area E12 is set at the entrance of user Y2 at crosswalk E1B. A reference coordinate system for the exit area E11 (or E12) is set based on the position and shape of the exit area E11 (or E12). That is, a reference coordinate system represented by two axes, a y-coordinate system and an x-coordinate system, is set for the exit area E11 based on the position and shape of the exit area E11, and a reference coordinate system represented by two axes, a y-coordinate system and an x-coordinate system, is set for the exit area E12 based on the position and shape of the exit area E12 (details will be described later). Setting the reference coordinate systems in this manner eliminates the need to prepare estimation logic (estimated results corresponding to the trajectory) for each detection unit installation condition, road shape condition, etc., and allows for easier preparation of estimation logic.

[0026] 5 is a configuration diagram of the entry estimation system 1 according to this embodiment. As shown in FIG. 5, the entry estimation system 1 includes a camera 20 (detection unit) and an estimation device 10 (control device).

[0027] The camera 20 is a detection unit whose detection range includes a crosswalk E1 (first area), which is a predetermined destination area, and a waiting area E2 (second area), which a user passing through immediately before entering the crosswalk E1. The camera 20 has a detection range that covers at least a portion of both the crosswalk E1 and the waiting area E2. The camera 20 is installed, for example, on a sidewalk including the waiting area E2. More specifically, the camera 20 is installed on a pole 50 installed on the sidewalk including the waiting area E2. The pole 50 may be, for example, a traffic signal pole, a sign pole, or a lighting pole. The pole 50 has a vertical extension portion 51 and a support portion 52. The vertical extension portion 51 extends vertically. The height h of the vertical extension portion 51 is set to be equal to or greater than the height of traffic signal poles in Japan and other countries, including the United States, and may be set to be 3 m or more. The support portion 52 extends horizontally from the upper end of the vertical extension portion 51 and supports the camera 20. The support portion 52 may be a portion that supports a support object (such as a traffic light or a sign) other than the camera 20. The camera 20 is provided on the lower surface of the support portion 52, and has a detection range directed downward.

[0028] The camera 20 is installed, for example, directly above the waiting area E2, and detects the user Y in the waiting area E2 by setting the detection range to the downward direction as viewed from the camera 20. Directly above the waiting area E2 means vertically above a portion of the waiting area E2. Note that the camera 20 does not necessarily have to be installed directly above the waiting area E2, and may be installed diagonally above the waiting area E2 as long as it is in a position where the entire waiting area E2 can be detected.

[0029] The waiting area E2 is set to be, for example, a square shape of several meters by several meters (square shape in plan view), and may be set to be, for example, an area of ​​4 m by 4 m in plan view. The waiting area E2 is set to be continuous with the exit area E11 of the crosswalk E1 (the entrance to the crosswalk). The boundary between the crosswalk E1 and the waiting area E2 may be a simple line. In this case, when a user enters the crosswalk E1, it means that the user passes over the boundary line. Furthermore, the waiting area E2 does not necessarily have to be square, as long as it is within the detection range of the camera 20; it may be rectangular, circular, or any other shape that is set arbitrarily. Note that the waiting area E2 may be the entire detection range of the camera 20, or may be a part of it.

[0030] The camera 20 continuously captures images of the detection range (i.e., the crosswalk E1 and waiting area E2) and continuously transmits the image capture results (detection results) to the estimation device 10. The camera 20 transmits the image capture results (detection results) to the estimation device 10 via a base station (not shown) or by direct wireless communication or wired communication without using a base station.

[0031] The estimation device 10 is composed of one or more control computers. FIG. 6 is a block diagram illustrating an example of the hardware configuration of the estimation device 10. For example, the estimation device 10 includes a circuit 120 shown in FIG. 6. The circuit 120 includes one or more processors 121, a memory 122, a storage 123, and an input / output port 124. The storage 123 includes a computer-readable storage medium, such as a hard disk. The storage medium stores a program for executing a predetermined intrusion estimation processing procedure. The storage medium may be a removable medium, such as a non-volatile semiconductor memory, a magnetic disk, or an optical disk. The memory 122 temporarily stores the program loaded from the storage medium of the storage 123 and the results of calculations performed by the processor 121. The processor 121 executes the program in cooperation with the memory 122 to configure each functional module described below. The input / output port 124 inputs and outputs electrical signals in accordance with instructions from the processor 121.

[0032] The hardware configuration of the estimation device 10 is not necessarily limited to configuring each functional module by a program. For example, each functional module of the estimation device 10 may be configured by a dedicated logic circuit or an ASIC (Application Specific Integrated Circuit) that integrates such a dedicated logic circuit.

[0033] The estimation device 10 continuously acquires detection results from the camera 20 and estimates whether the detected user Y will enter the crosswalk E1 based on the acquired detection results. The estimation device 10 is configured to perform pre-processing related to estimation and to perform estimation processing to estimate whether the detected user will enter the crosswalk E1 (hereinafter, referred to as crosswalk E1A as an example) based on the results of the pre-processing and the user detection results by the camera 20 in the waiting area E2. FIG. 7 is a functional block diagram of the estimation device 10. The estimation device 10 includes an acquisition unit 11, a pre-processing unit 12, a trajectory derivation unit 13, an estimation unit 14, and a control unit 15.

[0034] The acquisition unit 11 continuously receives imaging results (detection results) from the camera 20. The imaging results acquired by the acquisition unit 11 are input to the pre-processing unit 12 and the trajectory derivation unit 13. As will be described later, the processing of the pre-processing unit 12 is executed only when an exit area or the like is set. Therefore, after the setting is completed, the imaging results may be input only to the trajectory derivation unit 13.

[0035] Based on the detection results from the camera 20, the pre-processing unit 12 sets an exit area E11, which is the area that a user entering the crosswalk E1A from the waiting area E2 will first pass through on the crosswalk E1A, and sets a reference coordinate system for the exit area E11 based on the position and shape of the exit area E11.

[0036] The setting of the exit area E11 will now be described. The pre-processing unit 12 outputs the imaging results input by the acquisition unit 11 to the display unit 60. The display unit 60 is a display that displays the imaging results (detection results) and can accept screen operation instructions (screen operation instructions related to the detection results) from the operator. The display unit 60 may be configured to accept, for example, mouse operations or touch panel operations. The display unit 60 displays the imaging results as an image on a GUI, as shown in FIG. 8, and accepts mouse operations or touch panel operations by the operator on the image. The operator can freely specify the position and shape of the exit area E11 by performing mouse operations or the like on the image on the GUI. The pre-processing unit 12 sets the position and shape of the exit area E11 based on the screen operation instructions (instructions via mouse operations or touch panel operations) by the operator accepted by the display unit 60. After the exit area E11 is set, the display unit 60 may accept input of operations (such as zooming in, zooming out, or rotation) to adjust the shape and range of the set exit area E11, as shown in FIG. 9. In this case, the pre-processing unit 12 adjusts the exit area E11 based on the operation instruction received on the display unit 60.

[0037] Furthermore, the pre-processing unit 12 may analyze the imaging results (detection results) to provisionally set the exit area, and after the provisional setting, set the position and shape of the corrected exit area E11 based on a screen operation instruction related to the correction of the exit area by the operator, which is received by the display unit 60. The pre-processing unit 12 may automatically provisionally set the exit area using well-known AI technology or image processing.

[0038] The shape of the exit area E11 may be a rectangle (FIGS. 10(a) to 10(d)), a perfect circle or an ellipse (FIGS. 11(a) to 11(d)), or a semicircle (FIGS. 12(a) to 12(c)). The method for setting the shape of the exit area E11 may be changed depending on the installation location of the camera 20, the use of the first area, etc.

[0039] Setting of the reference coordinate system will now be described. The reference coordinate system here is a coordinate system used when deriving the user's trajectory, and is, for example, a coordinate system with two axes, x and y. The pre-processing unit 12 sets the reference coordinate system for the exit area E11 so as to match the position and shape of the exit area E11. For example, the pre-processing unit 12 may set a reference coordinate system in which the direction of travel of the user in the exit area E11 is the y-coordinate system and the direction perpendicular to the direction of travel is the x-coordinate system. The method for setting such a reference coordinate system may be changed depending on the installation location of the camera 20, the purpose of the first area, etc.

[0040] For example, when a rectangular exit area E11 is set as shown in Figure 10(a), the pre-processing unit 12 may set a reference coordinate system with the midpoint of the bottom side of the exit area E11 as x, y = (0, 0) (see Figure 10(b)), or may set a reference coordinate system with the center point C of the exit area E11 as x, y = (0, 0) (see Figure 10(c)), or may set a reference coordinate system with one of the four corners of the exit area E11 as x, y = (0, 0) (see Figure 10(d)). For example, in the case where an elliptical exit area E11 is set as shown in Figure 11(a), the pre-processing unit 12 may set the reference coordinate system by setting the point at the bottom of the minor axis of the exit area E11 as x, y = (0, 0) (see Figure 11(b)), or by setting the center point C of the exit area E11 as x, y = (0, 0) (see Figure 11(c)), or by setting the reference coordinate system by setting the intersection of the tangent line in the minor axis direction and the tangent line in the major axis direction of the exit area E11 as x, y = (0, 0) (see Figure 11(d)). For example, in the case where a semicircular exit area E11 is set as shown in Figure 12(a), the pre-processing unit 12 may set the reference coordinate system by setting the center of the exit area E11 as x, y = (0, 0) (see Figure 12(b)), or by setting the reference coordinate system by setting a point on a line segment passing through the center of the exit area E11 as x, y = (0, 0) (see Figure 12(c)).

[0041] Such processing by the pre-processing unit 12 may be performed once, for example, when the camera 20 is installed or when the crosswalk E1A is constructed. The processing by the trajectory derivation unit 13, the estimation unit 14, and the control unit 15 described below is performed on the assumption that the processing by the pre-processing unit 12 has been completed.

[0042] The trajectory derivation unit 13 derives a trajectory of the user's movement in a reference coordinate system from multiple detection results of the user in the waiting area E2. The trajectory derivation unit 13 detects the position and signs as feature quantities for each detected user (each pedestrian). The signs refer to information for predicting the detected user's future position. Here, the signs include the detected user's body orientation and facial orientation. Well-known techniques can be used to detect the body orientation and facial orientation, and for example, a detection method using HOG (Histograms of Oriented Gradients) features or Haar-Like features may be used. Alternatively, a detection method combining one of these features with AdaBoost (Adaptive Boosting) may be used. Each of the body orientation and facial orientation is calculated, for example, as a two-dimensional vector in a horizontal plane. The trajectory derivation unit 13 derives the user's trajectory based on the transition of the position in the reference coordinate system for the same user and the above-mentioned signs. The trajectory here includes information on the route the user has taken up to that point, as well as information on the future route the user will take. Such information on the future route is estimated from the above-mentioned body orientation, face orientation, etc. The trajectory derivation unit 13 outputs the derived information (trajectory for each user) to the estimation unit 14.

[0043] The estimation unit 14 estimates whether the user will enter the crosswalk E1 based on the user's trajectory, which is information derived by the trajectory derivation unit 13. Specifically, when the vectors of the users' trajectories point toward the crosswalk E1, such as the trajectories of users Y1 and Y2 shown in FIG. 4, the estimation unit 14 estimates that the users associated with these trajectories will enter the crosswalk E1. On the other hand, when the vectors of the users' trajectories do not point toward the crosswalk E1, the estimation unit 14 estimates that the users associated with these trajectories will not enter the crosswalk E1.

[0044] The estimation by the estimation unit 14 may take into account the learning results of the route prediction. In this case, when trajectory information for each user is input from the trajectory derivation unit 13, the learning unit (not shown) included in the estimation device 10 stores the information for a predetermined period of time. The learning unit then performs learning of the route prediction based on the trajectory information of multiple users. The learning unit may perform learning for each user attribute (e.g., pedestrian user or cyclist user), for each time period in which the trajectory is acquired, or for each season in which the trajectory is acquired. Any learning algorithm may be used, and for example, a neural network, a Markov decision process, a Gaussian process, or the like may be used depending on the model used. The learning unit outputs the learning results to the estimation unit 14. In this case, the estimation unit 14 may apply the learning results input from the learning unit to a predetermined prediction model and input the user's trajectory derived by the trajectory derivation unit 13 into the prediction model (the learned prediction model) to derive a future movement route and estimate whether the user will enter the crosswalk E1. The estimation unit 14 outputs the estimation result to the control unit 15 .

[0045] The control unit 15 controls the external device 30 based on the estimation result by the estimation unit 14. Here, the external device 30 is, for example, a signal control device that controls a push-button signal. The signal control device typically performs signal control to change the vehicle signal for the crosswalk E1 from green to yellow and then to red when a user who wants to cross the crosswalk E1 presses a button to request crossing. In this embodiment, as an alternative to the processing performed by such a push-button signal, when the estimation unit 14 estimates that the user will enter the crosswalk E1, an instruction signal is sent to the signal control device, and the above-mentioned signal control is performed. That is, when the estimation unit 14 estimates that the user will enter the crosswalk E1, the control unit 15 sends an instruction signal to the external device 30 (signal control device) so that the external device 30 (signal control device) performs the above-mentioned signal control. In this case, the external device 30 (signal control device) performs the above-mentioned signal control in response to the instruction signal.

[0046] The control unit 15 may control an external device 30 other than a signal control device based on the estimation result by the estimation unit 14. For example, the control unit 15 may transmit a signal indicating the estimation result by the estimation unit 14 to the external device 30, which is an in-vehicle device of a vehicle traveling near the crosswalk E1. In this case, the external device 30 (in-vehicle device) can use the estimation result by the estimation unit 14 as driving assistance information, for example.

[0047] Next, the pre-processing will be described with reference to the flowchart in Fig. 13. Fig. 13 is a flowchart illustrating the process of setting the exit area and the reference coordinate system. Note that the process described below assumes that the camera 20 is being installed in the target area for the first time.

[0048] 13, first, the camera 20 is installed (step S1). The camera 20 is installed at an arbitrary installation position.

[0049] Next, the operator determines whether the desired detection area (i.e., the crosswalk E1 and waiting area E2) is displayed in the image on the GUI on the display unit 60 (step S2). If it is not displayed, the camera installation position and installation angle are adjusted (step S3).

[0050] On the other hand, when it is determined in step S2 that the detection area is appropriately displayed in the image on the GUI, if the exit area is to be set manually, the display unit 60 accepts a screen operation instruction from the operator to specify the shape and range of the exit area, and the estimation device 10 sets the exit area based on the screen operation instruction (step S4). Also, when the exit area is to be set by AI or image processing, the estimation device 10 automatically sets a tentative exit area by image processing or the like (step S5), the display unit 60 accepts a screen operation instruction from the operator to adjust the shape and range of the exit area, and the estimation device 10 sets the exit area based on the screen operation instruction (step S6).

[0051] Next, the estimation device 10 sets a reference coordinate system for the exit area based on the position and shape of the exit area (step S7). Through the processing up to this point, on-site adjustment in the target area is completed (step S8).

[0052] Next, the approach estimation process executed by the estimation device 10 will be described with reference to the flowchart of Fig. 14. Fig. 14 is a flowchart illustrating the approach estimation process.

[0053] As shown in FIG. 14, the estimation device 10 receives an image capture result from the camera 20, thereby acquiring a detection result of the user Y in the waiting area E2 (step S101).

[0054] Next, the estimation device 10 performs an analysis process on the captured image results, and derives a trajectory of the user's movement in the reference coordinate system (step S102).

[0055] Next, the estimation device 10 estimates whether the user will enter the crosswalk E1 based on the derived user trajectory (step S103). Specifically, the estimation device 10 estimates that the user Y will enter the crosswalk E1 if the vector of the user's trajectory is pointing toward the crosswalk E1, and estimates that the user Y will not enter the crosswalk E1 if the vector of the user's trajectory is not pointing toward the crosswalk E1.

[0056] Then, the estimation device 10 controls the external device 30 based on the above-described estimation result (step S104). Specifically, when it is estimated that the user will enter the crosswalk E1, an instruction signal to change the vehicle signal to red may be transmitted to a signal control device that controls a push-button signal. Also, a signal indicating the above-described estimation result may be transmitted to an in-vehicle device of a vehicle traveling near the crosswalk E1.

[0057] Next, the effects of the approach estimation system 1 according to this embodiment will be described.

[0058] The entry estimation system 1 includes a camera 20 having a detection range that includes a crosswalk E1, which is a predetermined destination area, and a waiting area E2, which a user entering the crosswalk E1 passes through immediately before entering the crosswalk E1, and an estimation device 10 configured to perform pre-processing related to estimation and an estimation process that estimates whether a detected user will enter the crosswalk E1 based on the results of the pre-processing and the detection result of the user by the camera 20 in the waiting area E2. In the pre-processing, the estimation device 10 sets an exit area E11, which is the area that a user entering the crosswalk E1 from the waiting area E2 will first pass through on the crosswalk E1, based on the detection result by the camera 20, and sets a reference coordinate system related to the exit area E11 based on the position and shape of the exit area E11. In the estimation process, the estimation device 10 derives a trajectory of the user's movement in the reference coordinate system from multiple detection results of the user in the waiting area E2, and estimates whether the detected user will enter the crosswalk E1 based on the trajectory.

[0059] In this approach estimation system 1, in a pre-processing step, an exit area E11 is set on the crosswalk E1 based on the detection results from the camera 20, and a reference coordinate system for the exit area E11 is set taking into account the position and shape of the exit area E11. Then, in the estimation process, the approach estimation system 1 derives a trajectory of the user's movement based on the reference coordinate system, and estimates whether the user will enter the crosswalk E1 based on the trajectory. In this way, by deriving the user's trajectory based on a reference coordinate system that is set in advance taking into account the position and shape of the exit area E11, the user's trajectory can be derived under the same conditions regardless of differences in each exit area E11. For example, the trajectory may look different depending on the installation conditions of the camera 20, the shape of the road surrounding the crosswalk E1, etc. Therefore, it is usually necessary to prepare estimation logic (an estimation result corresponding to the trajectory) for each installation condition of the camera 20, the shape of the road, etc., which increases the preparation burden. In this regard, by setting a reference coordinate system for each exit area E11 as described above, it becomes possible to derive a user's trajectory under the same conditions as long as the setting rules are standardized. This eliminates the need to prepare estimation logic (estimated results corresponding to the trajectory) for each camera 20 installation condition, road shape condition, etc., and makes it easier to prepare estimation logic.

[0060] In the entry estimation system 1, the estimation device 10 may set a reference coordinate system in advance, in which the direction of travel of the user in the exit area E11 is the y-coordinate system and the direction perpendicular to the direction of travel is the x-coordinate system. With this configuration, the reference coordinate system can be set simply and appropriately using a common rule that takes into account the shape of the exit area E11.

[0061] The entry estimation system 1 further includes a display unit 60 that displays the detection results from the camera 20 and can receive screen operation instructions from an operator related to the detection results, and the estimation device 10 may set the position and shape of the exit area E11 in pre-processing based on the screen operation instructions from the operator related to the detection results received by the display unit 60. With this configuration, the exit area E11 can be easily and appropriately set based on the screen operation by the operator.

[0062] In the entry estimation system 1, the estimation device 10 may set a tentative exit area by analyzing the detection results in pre-processing, and then set the position and shape of the exit area E11 after the setting based on screen operation instructions related to modification of the tentative exit area by the operator, which are received by the display unit 60. In such a configuration, the tentative exit area is set automatically by, for example, AI or image processing, etc., which simplifies the screen operation instructions (instructions related to modification) by the operator, making it possible to set the exit area E11 more easily.

[0063] In the entry estimation system 1, the first area may be an area including the crosswalk E1, and the second area may be a waiting area E2 on a sidewalk adjacent to the crosswalk E1. With this configuration, it is possible to estimate with high accuracy whether a user on the sidewalk adjacent to the crosswalk E1 is a user entering the crosswalk E1.

[0064] Although various embodiments and modifications according to the present disclosure have been described above, the present disclosure is not limited to the exemplified embodiments.

[0065] For example, although the camera 20 is used as an example of the detector, the detector may be configured other than the camera 20 as long as it can detect a user. Specifically, the detector may be a detection mechanism such as a laser or lidar. Here, a detector such as a laser or lidar obtains detection results as reflection points of radio waves or light, and therefore cannot identify the exit area at a glance, as can be done with a captured image. Setting of the exit area and the reference coordinate system when using such a detector such as a laser or lidar will be described with reference to FIGS. 15 and 16 .

[0066] Fig. 15 is a flowchart illustrating the process of setting the exit area and the reference coordinate system according to the modified example, and Fig. 16 is a diagram illustrating the walking detection process (described later) for setting the exit area.

[0067] 15, first, a sensor (laser or lidar) is installed (step S201). The sensor is installed at an arbitrarily set installation position.

[0068] Next, it is determined whether the desired detection area (i.e., the crosswalk E1 and waiting area E2) is within the sensor field of view on the display unit 60 (step S202). If it is not within the field of view, the sensor installation position and installation angle are adjusted (step S203).

[0069] On the other hand, if it is determined in step S202 that the detection area is appropriately within the sensor field of view, the worker walks through the area Ex where the exit area is to be set, as shown in Fig. 16(a) (step S204). When the sensor detects the worker's walking, a detection point D detected by the sensor appears along the path of the worker's walking, as shown in Fig. 16(b). Based on this detection result, the exit area E11 is set on the GUI (step S205).

[0070] Next, the estimation device 10 sets a reference coordinate system for the exit area based on the position and shape of the exit area (step S206). Through the processing up to this point, on-site adjustment in the target area is completed (step S207).

[0071] Although the example in which the first area, which is the predetermined destination area, is a crosswalk has been described, the present invention is not limited to this, and the first area may be an area other than a crosswalk, specifically, a road other than a crosswalk or an area at the entrance to a predetermined facility. Even in this case, the second area may be an area adjacent to the first area and an area through which the user passing before entering the first area passes.

[0072] 17 , a restricted zone DZ such as a construction site or a dangerous area may be set as the first area. An entrance into the restricted zone DZ may then be set as an exit area E101, and a reference coordinate system corresponding to the exit area E101 may be set. By setting the restricted zone DZ as the first area and setting its entrance as the exit area E101 in this way, it is possible to estimate (detect) whether a person is about to enter the restricted zone DZ, which is a dangerous place, and to prevent entry in advance.

[0073] 18 , for example, an exit from the premises of a nursing home may be set as the first area, and an area where a door 500 opens and closes at the exit may be set as an exit area E201. With such a configuration, it is possible to detect and prevent, for example, an elderly person from unintentionally exiting through the door 500 in advance. Furthermore, the functions of the estimation device 10 described above may be allotted to a server device configured to be able to communicate with the camera 20, or may be allocated separately to the server device and a roadside device that communicates with the server device, or may be allotted to the roadside device.

[0074] The various embodiments and modifications described above may be combined as appropriate without departing from the spirit of the present disclosure.

[0075] REFERENCE SIGNS LIST 1... Entry estimation system 10... Estimation device (control device) 11... Acquisition unit 12... Pre-processing unit 13... Trajectory derivation unit 14... Estimation unit 15... Control unit 20... Camera (detection unit) 30... External device 50... Pillar 51... Vertical extension unit 52... Support unit 60... Display unit 120... Circuit 121... Processor 122... Memory 123... Storage 124... Input / output port E1... Crosswalk E2... Waiting area

Claims

1. An entry estimation system comprising: a detection unit whose detection range is a first area which is a specified destination area, and a second area which a user passes through before entering the first area; and a control unit which performs pre-processing related to estimating whether or not the user has entered the first area, wherein the control unit, in the pre-processing, sets an exit area which is the first area in the first area that a user passing through when entering the first area from the second area based on the detection result by the detection unit, and sets a reference coordinate system related to the exit area based on the position and shape of the exit area.

2. The entry estimation system of claim 1, wherein the control device further executes an estimation process after the pre-processing is completed, and in the estimation process, derives a trajectory of the user's movement in the reference coordinate system from multiple detection results of the user in the second area, and estimates whether the detected user will enter the first area based on the trajectory.

3. An entry estimation system as described in claim 1 or claim 2, wherein the control device, in the pre-processing, sets the reference coordinate system in which the user's traveling direction in the exit area is the y-coordinate system and a direction perpendicular to the traveling direction is the x-coordinate system.

4. An entry estimation system as described in any one of claims 1 to 3, further comprising a display unit capable of displaying the detection results by the detection unit and receiving screen operation instructions from an operator related to the detection results, wherein the control device, in the pre-processing, sets the position and shape of the exit area based on the screen operation instructions from the operator related to the detection results received by the display unit.

5. An entry estimation system as described in claim 4, wherein the control device, in the pre-processing, sets a tentative exit area by analyzing the detection results, and sets the position and shape of the modified exit area based on screen operation instructions by the operator to modify the tentative exit area received by the display unit after the setting.

6. An entry estimation system as described in any one of claims 1 to 5, wherein the first area is an area including a crosswalk, and the second area is a waiting area on a sidewalk through which a user passes before entering the crosswalk.

7. An entry estimation method executed by an entry estimation system, comprising the steps of: setting an exit area, which is the first area in the first area that a user passing through before entering the first area, based on detection results relating to a first area, which is a predetermined destination area, and a second area through which the user passes before entering the first area; and setting a reference coordinate system relating to the exit area based on the position and shape of the exit area.

8. The entry estimation method according to claim 7, further comprising a step of deriving a trajectory of the user's movement in the reference coordinate system from multiple detection results of the user in the second area, and estimating whether the detected user will enter the first area based on the trajectory.

9. A control device that performs pre-processing related to estimation of whether a user has entered a first area, which is a specified destination area, and obtains detection results from a detection unit whose detection range is the first area and a second area that the user passes through before entering the first area, and in the pre-processing, based on the detection results by the detection unit, sets an exit area, which is the first area in the first area that a user passing through when entering the first area from the second area, and sets a reference coordinate system related to the exit area based on the position and shape of the exit area.

Images