Information processing device, information processing method, program, and information processing system

Abstract

To solve the problem that capability of determining with high accuracy whether or not the whole or part of a mobile body exists in the range of measurement by a sensor is desired.SOLUTION: Provided is an information processing device comprising: a point cloud generation unit for acquiring a time including a timing at which the whole or part of a first mobile body belongs to the range of measurement by a sensor and generating a second point cloud by superposing on the time a first point cloud in the range of measurement having been obtained by the sensor; and a determination unit for determination whether or not the whole or part of a second mobile body exists in the range of measurement at a prescribed time on the basis of the second point cloud and a third point cloud in the range of measurement having been obtained by the sensor at a prescribed time.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and an information processing system. [Background technology]

[0002] In recent years, technology for detecting objects based on data obtained by sensors has become known. There are various types of sensors. For example, there are sensors that repeatedly scan light or radio waves two-dimensionally from a starting point to an end point to continuously obtain scanning results (hereinafter also referred to as "frames"). A set of detection positions of light or radio waves in a frame may correspond to a point cloud. Here, a point cloud indicates the distance from the sensor to an object for each two-dimensional coordinate. Therefore, by using a point cloud, it is possible to detect objects existing in real space three-dimensionally.

[0003] LiDAR (Laser Imaging Detection and Ranging) is known as an example of a sensor. For example, a technology for detecting an object based on a point cloud obtained by LiDAR has been disclosed (see, for example, Patent Document 1). As an example of an object, a moving object may also be detected by a sensor. A technology for determining whether or not a part or the whole of a moving object is present within a measurement range of a sensor is also known. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-219248 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is desirable to determine with high accuracy whether or not a part or the whole of a moving object is present in a measurement range of a sensor.

[0006] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a technology that can determine with high accuracy whether or not part or all of a moving object is present within a measurement range of a sensor. [Means for solving the problem]

[0007] In order to solve the above problems, according to one aspect of the present invention, an information processing device is provided, comprising: a point cloud generation unit that acquires a time including the timing when part or all of a first moving body falls within a measurement range of a sensor, and generates a second point cloud by overlaying a first point cloud of the measurement range acquired by the sensor at said time; and a determination unit that determines whether part or all of a second moving body is present in the measurement range at a specified time based on the second point cloud and a third point cloud of the measurement range acquired by the sensor at the specified time.

[0008] The determination unit may include a point cloud extraction unit that extracts a difference point cloud by subtracting the third point cloud from the second point cloud, and a moving body presence / absence determination unit that determines whether or not part or all of the second moving body is present based on the difference point cloud.

[0009] The moving object presence / absence determination unit may determine that part or all of the second moving object is present when the number of points included in the difference point cloud is smaller than a first threshold value, and may determine that part or all of the second moving object is not present when the number of points included in the difference point cloud is greater than or equal to the first threshold value.

[0010] The moving object presence / absence determination unit may determine that the leading or trailing portion of the second moving object is within the measurement range when the number of points included in the difference point cloud is greater than or equal to a second threshold value that is smaller than the first threshold value and is smaller than the first threshold value, and may determine that neither the leading nor trailing portion of the second moving object is within the measurement range when the number of points included in the difference point cloud is smaller than the second threshold value.

[0011] When the number of points included in the difference point cloud is greater than or equal to a second threshold value that is smaller than the first threshold value and is smaller than the first threshold value, the moving body presence / absence determination unit may determine whether the leading portion or the trailing portion of the second moving body is within the measurement range based on the number of points included in a first range of the difference point cloud and the number of points included in a second range of the difference point cloud.

[0012] The moving object presence / absence determination unit may determine that part or all of the second moving object is not present when the difference between the number of points included in the second point cloud and the number of points included in the difference point cloud is smaller than a first threshold, and may determine that part or all of the second moving object is present when the difference is greater than or equal to the first threshold.

[0013] The information processing device may include an object detection unit that detects a first object based on the difference point cloud.

[0014] The information processing device may include a display control unit that controls display of display information according to a position of the first object by a display device.

[0015] The display control unit may change a display mode of the display information based on whether or not a part or all of the second moving object is present.

[0016] The information processing device may include an event detection unit that attempts to detect an event that the first object is present within a predetermined range based on a position of the first object.

[0017] The event detection unit may control, based on the detection of the event, a notification device to notify the user of the detection of the event.

[0018] The event detection unit may change the predetermined range based on whether or not a part or all of the second moving object is present.

[0019] The event detection unit may attempt to detect the event based on a determination that a part or all of the second moving object is present.

[0020] The information processing device may include an object tracking unit that performs a tracking process to associate the position of the first object with the position of the second object based on the position of the first object and the position of the second object detected from a past point cloud.

[0021] The information processing device may include a display control unit that controls display, by a display device, of a movement trajectory connecting a position of the first object and a position of the second object.

[0022] The display control unit may change a display mode of the movement trajectory based on whether or not a part or an entirety of the second moving object is present.

[0023] Each of the first moving object and the second moving object may be a person riding an escalator, a train, a ship, or a car.

[0024] In addition, according to another aspect of the present invention, in order to solve the above-mentioned problem, there is provided an information processing method executed by a computer, which includes acquiring a time including a timing when part or all of a first moving body falls within a measurement range by a sensor, generating a second point cloud by overlaying a first point cloud of the measurement range acquired by the sensor at said time, and determining whether part or all of a second moving body is present in the measurement range at a specified time based on the second point cloud and a third point cloud of the measurement range acquired by the sensor at the specified time.

[0025] In addition, according to another aspect of the present invention, in order to solve the above-mentioned problems, a program is provided that causes a computer to function as: a point cloud generation unit that acquires a time including the timing when part or all of a first moving body falls within a measurement range measured by a sensor, and generates a second point cloud by overlaying a first point cloud of the measurement range obtained by the sensor at said time; and a determination unit that determines whether part or all of a second moving body is present in the measurement range at a specified time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the specified time.

[0026] In addition, according to another aspect of the present invention, in order to solve the above-mentioned problems, an information processing system is provided, comprising: a sensor that obtains a point cloud; a point cloud generation unit that obtains a time including the timing when part or all of a first moving body falls within a measurement range of the sensor, and generates a second point cloud by overlaying the first point cloud of the measurement range obtained by the sensor at the time; and a determination unit that determines whether part or all of a second moving body is present in the measurement range at a specified time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the specified time. Effect of the Invention

[0027] As described above, according to the present invention, a technique is provided that can determine with high accuracy whether or not a part or the whole of a moving object is present within a measurement range of a sensor. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 is a diagram for explaining an example of human detection using a general background subtraction technique. [Diagram 2] 1 is a block diagram showing an example of a functional configuration of an information processing system according to an embodiment of the present invention. [Diagram 3] 13 is a flowchart showing an example of the operation of the information processing device 10 in a preparatory stage. [Figure 4]13 is a diagram showing an example of a background trajectory point cloud created by a background trajectory point cloud creating unit 120. FIG. [Diagram 5] 13 is a flowchart showing an example of the operation of the information processing device 10 in the operation stage. [Figure 6] FIG. 13 is a diagram for explaining an example of a difference point group when a vehicle is not on the line; [Figure 7] FIG. 13 is a diagram for explaining an example of a difference point cloud when the vehicle is on the line; [Figure 8] FIG. 13 is a diagram showing an example of a display of a movement trajectory of a person when the person is on the train. [Figure 9] FIG. 13 is a diagram showing an example of a display of a movement trajectory of a person when not present on the train. [Figure 10] FIG. 11 is a diagram for explaining a first modified example. [Figure 11] FIG. 11 is a diagram for explaining a second modified example. [Figure 12] FIG. 11 is a diagram for explaining a second modified example. [Figure 13] FIG. 1 is a diagram showing a hardware configuration of an information processing device 900 as an example of the information processing device 10 according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals, and duplicated explanations will be omitted.

[0030] <0.Background> First, the background of the embodiment of the present invention will be described.

[0031] In recent years, a technique for detecting an object based on data obtained by a sensor has been known. As an example of an object, a moving object may be detected by a sensor. For example, a technique for determining whether or not a part or the whole of a moving object is present within a measurement range of a sensor is also known. However, it is desired to determine with high accuracy whether or not a part or the whole of a moving object is present within a measurement range of a sensor.

[0032] In this specification, it is mainly assumed that the moving object whose presence or absence in the measurement range of the sensor is determined is a train. However, the moving object whose presence or absence in the measurement range of the sensor is determined is not limited to a train. For example, the moving object whose presence or absence in the measurement range of the sensor is determined may be a ship, a car, a person riding an escalator, or the like.

[0033] In this specification, the presence or absence of a part or all of a train within the sensor's measurement range is referred to as "on the line," and the absence or absence of a part or all of a train within the sensor's measurement range is referred to as "not on the line." The reason "part of a train" is used here is because a train is not always made up of a single car, but can be made up of multiple cars connected together, and it is fully expected that the entire train will not fit within the sensor's measurement range.

[0034] In addition, in this specification, it is assumed that whether or not a part or the whole of a moving body is present within a measurement range of a sensor is mainly used to reduce the possibility of contact between the moving body and an object. An object may also be detected using a sensor. In this specification, it is mainly assumed that the object detected by the sensor is a person. However, the object detected by the sensor is not limited to a person. For example, the object detected by the sensor may be an animal other than a human, a robot, or the like.

[0035] In the following explanation, we consider reducing the possibility of contact between trains and people at railway stations. Here, it is essential to take safety measures on railway station platforms (hereinafter simply referred to as "platforms") to prevent situations in which people fall or get trapped in the gap between the train and the platform while getting on or off the train.

[0036] An example of a safety measure on platforms is the installation of platform doors. However, the structure of the platform or the weight of the platform doors may impose restrictions on the installation of the platform doors. In addition, the installation of platform doors requires large-scale construction work and is very costly, so there is a demand for easier and less costly safety measures.

[0037] As an easier and lower-cost safety measure, a sensor may be installed on the platform and used to detect the position of people on or around the platform. An example of the sensor is a laser sensor. Here, a background subtraction technique may be used as an example of a method for detecting people using a laser sensor. An example of detecting people using a general background subtraction technique will be described with reference to FIG. 1.

[0038] Fig. 1 is a diagram for explaining an example of human detection using a general background subtraction technique. Referring to Fig. 1, there are shown an input point cloud P1 obtained in real time by a sensor, a background point cloud P2 prepared in advance, and a difference point cloud P3 obtained by subtracting the background point cloud P2 from the input point cloud P1. In Fig. 1, a minus sign (-) may mean subtracting the following point cloud from the previous point cloud.

[0039] The input point cloud P1 includes a platform point cloud P11, which is a point cloud obtained from the platform, a train point cloud P12, which is a point cloud obtained from the train, and a person point cloud P13, which is a point cloud obtained from people. In the embodiment of the present invention, the moving direction of the train is assumed to be from bottom to top of the drawing unless otherwise specified. That is, the upstream of the moving direction of the train is the downward direction of the drawing, and the downstream of the moving direction of the train is the upward direction of the drawing, and the train point cloud P12 includes a point cloud P121 of the downstream vehicle and a point cloud P122 of the upstream vehicle.

[0040] As an example, if the point group to be subtracted is A and the point group to be subtracted is B, subtracting point group B from point group A may mean excluding from point group A any point included in point group A whose distance to any point included in point group B is less than a threshold value.

[0041] When the background subtraction technique is used, a background point cloud P2 prepared in advance is subtracted from an input point cloud P1 obtained by a sensor to extract a difference point cloud P3, and a person is detected from the difference point cloud P3. In other words, a person can be detected from the input point cloud P1 that is not included in the background point cloud P2.

[0042] Generally, the background point cloud P2 is created by overlaying the point clouds of multiple frames obtained by the sensor so that a certain degree of point cloud fluctuation is reflected in the background point cloud during times when there are no moving objects within the sensor's measurement range.

[0043] However, as shown in Fig. 1, if the background point cloud P2 is created by superimposing point clouds from a time period when neither people nor trains are present in the sensor measurement range, the person point cloud P13 and train point cloud P12 remain in the difference point cloud P3 obtained by subtracting the background point cloud P2 from the input point cloud P1. In other words, there is a possibility that both people and trains will be detected from the difference point cloud P3. Furthermore, a huge processing load is required to process the point cloud obtained from a large train.

[0044] On the other hand, a method of excluding the train point cloud P12 by performing mask processing on the train point cloud P12 may be considered. However, since the target of the mask processing is generally specified by a straight line, in cases where the platform is curved, it may be difficult to perform mask processing on the train point cloud P12 along the curved platform.

[0045] Even if the train point cloud P12 is removed by masking, it becomes difficult to determine whether a train is on the line or not. For example, locations that are dangerous to people may differ depending on whether a train is on the line or not, so it is also important to determine whether a train is on the line or not with high accuracy. Therefore, this specification mainly proposes a technology for determining whether a train is on the line or not with high accuracy.

[0046] The background of the embodiment of the present invention has been described above.

[0047] <1. Details of the embodiment> Next, the details of the embodiment of the present invention will be described.

[0048] [1-1. Example of information processing system configuration] First, a functional configuration example of an information processing system according to an embodiment of the present invention will be described. Fig. 2 is a block diagram showing a functional configuration example of an information processing system according to an embodiment of the present invention. As shown in Fig. 2, an information processing system 1 according to an embodiment of the present invention includes an information processing device 10, a sensor 100, a display device 190, and a notification device 191. The information processing device 10, the sensor 100, the display device 190, and the notification device 191 may be connected via a network.

[0049] (Sensor 100) The sensor 100 is installed on or near a platform. The sensor 100 has a function of acquiring a platform point cloud, a train point cloud, and a person point cloud. Here, it is mainly assumed that the sensor 100 is realized by a laser sensor. However, the sensor 100 is not limited to a laser sensor. The laser sensor can also be referred to as a "LiDAR sensor."

[0050] Note that various sensors other than a laser sensor (e.g., a distance measuring sensor) may be used as the sensor 100. For example, the sensor 100 may be a radar (e.g., a millimeter wave radar) that uses electromagnetic waves instead of laser light. Alternatively, the sensor 100 may be a sonar that uses sound waves instead of laser light.

[0051] (Information processing device 10) The information processing device 10 may be realized by a computer. As shown in Fig. 2, the information processing device 10 includes a point cloud acquisition unit 110, a background trajectory point cloud creation unit 120, a background trajectory point cloud storage unit 130, a determination unit 135, an object detection unit 150, a past frame detected object information storage unit 160, an object tracking unit 170, a display control unit 181, and an event detection unit 182. The determination unit 135 includes a point cloud extraction unit 140 and an on-rail determination unit 180 (a moving object presence / absence determination unit). Details of these configurations included in the information processing device 10 will be described later.

[0052] For example, the point cloud acquisition unit 110, the background trajectory point cloud creation unit 120, the determination unit 135, the object detection unit 150, the object tracking unit 170, the display control unit 181, and the event detection unit 182 may be realized by a control unit (not shown). On the other hand, the background trajectory point cloud storage unit 130 and the past frame detected object information storage unit 160 may be realized by a storage unit (not shown).

[0053] The control unit (not shown) includes a CPU (Central Processing Unit) and the like, and its functions can be realized by the CPU expanding a program stored in a non-volatile storage device into a RAM (Random Access Memory) and executing it. At this time, a computer-readable recording medium on which the program is recorded can also be provided. Alternatively, the control unit (not shown) may be configured with dedicated hardware, or may be configured with a combination of multiple hardware components.

[0054] The storage unit (not shown) is a storage device capable of storing programs and data for operating the control unit (not shown). The storage unit (not shown) can also temporarily store various data required in the process of operation of the control unit (not shown). For example, the storage device may be a non-volatile storage device.

[0055] (Display device 190) The display device 190 is configured with a display. For example, the display device 190 may display display information according to the position of a person detected by the information processing device 10 in accordance with the control of the information processing device 10. The display information according to the position of the person may include a frame (object detection frame) surrounding the person. Alternatively, the display device 190 may display a movement trajectory connecting multiple positions of the same person associated by a tracking process in accordance with the control of the information processing device 10.

[0056] There is no particular limitation on the location where the display device 190 is installed. For example, the display device 190 may be installed in a station office where station staff are present, or in a monitoring center where staff who monitor the train operation status are present.

[0057] (Notification device 191) The notification device 191 notifies that an event has been detected by the information processing device 10 in accordance with the control of the information processing device 10. Here, it is mainly assumed that the event is a person falling from a platform. For example, the notification device 191 may be configured to include a lamp, and may notify that an event has been detected by turning on the lamp. Alternatively, the notification device 191 may be configured with a monitor, and may notify that an event has been detected by displaying a predetermined color (e.g., red) on the monitor screen.

[0058] There is no particular limitation on the location where the notification device 191 is installed. For example, the notification device 191 may be installed at the home or at a monitoring center.

[0059] An example of the functional configuration of the information processing system 1 according to the embodiment of the present invention has been described above.

[0060] [1-2. Example of operation of information processing device] Next, an operation example of the information processing device 10 according to the embodiment of the present invention will be described. The operation example of the information processing device 10 is mainly divided into two stages. The first stage is a preparatory stage. The second stage is an operational stage. Hereinafter, an operation example of the information processing device 10 relating to the preparatory stage will be described with reference to Figs. 3 and 4.

[0061] (Preparation stage) 3 is a flowchart showing an example of the operation of the information processing device 10 in the advance preparation stage. In the advance preparation stage, the background trajectory point cloud creation unit 120 acquires a time (hereinafter also referred to as "on-track time") including a timing when a part or the whole of a train (first moving object) falls within the measurement range of the sensor 100. Note that the measurement range of the sensor 100 may be the entire range measurable by the sensor 100, or may be a part of the range measurable by the sensor 100.

[0062] For example, the on-rail time may be the time from when the front part of the train enters the measurement range of the sensor 100 to when the rear part of the train exits the measurement range of the sensor 100. Note that the on-rail time may include a time when a part or the whole of the train does not belong to the measurement range of the sensor 100. A plurality of frames are output from the sensor 100 to the information processing device 10 continuously in a time series.

[0063] The background trajectory point cloud creation unit 120 acquires a point cloud (first point cloud) of a measurement range acquired by the sensor 100 during the train's presence on the line from the point cloud acquisition unit 110. Then, the background trajectory point cloud creation unit 120 generates a background trajectory point cloud (second point cloud) by superimposing the acquired point cloud of the measurement range (S11). Of the background trajectory point clouds, the point cloud acquired from the platform is the background point cloud, and the point cloud acquired from the train is the trajectory point cloud. The background trajectory point cloud creation unit 120 stores the created background trajectory point cloud in the background trajectory point cloud storage unit 130.

[0064] More specifically, if the point cloud acquired by the point cloud acquisition unit 110 at time u is denoted by pc(u), the time when the front part of the train enters the measurement range of the sensor 100 is denoted by u1, and the time when the rear part of the train exits the measurement range of the sensor 100 is denoted by u2, the background trajectory point cloud pc_traj from time u1 to time u2 can be created as shown in the following equation (1).

[0065]

number

[0066] Fig. 4 is a diagram showing an example of a background trajectory point cloud created by the background trajectory point cloud creation unit 120. Referring to Fig. 4, a measurement range R0 by the sensor 100 is shown. Also shown is a background trajectory point cloud P4 created by the background trajectory point cloud creation unit 120. The background trajectory point cloud P4 includes a platform point cloud P41 in which a point cloud obtained from the platform from time u1 to time u2 is superimposed, and a train point cloud P42 in which a point cloud obtained from the train from time u1 to time u2 is superimposed.

[0067] Here, the trajectory point cloud creation unit 120 may acquire the on-line time in any manner. For example, the point cloud acquired by the point cloud acquisition unit 110 may be displayed on a display device used by the worker, and the worker may determine the on-line time while visually checking the displayed point cloud, and input the on-line time into an input device used by the worker. In this case, the background trajectory point cloud creation unit 120 may acquire the on-line time input by the worker.

[0068] Alternatively, the background trajectory point cloud creation unit 120 may automatically acquire the on-track time without manual operation. For example, when a train detection sensor installed around the track detects the time when a train arrives at a predetermined position on the track and the detected time is acquired by the information processing device 10 from the train detection sensor, the background trajectory point cloud creation unit 120 may predict the on-track time from the detection time.

[0069] An example of the operation of the information processing device 10 in the advance preparation stage has been described above with reference to FIGS.

[0070] (Operational stage) Next, an example of the operation of the information processing device 10 in the operation stage will be described with reference to FIGS.

[0071] Fig. 5 is a flowchart showing an example of the operation of the information processing device 10 in the operation stage. As shown in Fig. 5, in the operation stage, the point cloud extraction unit 140 acquires a background trajectory point cloud from the background trajectory point cloud storage unit 130. Furthermore, the point cloud acquisition unit 110 acquires a point cloud (third point cloud) at the current time (predetermined time) of the measurement range acquired by the sensor 100 from the sensor 100 as an input point cloud, and the point cloud extraction unit 140 acquires the input point cloud (S21). The determination unit 135 determines whether a vehicle is present on a line or not based on the background trajectory point cloud and the input point cloud.

[0072] First, the point cloud extraction unit 140 extracts a difference point cloud based on the background trajectory point cloud and the input point cloud (S22). More specifically, the point cloud extraction unit 140 extracts the difference point cloud by subtracting the input point cloud from the background trajectory point cloud. When subtracting the input point cloud from the background trajectory point cloud, the point cloud extraction unit 140 may appropriately rotate the coordinate axis serving as the reference for the background trajectory point cloud and the coordinate axis serving as the reference for the input point cloud in order to facilitate the calculation of subtracting the input point cloud from the background trajectory point cloud.

[0073] The object detection unit 150 detects an object (first object) based on the difference point cloud (S23). For example, the object detection unit 150 may detect the object by a clustering process. Note that the object detected by the object detection unit 150 is obtained as a three-dimensional point cloud of the object. Also, the object detected by the object detection unit 150 is mainly a person.

[0074] The object tracking section 170 associates the detected object in the current frame with the detected object in the previous frame (S24). More specifically, the object tracking section 170 acquires the position of the object in the previous frame from the past frame detected object information storage section 160.

[0075] Furthermore, the object tracking unit 170 detects the position of the object at the current time (the position of the object in the current frame) based on the detection result of the object. The position of the object detected by the object tracking unit 170 may be any position of the object. For example, the object tracking unit 170 may detect the position of the center of gravity of the three-dimensional point cloud of the object as the position of the object, or may detect a position estimated based on the position of the three-dimensional point cloud of the object and the speed of the object as the position of the object.

[0076] When the distance between the object position in the previous frame and the object position in the current frame is equal to or less than a threshold, the object tracking unit 170 stores the object position in the current frame in the past frame detected object information storage unit 160 in association with the object position in the previous frame that is closest to the object position in the current frame. For example, the distance between the object position in the previous frame and the object position in the current frame may be the Euclidean distance. The previous frame may also be referred to as a frame that is earlier than the current frame (past frame).

[0077] On the other hand, when the distance between the object's position in the previous frame and the object's position in the current frame exceeds a threshold, the object tracking unit 170 stores the object's position in the current frame in the past frame detected object information memory unit 160 without corresponding the object's position in the previous frame.

[0078] Next, if the predetermined time has not passed ("NO" in S25), the operation proceeds to S28. On the other hand, if the predetermined time has passed ("YES" in S25), the on-rail determination unit 180 determines whether the vehicle is on-rail or not based on the difference point group (S26). Note that the predetermined time may arrive at a predetermined time interval. The predetermined time interval may be a short period of time such as a few ms or a few seconds, or a long period of time such as a few days or a few months.

[0079] Fig. 6 is a diagram for explaining an example of a difference point cloud when a train is not present on the line. Referring to Fig. 6, a background trajectory point cloud P4 and an input point cloud P5 when a train is not present on the line are shown. When a train is not present on the line, there is no train in the measurement range of the sensor 100, so the input point cloud P5 when a train is not present on the line includes a platform point cloud P11, which is a point cloud obtained from the platform, and a person point cloud P13, which is a point cloud obtained from people, but does not include a train point cloud, which is a point cloud obtained from the train.

[0080] On the other hand, since the background trajectory point cloud P4 includes a train point cloud P42 which is an overlay of point clouds obtained from the train, the difference point cloud P6 obtained by subtracting the input point cloud P5 when the train is not on the line from the background trajectory point cloud P4 contains a large number of points.

[0081] Fig. 7 is a diagram for explaining an example of a difference point cloud when the train is on the line. Referring to Fig. 7, a background trajectory point cloud P4 and an input point cloud P7 when the train is on the line are shown. When the train is on the line, a train is present within the measurement range of the sensor 100, so the input point cloud P7 when the train is on the line includes not only a platform point cloud P11, which is a point cloud obtained from the platform, and a person point cloud P13, which is a point cloud obtained from people, but also a train point cloud P12, which is a point cloud obtained from the train.

[0082] On the other hand, the background trajectory point cloud P4 also includes a train point cloud P42 which is an overlay of point clouds obtained from the train, so the difference point cloud P8 obtained by subtracting the input point cloud P7 when the train is on the line from the background trajectory point cloud P4 contains a small number of points.

[0083] Therefore, the on-track determination unit 180 may determine that the train is not on the track when the number of points included in the difference point cloud is equal to or greater than the first threshold value. On the other hand, the on-track determination unit 180 may determine that the train is on the track when the number of points included in the difference point cloud is smaller than the first threshold value.

[0084] More specifically, if the input point cloud is pc_input, the difference point cloud is pc_invert, and, as above, the background trajectory point cloud is pc_traj, the relationship between the difference point cloud pc_invert, the background trajectory point cloud pc_traj, and the input point cloud pc_input is as shown in the following equation (2).

[0085] pc_invert=pc_traj-pc_input (2)

[0086] In addition, if the function that takes a point cloud as input and outputs the number of points contained in the point cloud is count() and the first threshold is Th_pc, the condition for determining that a train is on the line is as shown in the following equation (3).

[0087] count(pc_invert) <Th_pc (3)

[0088] The display control unit 181 controls the display of the determination result of whether the vehicle is on the line or not on the line by the display device 190 (S27). Furthermore, the display control unit 181 controls the display of the movement trajectory of the person detected by the object tracking unit 170 by the display device 190 (S28). At this time, the display control unit 181 may control the display of display information according to the position of the person detected by the object tracking unit 170 by the display device 190. The display information according to the position of the person may include a frame surrounding the person (person detection frame).

[0089] Furthermore, the display control unit 181 may control the display by the display device 190 of a fall detection range, which is a range (predetermined range) for detecting a person falling from the platform. In this case, when a train is present on the tracks, a train is present on the tracks, but when a train is not present on the tracks, the range in which a person may fall from the platform may differ between when a train is present and when a train is not present. Thus, the event detection unit 182 may set the fall detection range to be different when a train is present and when a train is not present. Alternatively, the difference in the fall detection range between when a train is present and when a train is not present may include a case in which a fall detection range is provided only when a train is present and no fall detection range is provided when a train is not present.

[0090] Fig. 8 is a diagram showing an example of display of a movement trajectory of a person when the train is on the line. Referring to Fig. 8, the display control unit 181 controls the display device 190 to display that the train is on the line, the platform point group P11, and the fall detection range r1 from time t0 to t2. Furthermore, the display control unit 181 controls the display device 190 to display the movement trajectory T0 of the person at time t0, controls the display device 190 to display the movement trajectory T1 of the person at time t1 after time t0, and controls the display device 190 to display the movement trajectory T2 of the person at time t2 after time t1.

[0091] Fig. 9 is a diagram showing an example of display of a person's movement trajectory when a train is not present on the line. Referring to Fig. 9, the display control unit 181 controls the display device 190 to display that a train is not present on the line, the platform point group P11, and the fall detection range r2 from time t0 to t2. Furthermore, the display control unit 181 controls the display device 190 to display the person's movement trajectory T0 at time t0, controls the display device 190 to display the person's movement trajectory T1 at time t1 after time t0, and controls the display device 190 to display the person's movement trajectory T2 at time t2 after time t1.

[0092] The display control unit 181 may differentiate the display mode of the frame of the fall detection range r1 (FIG. 8) when a train is present on the rail and the display mode of the frame of the fall detection range r2 (FIG. 9) when a train is not present on the rail. In this case, the display mode of the fall detection range that the display control unit 181 differentiates may be the thickness of the line of the frame, the color of the line of the frame, or the like.

[0093] Furthermore, the display control unit 181 may differentiate the display mode of the movement trajectory T0-T2 of the person when on the line (FIG. 8) from the display mode of the movement trajectory T0-T2 of the person when on the line (FIG. 9). In this case, the display mode of the movement trajectory that the display control unit 181 differentiates may be the thickness of the movement trajectory, the color of the movement trajectory, or the like.

[0094] Furthermore, the display control unit 181 may differentiate the display mode of the person point cloud P13 (FIG. 8) when the person is on the line from the display mode of the person point cloud P13 (FIG. 9) when the person is on the line. In this case, the display mode of the person point cloud that the display control unit 181 differentiates may be the color of the person point cloud, or the like.

[0095] The event detection unit 182 attempts to detect an event that a person is present within the fall detection range (r1 when present on the train, and r2 when not present on the train) based on the position of the person at the current time detected by the object tracking unit 170. If the event that a person is present within the fall detection range at the current time is not detected ("NO" in S29), the operation proceeds to S22.

[0096] On the other hand, when the event detection unit 182 detects an event that a person is currently present within the fall detection range ("YES" in S29), it notifies the notification device 191 of the detection of the event (S30). This controls the notification by the notification device 191 of the detection of the event. For example, a person who receives the notification by the notification device 191 can take some kind of action regarding the person present within the fall detection range. Note that the event detection unit 182 may not attempt to detect the event that a person is present within the fall detection range when not present on the train, and may only attempt to detect the event that a person is present within the fall detection range when present on the train.

[0097] An example of the operation of the information processing device 10 according to the embodiment of the present invention has been described above.

[0098] [1-3. Effects of the embodiment] As described above, according to the information processing device 10 according to the embodiment of the present invention, the background trajectory point cloud P4 including both the platform point cloud P11 and the train point cloud P12 is created in the advance preparation stage. Then, based on the background trajectory point cloud P4 and the input point clouds P5 and P7, it is determined whether a train is present on the line or not. This allows the determination of whether a train is present on the line or not to be made with high accuracy.

[0099] In particular, according to the information processing device 10 of the embodiment of the present invention, there is no need to perform mask processing on the train point cloud P12, so whether a train is present on the track or not can be determined with high accuracy even when the platform is curved.

[0100] The effects of the embodiments of the present invention have been described above.

[0101] <2. Various modified examples> Next, various modified examples will be described.

[0102] (First Modification) In the above, an example has been described in which it is determined whether the state of a train is one of two states (on the line or not on the line). That is, an example has been described in which the on-line determination unit 180 determines that the train is not on the line when the number of points included in the difference point cloud is equal to or greater than a first threshold, and determines that the train is on the line when the number of points included in the difference point cloud is smaller than the first threshold. However, it may also be determined that the state of the train is one of three or more states. For example, assume a case in which a second threshold smaller than the first threshold is further provided.

[0103] At this time, the on-track determination unit 180 may determine that the front or rear part of the train is within the measurement range of the sensor 100 when the number of points included in the difference point cloud is equal to or greater than the second threshold value and smaller than the first threshold value. Hereinafter, the state in which the front part of the train is within the measurement range of the sensor 100 is also referred to as the state in which the train is "approaching" the platform. Also, the state in which the rear part of the train is within the measurement range of the sensor 100 is also referred to as the state in which the train is "leaving" the platform.

[0104] On the other hand, when the number of points included in the difference point cloud is smaller than the second threshold value, the on-track determination unit 180 may determine that neither the front nor the rear of the train is within the measurement range of the sensor 100. Hereinafter, the state in which neither the front nor the rear of the train is within the measurement range of the sensor 100 is also referred to as the state in which the train has "approached but not yet left" the platform.

[0105] FIG. 10 is a diagram for explaining the first modified example. Referring to FIG. 10, it is a graph showing the relationship between time and the number of points included in the difference point cloud. As shown in FIG. 10, as time passes, the number of points included in the difference point cloud decreases, then becomes constant, and then increases. As time passes, the state of the train transitions from not being on the line to approaching, transitions after approaching and before separating, transitions to separating, and returns to not being on the line. For example, the point cloud included in the difference point cloud at the start of the approach is the first threshold value Th1, and the point cloud included in the difference point cloud at the start of the separation is the second threshold value Th2.

[0106] (Second Modification) Furthermore, the on-rail determining unit 180 may determine with higher accuracy whether the state of the train is approaching or leaving.

[0107] 11 and 12 are diagrams for explaining the second modified example. As shown in Fig. 11 and 12, a first range (hereinafter also referred to as "downstream range E1") and a second range (hereinafter also referred to as "upstream range E2") located upstream of the first range in the moving direction of the train may be set in advance in the measurement range R0 of the sensor 100.

[0108] When the number of points included in the difference point cloud is greater than or equal to the second threshold and less than the first threshold, the on-track determination unit 180 may determine whether the train is approaching or leaving based on the number of points included in the downstream range E1 of the difference point cloud and the number of points included in the upstream range E2 of the difference point cloud.

[0109] When the train is approaching, the number of points included in the point cloud belonging to the downstream range E1 is smaller than the number of points in the point cloud P121 belonging to the upstream range E2, as shown in Fig. 11. On the other hand, when the train is leaving, the number of points included in the point cloud P129 belonging to the downstream range E1 is equal to or greater than the number of points in the point cloud belonging to the upstream range E2, as shown in Fig. 12.

[0110] Therefore, the on-track determination unit 180 may determine that the state of the train is approaching when the number of points included in the point group belonging to the downstream range E1 is smaller than the number of points in the point group P121 belonging to the upstream range E2. On the other hand, the on-track determination unit 180 may determine that the state of the train is leaving when the number of points included in the point group P129 belonging to the downstream range E1 is equal to or greater than the number of points in the point group belonging to the upstream range E2, as shown in Fig. 12.

[0111] (Third Modification) In the above, it is mainly assumed that the background trajectory point cloud P4 is created by superimposing the trajectory point clouds of one train. However, the background trajectory point cloud P4 may be created by superimposing the trajectory point clouds of multiple trains. Alternatively, the background trajectory point cloud P4 may be created by superimposing the trajectory point clouds of multiple types of trains.

[0112] (Fourth Modification) In the above, an example has been described in which the on-line presence determination unit 180 determines that the train is not on the line when the number of points included in the difference point cloud is equal to or greater than the first threshold, and determines that the train is on the line when the number of points included in the difference point cloud is smaller than the first threshold. However, the on-line presence determination unit 180 may determine that the state of the train is not on the line when the difference between the number of points included in the background trajectory point cloud and the number of points included in the difference point cloud is smaller than the first threshold, and may determine that the state of the train is on the line when the difference is equal to or greater than the first threshold.

[0113] Various modified examples have been described above.

[0114] <3. Hardware configuration example> Next, a hardware configuration example of the information processing device 10 according to the embodiment of the present invention will be described. Hereinafter, a hardware configuration example of the information processing device 900 will be described as a hardware configuration example of the information processing device 10 according to the embodiment of the present invention. Note that the hardware configuration example of the information processing device 900 described below is merely one example of the hardware configuration of the information processing device 10. Therefore, the hardware configuration of the information processing device 10 may be such that unnecessary configurations are deleted from the hardware configuration of the information processing device 900 described below, or new configurations are added. Note that the hardware of the information processing device 10 can also be realized in a similar manner.

[0115] 13 is a diagram showing a hardware configuration of an information processing device 900 as an example of the information processing device 10 according to an embodiment of the present invention. The information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, and a communication device 911.

[0116] The CPU 901 functions as an arithmetic processing device and control device, and controls the overall operation of the information processing device 900 in accordance with various programs. The CPU 901 may also be a microprocessor. The ROM 902 stores programs and arithmetic parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901 and parameters that change appropriately during the execution. These are connected to each other by a host bus 904 consisting of a CPU bus etc.

[0117] The host bus 904 is connected to an external bus 906, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 905. It is not necessary to configure the host bus 904, bridge 905, and external bus 906 separately, and these functions may be implemented in a single bus.

[0118] The input device 908 is composed of input means such as a mouse, keyboard, touch panel, button, microphone, switch, and lever for the user to input information, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901. A user who operates the information processing device 900 can input various data to the information processing device 900 and instruct the information processing device 900 to perform processing operations by operating this input device 908.

[0119] The output device 909 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a Liquid Crystal Display (LCD) device, an OLED (Organic Light Emitting Diode) device, or a lamp, and an audio output device such as a speaker.

[0120] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data in the storage medium, a reading device for reading data from the storage medium, and a deleting device for deleting data recorded in the storage medium. The storage device 910 is configured, for example, with an HDD (Hard Disk Drive). This storage device 910 drives a hard disk and stores programs executed by the CPU 901 and various data.

[0121] The communication unit 911 is, for example, a communication interface configured with a communication device for connecting to a network, etc. Also, the communication unit 911 may be compatible with either wireless communication or wired communication.

[0122] An example of the hardware configuration of the information processing device 10 according to the embodiment of the present invention has been described above.

[0123] <4. Supplementary Information> Although the preferred embodiment of the present invention has been described in detail above with reference to the accompanying drawings, the present invention is not limited to such an example. It is clear that a person having ordinary knowledge in the technical field to which the present invention pertains can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0124] 1. Information Processing Systems 10. Information processing device 100 Sensors 110 Point cloud acquisition part 120 Background trajectory point cloud creation section 135 Judgment section 140 Point cloud extraction part 150 Object detection unit 160 Past frame detection object information storage unit 170 Object Tracking Unit 180 Line location determination section 181 Display control section 182 Event detection unit 190 Display device 191 Notification device

Claims

1. a point cloud generation unit that acquires a time including a timing when a part or all of a first moving object falls within a measurement range of a sensor, and generates a second point cloud by superimposing a first point cloud of the measurement range acquired by the sensor at the acquired time; a determination unit that determines whether or not a part or all of a second moving object is present in the measurement range at a predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time; An information processing device comprising:

2. The determination unit is a point cloud extraction unit that extracts a difference point cloud by subtracting the third point cloud from the second point cloud; a moving object presence / absence determination unit that determines whether or not a part or all of the second moving object is present based on the difference point cloud; The information processing device according to claim 1 .

3. the moving object presence / absence determination unit determines that a part or all of the second moving object is present when the number of points included in the difference point cloud is smaller than a first threshold, and determines that a part or all of the second moving object is not present when the number of points included in the difference point cloud is equal to or greater than the first threshold. The information processing device according to claim 2 .

4. the moving object presence / absence determination unit determines that the leading portion or the trailing portion of the second moving object is within the measurement range when the number of points included in the difference point cloud is equal to or greater than a second threshold value that is smaller than the first threshold value and is smaller than the first threshold value, and determines that neither the leading portion nor the trailing portion of the second moving object is within the measurement range when the number of points included in the difference point cloud is smaller than the second threshold value. The information processing device according to claim 3 .

5. when the number of points included in the difference point cloud is equal to or greater than a second threshold value that is smaller than the first threshold value and is smaller than the first threshold value, the moving object presence / absence determination unit determines whether a leading portion or a trailing portion of the second moving object is within the measurement range based on the number of points included in a first range of the difference point cloud and the number of points included in a second range of the difference point cloud. The information processing device according to claim 4.

6. the moving object presence determination unit determines that a part or all of the second moving object does not exist when a difference between a number of points included in the second point cloud and a number of points included in the difference point cloud is smaller than a first threshold, and determines that a part or all of the second moving object exists when the difference is equal to or greater than the first threshold. The information processing device according to claim 2 .

7. The information processing device includes: an object detection unit that detects a first object based on the difference point cloud; The information processing device according to claim 2 .

8. The information processing device includes: a display control unit that controls display of display information according to the position of the first object by a display device; The information processing device according to claim 7.

9. The display control unit changes a display mode of the display information based on whether or not a part or all of the second moving object is present. The information processing device according to claim 8.

10. The information processing device includes: an event detection unit that attempts to detect an event that the first object is present within a predetermined range based on the position of the first object; The information processing device according to claim 7.

11. The event detection unit controls a notification of the detection of the event by a notification device based on the detection of the event. The information processing device according to claim 10.

12. the event detection unit changes the predetermined range based on whether or not a part or all of the second moving object is present; The information processing device according to claim 10.

13. the event detection unit attempts to detect the event based on the determination that a part or all of the second moving object is present; The information processing device according to claim 10.

14. The information processing device includes: an object tracking unit that performs a tracking process to associate the position of the first object with the position of the second object based on the position of the first object and the position of a second object detected from a past point cloud; The information processing device according to claim 7.

15. The information processing device includes: a display control unit that controls display, by a display device, of a movement trajectory connecting a position of the first object and a position of the second object; The information processing device according to claim 14.

16. the display control unit changes a display mode of the movement trajectory based on whether or not a part or all of the second moving object is present. The information processing device according to claim 15.

17. Each of the first moving body and the second moving body is a person riding an escalator, a train, a ship, or an automobile. The information processing device according to claim 1 .

18. acquiring a time including a timing when a part or all of a first moving object falls within a measurement range of a sensor, and generating a second point cloud by superimposing a first point cloud of the measurement range acquired by the sensor at the time; determining whether or not a part or all of a second moving object is present in the measurement range at a predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time; 2. An information processing method implemented by a computer, comprising:

19. Computer, a point cloud generation unit that acquires a time including a timing when a part or all of a first moving object falls within a measurement range of a sensor, and generates a second point cloud by superimposing a first point cloud of the measurement range acquired by the sensor at the acquired time; a determination unit that determines whether or not a part or all of a second moving object is present in the measurement range at a predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time; A program that functions as a

20. A sensor for obtaining a point cloud; a point cloud generation unit that acquires a time including a timing when a part or all of a first moving object falls within a measurement range of the sensor, and generates a second point cloud by superimposing a first point cloud of the measurement range acquired by the sensor at the acquired time; a determination unit that determines whether or not a part or all of a second moving object is present in the measurement range at a predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time; An information processing system comprising:

Images