Information processing device, information processing method, program, and information processing system

Abstract

To solve the problem that capability of determining with high accuracy whether or not the whole or part of a mobile body exists in the range of measurement by a sensor is desired.SOLUTION: Provided is an information processing device comprising: a point cloud generation unit for acquiring a time including a timing at which the whole or part of a first mobile body belongs to the range of measurement by a sensor and generating a second point cloud by superposing on the time a first point cloud in the range of measurement having been obtained by the sensor; and a determination unit for determination whether or not the whole or part of a second mobile body exists in the range of measurement at a prescribed time on the basis of the second point cloud and a third point cloud in the range of measurement having been obtained by the sensor at a prescribed time.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and an information processing system. [Background technology]

[0002] In recent years, technology for detecting objects based on data obtained by sensors has become known. There are various types of sensors. For example, there are sensors that repeatedly scan light or radio waves two-dimensionally from the start point to the end point to continuously obtain scanning results (hereinafter also referred to as "frames"). A collection of detected positions of light or radio waves in a frame can correspond to a point cloud. Here, a point cloud indicates the distance from the sensor to the object in each two-dimensional coordinate. Therefore, by using a point cloud, it is possible to detect objects existing in real space three-dimensionally.

[0003] LiDAR (Laser Imaging Detection and Ranging) is known as an example of a sensor. For example, a technology for detecting an object based on a point cloud obtained by LiDAR has been disclosed (see, for example, Patent Document 1). Another example of an object that can be detected by a sensor is a moving body. A technology for determining whether a part or all of a moving body is present within a measurement range of a sensor is also known. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-219248 Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is desirable to determine with high accuracy whether or not a part or all of a moving object is present within the measurement range of a sensor.

[0006] Therefore, the present invention has been made in consideration of the above problems, and an object of the present invention is to provide a technology that can determine with high accuracy whether part or all of a moving object is present within the measurement range of a sensor. [Means for solving the problem]

[0007] In order to solve the above problem, according to one aspect of the present invention, the measurement range of the sensor is On the tracks a point cloud generation unit that acquires a time including a timing when a part or all of a first moving object belongs, and generates a second point cloud by superimposing a first point cloud of the measurement range obtained by the sensor at the time; and a point cloud generation unit that generates a second point cloud of the measurement range at the predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time. On the track a determination unit that determines whether or not a part or all of the second moving object is present, the determination unit includes a point cloud extraction unit that extracts a difference point cloud by subtracting the third point cloud from the second point cloud, and a moving object presence determination unit that determines whether a part or all of the second moving object is present based on the difference point cloud, and each of the first moving object and the second moving object is a train. An information processing device is provided.

[0009] The moving object presence / absence determination unit may determine that part or all of the second moving object is present when the number of points included in the difference point cloud is smaller than a first threshold, and may determine that part or all of the second moving object is not present when the number of points included in the difference point cloud is greater than or equal to the first threshold.

[0010] The moving body presence / absence determination unit may determine that the leading or trailing portion of the second moving body is within the measurement range when the number of points included in the difference point cloud is greater than or equal to a second threshold value that is smaller than the first threshold value and is smaller than the first threshold value, and may determine that neither the leading nor trailing portion of the second moving body is within the measurement range when the number of points included in the difference point cloud is smaller than the second threshold value.

[0011] The moving body presence / absence determination unit may determine whether the leading or trailing part of the second moving body is within the measurement range based on the number of points included in a first range of the difference point cloud and the number of points included in a second range of the difference point cloud when the number of points included in the difference point cloud is greater than or equal to a second threshold value that is smaller than the first threshold value and is smaller than the first threshold value.

[0012] The moving object presence / absence determination unit may determine that part or all of the second moving object does not exist if the difference between the number of points included in the second point cloud and the number of points included in the difference point cloud is smaller than a first threshold, and may determine that part or all of the second moving object exists if the difference is greater than or equal to the first threshold.

[0013] The information processing device may include an object detection unit that detects a first object based on the difference point cloud.

[0014] The information processing device may include a display control unit that controls display of display information according to the position of the first object by a display device.

[0015] The display control unit may change a display mode of the display information based on whether or not a part or all of the second moving object is present.

[0016] The information processing device may include an event detection unit that attempts to detect an event that the first object is present within a predetermined range, based on a position of the first object.

[0017] The event detection unit may control a notification device to notify the user that the event has been detected, based on the detection of the event.

[0018] The event detection unit may change the predetermined range based on whether or not a part or all of the second moving object is present.

[0019] The event detection unit may attempt to detect the event based on the determination that a part or all of the second moving object is present.

[0020] The information processing device includes an object tracking unit that performs a tracking process to associate the position of the first object with the position of the second object based on the position of the first object and the position of the second object detected from a past point cloud. The first object and the second object may be the same object. .

[0021] The information processing device may include a display control unit that controls display, by a display device, of a movement trajectory connecting the position of the first object and the position of the second object.

[0022] The display control unit may change a display mode of the movement trajectory based on whether or not a part or all of the second moving object is present.

[0023] Each of the first moving body and the second moving body may be a person riding an escalator, a train, a ship, or a car.

[0024] In order to solve the above problem, according to another aspect of the present invention, the measurement range of the sensor is On the tracks a time including a timing when a part or all of a first moving object belongs to the first moving object, and a second point cloud is generated by superimposing a first point cloud of the measurement range obtained by the sensor at the time; and a third point cloud of the measurement range obtained by the sensor at a predetermined time based on the second point cloud and the third point cloud of the measurement range obtained by the sensor at the predetermined time. On the track determining whether or not a part or all of the second moving object is present. and the determining step includes extracting a difference point cloud by subtracting the third point cloud from the second point cloud, and determining whether or not a part or all of the second moving body is present based on the difference point cloud, wherein each of the first moving body and the second moving body is a train. A computer-implemented information processing method is provided.

[0025] According to another aspect of the present invention, the computer is configured to detect a location within a measurement range of a sensor. On the tracksa point cloud generation unit that acquires a time including a timing when a part or all of a first moving object belongs, and generates a second point cloud by superimposing a first point cloud of the measurement range obtained by the sensor at the time; and a point cloud generation unit that generates a second point cloud of the measurement range at the predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time. On the track a determination unit that determines whether or not a part or all of the second moving object is present. the determination unit includes a point cloud extraction unit that extracts a difference point cloud by subtracting the third point cloud from the second point cloud, and a moving object presence determination unit that determines whether a part or all of the second moving object is present based on the difference point cloud, and each of the first moving object and the second moving object is a train. Programs are offered.

[0026] According to another aspect of the present invention, there is provided a method for solving the above-mentioned problems, comprising: a sensor for obtaining a point cloud; On the tracks a point cloud generation unit that acquires a time including a timing when a part or all of a first moving object belongs, and generates a second point cloud by superimposing a first point cloud of the measurement range obtained by the sensor at the time; and a point cloud generation unit that generates a second point cloud of the measurement range at the predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time. On the track a determination unit that determines whether or not a part or all of the second moving object is present, the determination unit includes a point cloud extraction unit that extracts a difference point cloud by subtracting the third point cloud from the second point cloud, and a moving object presence determination unit that determines whether a part or all of the second moving object is present based on the difference point cloud, and each of the first moving object and the second moving object is a train. An information processing system is provided. [Effects of the Invention]

[0027] As described above, the present invention provides a technique that can determine with high accuracy whether or not a part or all of a moving object is present within a measurement range of a sensor. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 10 is a diagram illustrating an example of human detection using a general background subtraction technique. [Figure 2] 1 is a block diagram illustrating an example of a functional configuration of an information processing system according to an embodiment of the present invention. [Figure 3] 10 is a flowchart showing an example of the operation of the information processing device 10 in the advance preparation stage. [Figure 4] 10 is a diagram showing an example of a background trajectory point cloud created by a background trajectory point cloud creating unit 120. FIG. [Figure 5] 10 is a flowchart showing an example of the operation of the information processing device 10 in the operation stage. [Figure 6] FIG. 10 is a diagram for explaining an example of a difference point cloud when a train is not on the line. [Figure 7] FIG. 10 is a diagram illustrating an example of a difference point cloud when a train is on the line. [Figure 8] FIG. 10 is a diagram showing an example of a display of a movement trajectory of a person while on the train. [Figure 9] FIG. 10 is a diagram showing an example of a display of a person's movement trajectory when the person is not present on the train. [Figure 10] FIG. 10 is a diagram for explaining a first modified example. [Figure 11] FIG. 10 is a diagram for explaining a second modified example. [Figure 12] FIG. 10 is a diagram for explaining a second modified example. [Figure 13] 1 is a diagram showing a hardware configuration of an information processing device 900 as an example of the information processing device 10 according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0029] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant explanations will be omitted.

[0030] <0.Background> First, the background of the embodiment of the present invention will be described.

[0031] In recent years, techniques for detecting objects based on data obtained by sensors have become known. As an example of an object, a moving object can also be detected by a sensor. For example, techniques for determining whether a part or all of a moving object is present within a measurement range of a sensor are also known. However, there is a demand for highly accurate determination of whether a part or all of a moving object is present within a measurement range of a sensor.

[0032] In this specification, it is mainly assumed that the moving object whose presence or absence in the measurement range of the sensor is determined is a train. However, the moving object whose presence or absence in the measurement range of the sensor is determined is not limited to a train. For example, the moving object whose presence or absence in the measurement range of the sensor is determined may be a ship, a car, a person riding an escalator, or the like.

[0033] In this specification, the presence of a part or all of a train within the sensor's measurement range is also referred to as "being on the line," and the absence of a part or all of a train within the sensor's measurement range is also referred to as "not being on the line." The reason we use the expression "part of a train" is because a train is not always made up of a single vehicle, but can be made up of multiple vehicles connected together, and it is therefore fully expected that the entire train will not fit within the sensor's measurement range.

[0034] Furthermore, in this specification, it is assumed that whether or not a part or all of a moving body is present within a measurement range of a sensor is primarily used to reduce the possibility of contact between the moving body and an object. Objects can also be detected using sensors. In this specification, it is primarily assumed that the object detected by the sensor is a person. However, the object detected by the sensor is not limited to a person. For example, the object detected by the sensor may be an animal other than a person, a robot, or the like.

[0035] In the following explanation, we consider reducing the possibility of contact between trains and people at railway stations. Here, it is essential to take safety measures on railway station platforms (hereinafter simply referred to as "platforms") to prevent situations in which people fall into or are trapped in the gap between the train and the platform while getting on or off the train.

[0036] One example of a safety measure on platforms is the installation of platform doors. However, the structure of the platform or the weight of the platform doors can sometimes impose restrictions on their installation. In addition, installing platform doors requires large-scale construction work and is very costly, so there is a need for easier, less expensive safety measures.

[0037] One of the easier and less costly safety measures is to install sensors on the platform and use them to detect the location of people on or around the platform. An example of such a sensor is a laser sensor. Here, background subtraction technology can be used as an example of a method for detecting people using a laser sensor. An example of person detection using a general background subtraction technology will be described with reference to Figure 1.

[0038] Fig. 1 is a diagram illustrating an example of human detection using a general background subtraction technique. Fig. 1 shows an input point cloud P1 obtained in real time by a sensor, a background point cloud P2 prepared in advance, and a difference point cloud P3 obtained by subtracting the background point cloud P2 from the input point cloud P1. In Fig. 1, a minus sign (-) may mean subtracting the following point cloud from the preceding point cloud.

[0039] The input point cloud P1 includes a platform point cloud P11, which is a point cloud obtained from the platform, a train point cloud P12, which is a point cloud obtained from the train, and a person point cloud P13, which is a point cloud obtained from people. In addition, in the embodiment of the present invention, the movement direction of the train is assumed to be from bottom to top of the drawing unless otherwise specified. That is, the upstream of the train's movement direction is the downward direction on the drawing, and the downstream of the train's movement direction is the upward direction on the drawing, and the train point cloud P12 includes a point cloud P121 of a downstream vehicle and a point cloud P122 of an upstream vehicle.

[0040] As an example, if the point group to be subtracted is A and the point group from which it is subtracted is B, subtracting point group B from point group A may mean excluding from point group A any point included in point group A whose distance to any point included in point group B is less than a threshold value.

[0041] When using background subtraction technology, a prepared background point cloud P2 is subtracted from an input point cloud P1 obtained by a sensor to extract a difference point cloud P3, and people are detected from the difference point cloud P3. In other words, people can be detected from the input point cloud P1 that is not included in the background point cloud P2.

[0042] Generally, the background point cloud P2 is created by overlaying point clouds from multiple frames obtained by the sensor so that a certain degree of fluctuation in the point cloud is reflected in the background point cloud during times when there are no moving objects within the sensor's measurement range.

[0043] However, as shown in Figure 1, if the background point cloud P2 is created by superimposing point clouds from a time period when neither people nor trains are present in the sensor's measurement range, the difference point cloud P3 obtained by subtracting the background point cloud P2 from the input point cloud P1 will contain the person point cloud P13 and the train point cloud P12. In other words, there is a possibility that both people and trains will be detected in the difference point cloud P3. Furthermore, processing the point cloud obtained from a large train requires a huge processing load.

[0044] On the other hand, a method of excluding the train point cloud P12 by performing mask processing on the train point cloud P12 is also conceivable. However, since the target of the mask processing is generally specified by a straight line, if the platform is curved, it may be difficult to perform mask processing on the train point cloud P12 along the curved platform.

[0045] Furthermore, even if the train point cloud P12 were removed by masking, it would still be difficult to determine whether a train is on the track or not. For example, because dangerous locations for people may differ depending on whether a train is on the track or not, it is also important to determine whether a train is on the track or not with high accuracy. Therefore, this specification mainly proposes a technology for determining whether a train is on the track or not with high accuracy.

[0046] The background of the embodiments of the present invention has been described above.

[0047] <1. Details of the embodiment> Next, details of the embodiment of the present invention will be described.

[0048] [1-1. Example of information processing system configuration] First, an example of the functional configuration of an information processing system according to an embodiment of the present invention will be described. Fig. 2 is a block diagram showing an example of the functional configuration of an information processing system according to an embodiment of the present invention. As shown in Fig. 2, the information processing system 1 according to an embodiment of the present invention includes an information processing device 10, a sensor 100, a display device 190, and a notification device 191. The information processing device 10, the sensor 100, the display device 190, and the notification device 191 may be connected via a network.

[0049] (Sensor 100) The sensor 100 is installed on or near the platform. The sensor 100 has the function of acquiring a platform point cloud, a train point cloud, and a person point cloud. Here, it is mainly assumed that the sensor 100 is realized by a laser sensor. However, the sensor 100 is not limited to a laser sensor. The laser sensor can also be referred to as a "LiDAR sensor."

[0050] Note that various types of sensors other than a laser sensor (such as a distance measurement sensor) may be used as the sensor 100. For example, the sensor 100 may be a radar (such as a millimeter wave radar) that uses electromagnetic waves instead of laser light. Alternatively, the sensor 100 may be a sonar that uses sound waves instead of laser light.

[0051] (Information processing device 10) The information processing device 10 can be realized by a computer. As shown in Fig. 2, the information processing device 10 includes a point cloud acquisition unit 110, a background trajectory point cloud creation unit 120, a background trajectory point cloud storage unit 130, a determination unit 135, an object detection unit 150, a past frame detected object information storage unit 160, an object tracking unit 170, a display control unit 181, and an event detection unit 182. The determination unit 135 includes a point cloud extraction unit 140 and an on-rail determination unit 180 (a moving object presence / absence determination unit). Details of these components included in the information processing device 10 will be described later.

[0052] For example, the point cloud acquisition unit 110, the background trajectory point cloud creation unit 120, the determination unit 135, the object detection unit 150, the object tracking unit 170, the display control unit 181, and the event detection unit 182 may be realized by a control unit (not shown). On the other hand, the background trajectory point cloud storage unit 130 and the past frame detected object information storage unit 160 may be realized by a storage unit (not shown).

[0053] The control unit (not shown) includes a CPU (Central Processing Unit) and the like, and its functions can be realized by the CPU expanding a program stored in a non-volatile storage device into RAM (Random Access Memory) and executing it. In this case, a computer-readable recording medium on which the program is recorded can also be provided. Alternatively, the control unit (not shown) can be configured with dedicated hardware or a combination of multiple pieces of hardware.

[0054] The storage unit (not shown) is a storage device capable of storing programs and data for operating the control unit (not shown). The storage unit (not shown) can also temporarily store various data required in the operation of the control unit (not shown). For example, the storage device may be a non-volatile storage device.

[0055] (Display device 190) The display device 190 is configured with a display. For example, the display device 190 may display display information according to the position of a person detected by the information processing device 10, under the control of the information processing device 10. The display information according to the position of the person may include a frame (object detection frame) surrounding the person. Alternatively, the display device 190 may display a movement trajectory connecting multiple positions of the same person associated by tracking processing, under the control of the information processing device 10.

[0056] There is no particular limitation on the location where the display device 190 is installed. For example, the display device 190 may be installed in a station office where station staff are present, or in a monitoring center where monitors who monitor the train operation status are present.

[0057] (Notification device 191) The notification device 191 notifies that an event has been detected by the information processing device 10, in accordance with the control of the information processing device 10. Here, it is mainly assumed that the event is a person falling from a platform. For example, the notification device 191 may be configured to include a lamp, and may notify that an event has been detected by turning on the lamp. Alternatively, the notification device 191 may be configured with a monitor, and may notify that an event has been detected by displaying a predetermined color (for example, red) on the monitor screen.

[0058] There is no particular limitation on the location where the notification device 191 is installed. For example, the notification device 191 may be installed on the platform or in a monitoring center.

[0059] An example of the functional configuration of the information processing system 1 according to the embodiment of the present invention has been described above.

[0060] [1-2. Example of operation of information processing device] Next, an example of operation of the information processing device 10 according to the embodiment of the present invention will be described. The example of operation of the information processing device 10 is mainly divided into two stages. The first stage is a preparation stage. The second stage is an operation stage. Hereinafter, an example of operation of the information processing device 10 relating to the preparation stage will be described with reference to Figs. 3 and 4.

[0061] (Preparation stage) 3 is a flowchart showing an example of the operation of the information processing device 10 in the advance preparation stage. In the advance preparation stage, the background trajectory point cloud creation unit 120 acquires a time (hereinafter also referred to as "on-track time") that includes a timing when part or all of a train (first moving object) falls within the measurement range of the sensor 100. Note that the measurement range of the sensor 100 may be the entire range that can be measured by the sensor 100, or may be a part of the range that can be measured by the sensor 100.

[0062] For example, the on-rail time may be the time from when the front part of the train enters the measurement range of the sensor 100 until the rear part of the train exits the measurement range of the sensor 100. Note that the on-rail time may also include a time when part or all of the train does not fall within the measurement range of the sensor 100. A plurality of frames are output continuously in chronological order from the sensor 100 to the information processing device 10.

[0063] The background trajectory point cloud creation unit 120 acquires a point cloud (first point cloud) of the measurement range obtained by the sensor 100 during the train's on-track time from the point cloud acquisition unit 110. Then, the background trajectory point cloud creation unit 120 generates a background trajectory point cloud (second point cloud) by overlaying the acquired point clouds of the measurement range (S11). Of the background trajectory point clouds, the point cloud obtained from the platform is the background point cloud, and the point cloud obtained from the train is the trajectory point cloud. The background trajectory point cloud creation unit 120 stores the created background trajectory point cloud in the background trajectory point cloud storage unit 130.

[0064] More specifically, if the point cloud acquired by the point cloud acquisition unit 110 at time u is denoted by pc(u), the time when the front part of the train enters the measurement range of the sensor 100 is denoted by u1, and the time when the rear part of the train exits the measurement range of the sensor 100 is denoted by u2, the background trajectory point cloud pc_traj from time u1 to time u2 can be created as shown in the following equation (1).

[0065]

number

[0066] Fig. 4 is a diagram showing an example of a background trajectory point cloud created by the background trajectory point cloud creation unit 120. Referring to Fig. 4, the measurement range R0 of the sensor 100 is shown. Also shown is a background trajectory point cloud P4 created by the background trajectory point cloud creation unit 120. The background trajectory point cloud P4 includes a platform point cloud P41 formed by superimposing point clouds obtained from the platform from time u1 to time u2, and a train point cloud P42 formed by superimposing point clouds obtained from the train from time u1 to time u2.

[0067] Here, the trajectory point cloud creation unit 120 may acquire the on-train time in any manner. For example, the point cloud acquired by the point cloud acquisition unit 110 may be displayed on a display device used by the worker, and the worker may determine the on-train time while visually checking the displayed point cloud and input the on-train time into an input device used by the worker. In this case, the background trajectory point cloud creation unit 120 may acquire the on-train time input by the worker.

[0068] Alternatively, the background trajectory point cloud creation unit 120 may automatically acquire the on-track time without manual operation. For example, when a train detection sensor installed around the tracks detects the time when a train arrives at a predetermined position on the tracks and the detected time is acquired by the information processing device 10 from the train detection sensor, the background trajectory point cloud creation unit 120 may predict the on-track time from the detected time.

[0069] An example of the operation of the information processing device 10 in the advance preparation stage has been described above with reference to FIGS.

[0070] (Operational stage) Next, an example of the operation of the information processing device 10 in the operation stage will be described with reference to FIGS.

[0071] Fig. 5 is a flowchart showing an example of operation of the information processing device 10 in the operation stage. As shown in Fig. 5, in the operation stage, the point cloud extraction unit 140 acquires a background trajectory point cloud from the background trajectory point cloud storage unit 130. Furthermore, the point cloud acquisition unit 110 acquires a point cloud (third point cloud) at the current time (predetermined time) in the measurement range obtained by the sensor 100 from the sensor 100 as an input point cloud, and the point cloud extraction unit 140 acquires the input point cloud (S21). The determination unit 135 determines whether a vehicle is present on a line or not based on the background trajectory point cloud and the input point cloud.

[0072] First, the point cloud extraction unit 140 extracts a difference point cloud based on the background trajectory point cloud and the input point cloud (S22). More specifically, the point cloud extraction unit 140 extracts the difference point cloud by subtracting the input point cloud from the background trajectory point cloud. When subtracting the input point cloud from the background trajectory point cloud, the point cloud extraction unit 140 may appropriately rotate the coordinate axes serving as the reference for the background trajectory point cloud and the coordinate axes serving as the reference for the input point cloud to facilitate the calculation of subtracting the input point cloud from the background trajectory point cloud.

[0073] The object detection unit 150 detects an object (first object) based on the difference point cloud (S23). For example, the object detection unit 150 may detect an object by a clustering process. Note that the object detected by the object detection unit 150 is obtained as a three-dimensional point cloud of the object. Furthermore, the object detected by the object detection unit 150 is mainly a person.

[0074] The object tracking section 170 associates the detected object in the current frame with the detected object in the previous frame (S24). More specifically, the object tracking section 170 acquires the position of the object in the previous frame from the past frame detected object information storage section 160.

[0075] Furthermore, the object tracking unit 170 detects the position of the object at the current time (the position of the object in the current frame) based on the object detection result. The position of the object detected by the object tracking unit 170 may be any position of the object. For example, the object tracking unit 170 may detect the position of the center of gravity of the three-dimensional point cloud of the object as the position of the object, or may detect a position estimated based on the position of the three-dimensional point cloud of the object and the velocity of the object as the position of the object.

[0076] When the distance between the object's position in the previous frame and the object's position in the current frame is equal to or less than a threshold, the object tracking unit 170 associates the object's position in the current frame with the object's position in the previous frame that is closest to the object's position in the current frame and stores the associated object position in the previous frame in the past frame detected object information storage unit 160. For example, the distance between the object's position in the previous frame and the object's position in the current frame may be the Euclidean distance. Note that the previous frame may also be referred to as a frame that precedes the current frame (a past frame).

[0077] On the other hand, when the distance between the position of the object in the previous frame and the position of the object in the current frame exceeds a threshold, the object tracking unit 170 stores the position of the object in the current frame in the past frame detected object information storage unit 160 without correlating it with the position of the object in the previous frame.

[0078] Next, if the predetermined time has not passed ("NO" in S25), the operation proceeds to S28. On the other hand, if the predetermined time has passed ("YES" in S25), the on-rail determination unit 180 determines whether a train is on a rail or not based on the difference point cloud (S26). Note that the predetermined time may arrive at a predetermined time interval. The predetermined time interval may be a short time such as a few milliseconds or a few seconds, or a long time such as a few days or a few months.

[0079] Fig. 6 is a diagram illustrating an example of a difference point cloud when a train is not present on the platform. Fig. 6 shows a background trajectory point cloud P4 and an input point cloud P5 when a train is not present on the platform. When a train is not present on the platform, there is no train within the measurement range of the sensor 100. Therefore, the input point cloud P5 when a train is not present on the platform includes a platform point cloud P11, which is a point cloud obtained from the platform, and a person point cloud P13, which is a point cloud obtained from people, but does not include a train point cloud, which is a point cloud obtained from the train.

[0080] On the other hand, the background trajectory point cloud P4 includes a train point cloud P42, which is an overlay of point clouds obtained from trains, and therefore the difference point cloud P6 obtained by subtracting the input point cloud P5 when the train is not on the line from the background trajectory point cloud P4 contains a large number of points.

[0081] Fig. 7 is a diagram illustrating an example of a difference point cloud when a train is on the track. Fig. 7 shows a background trajectory point cloud P4 and an input point cloud P7 when a train is on the track. When a train is present within the measurement range of sensor 100, the input point cloud P7 when a train is on the track not only includes a platform point cloud P11, which is a point cloud obtained from the platform, and a person point cloud P13, which is a point cloud obtained from people, but also a train point cloud P12, which is a point cloud obtained from the train.

[0082] On the other hand, the background trajectory point cloud P4 also includes a train point cloud P42, which is an overlay of point clouds obtained from the train, so the difference point cloud P8 obtained by subtracting the input point cloud P7 when the train is on the line from the background trajectory point cloud P4 contains a small number of points.

[0083] Therefore, the on-track determination unit 180 may determine that the train is not on track when the number of points included in the difference point cloud is equal to or greater than the first threshold value. On the other hand, the on-track determination unit 180 may determine that the train is on track when the number of points included in the difference point cloud is smaller than the first threshold value.

[0084] More specifically, if the input point cloud is pc_input, the difference point cloud is pc_invert, and, as above, the background trajectory point cloud is pc_traj, the relationship between the difference point cloud pc_invert, the background trajectory point cloud pc_traj, and the input point cloud pc_input is as shown in equation (2) below.

[0085] pc_invert=pc_traj-pc_input ···(2)

[0086] Furthermore, if the function that takes a point cloud as input and outputs the number of points contained in the point cloud is count() and the first threshold is Th_pc, the condition for determining that a train is on the track is as shown in equation (3) below.

[0087] count(pc_invert) <Th_pc ···(3)

[0088] The display control unit 181 controls the display of the determination result of whether the train is on the rail or not on the display device 190 (S27). Furthermore, the display control unit 181 controls the display of the movement trajectory of the person detected by the object tracking unit 170 on the display device 190 (S28). At this time, the display control unit 181 may control the display of display information according to the position of the person detected by the object tracking unit 170 on the display device 190. The display information according to the position of the person may include a frame surrounding the person (person detection frame).

[0089] Furthermore, the display control unit 181 may control the display by the display device 190 of a fall detection range, which is a range (predetermined range) for detecting a person falling from the platform. In this case, when a train is present on the tracks, there is a train on the tracks, but when no train is present on the tracks, there is no train on the tracks, so the range within which a person may fall from the platform may differ between when a train is present and when no train is present. Therefore, the event detection unit 182 may make the fall detection range different between when a train is present and when no train is present. Alternatively, having a fall detection range that is different between when a train is present and when no train is present may include a case where a fall detection range is set only when a train is present and no fall detection range is set when no train is present.

[0090] Fig. 8 is a diagram showing an example of displaying a movement trajectory of a person when the train is on the track. Referring to Fig. 8, the display control unit 181 controls the display device 190 to display that a train is on the track, the platform point cloud P11, and the fall detection range r1 from time t0 to t2. Furthermore, the display control unit 181 controls the display device 190 to display the movement trajectory T0 of the person at time t0, controls the display device 190 to display the movement trajectory T1 of the person at time t1 after time t0, and controls the display device 190 to display the movement trajectory T2 of the person at time t2 after time t1.

[0091] Fig. 9 is a diagram showing an example of displaying a movement trajectory of a person when a train is not present on the line. Referring to Fig. 9, the display control unit 181 controls the display device 190 to display that a train is not present on the line, the platform point cloud P11, and the fall detection range r2. Furthermore, the display control unit 181 controls the display device 190 to display the movement trajectory T0 of the person at time t0, controls the display device 190 to display the movement trajectory T1 of the person at time t1 after time t0, and controls the display device 190 to display the movement trajectory T2 of the person at time t2 after time t1.

[0092] The display control unit 181 may differentiate the display mode of the frame of the fall detection range r1 (FIG. 8) when a train is present on the rail from the display mode of the frame of the fall detection range r2 (FIG. 9) when a train is not present on the rail. In this case, the display mode of the fall detection range that the display control unit 181 differentiates may be the thickness of the line of the frame, the color of the line of the frame, or the like.

[0093] Furthermore, the display control unit 181 may differentiate the display mode of the movement trajectories T0 to T2 of a person when the person is on the train (FIG. 8) from the display mode of the movement trajectories T0 to T2 of a person when the person is on the train (FIG. 9). In this case, the display mode of the movement trajectories that the display control unit 181 differentiates may be the thickness of the movement trajectories, the color of the movement trajectories, or the like.

[0094] Furthermore, the display control unit 181 may differentiate the display mode of the person point cloud P13 (FIG. 8) when the person is on the line from the display mode of the person point cloud P13 (FIG. 9) when the person is on the line. In this case, the display mode of the person point cloud that the display control unit 181 differentiates may be the color of the person point cloud, or the like.

[0095] The event detection unit 182 attempts to detect an event that a person is present within the fall detection range (r1 when present on the train, r2 when not present on the train) based on the position of the person at the current time detected by the object tracking unit 170. If the event that a person is present within the fall detection range at the current time is not detected ("NO" in S29), the operation proceeds to S22.

[0096] On the other hand, when the event detection unit 182 detects an event that a person is currently present within the fall detection range ("YES" in S29), it notifies the notification device 191 that an event has been detected (S30). This controls the notification by the notification device 191 that an event has been detected. For example, a person who has received a notification from the notification device 191 can take some kind of action regarding the person who is present within the fall detection range. Note that the event detection unit 182 may not attempt to detect an event that a person is present within the fall detection range when the train is not present on the tracks, but may attempt to detect an event that a person is present within the fall detection range only when the train is present on the tracks.

[0097] An example of the operation of the information processing device 10 according to the embodiment of the present invention has been described above.

[0098] [1-3. Effects of the embodiment] As described above, according to the information processing device 10 according to the embodiment of the present invention, the background trajectory point cloud P4 including both the platform point cloud P11 and the train point cloud P12 is created in the advance preparation stage. Then, based on the background trajectory point cloud P4 and the input point clouds P5 and P7, it is determined whether a train is present or absent on the train. This allows for highly accurate determination of whether a train is present or absent on the train.

[0099] In particular, according to the information processing device 10 of the embodiment of the present invention, there is no need to perform masking processing on the train point cloud P12, so whether a train is present on the track or not can be determined with high accuracy even when the platform is curved.

[0100] The effects of the embodiments of the present invention have been described above.

[0101] <2. Various Modifications> Next, various modified examples will be described.

[0102] (First Modification) The above describes an example in which it is determined whether the state of a train is one of two states (on track or not on track). That is, it describes an example in which the on-track presence determination unit 180 determines that the train is not on track when the number of points included in the difference point cloud is equal to or greater than a first threshold, and determines that the train is on track when the number of points included in the difference point cloud is less than the first threshold. However, it may also be possible to determine which of three or more states the state of the train is. For example, it is assumed that a second threshold smaller than the first threshold is further set.

[0103] At this time, the on-track determination unit 180 may determine that the front or rear of the train is within the measurement range of the sensor 100 if the number of points included in the difference point cloud is equal to or greater than the second threshold and smaller than the first threshold. Hereinafter, the state in which the front of the train is within the measurement range of the sensor 100 will also be referred to as the state in which the train is "approaching" the platform. Also, the state in which the rear of the train is within the measurement range of the sensor 100 will also be referred to as the state in which the train is "leaving" the platform.

[0104] On the other hand, when the number of points included in the difference point cloud is smaller than the second threshold, the on-track determination unit 180 may determine that neither the front nor the rear of the train is within the measurement range of the sensor 100. Hereinafter, the state in which neither the front nor the rear of the train is within the measurement range of the sensor 100 will also be referred to as the state in which the train has "approached but not yet left" the platform.

[0105] FIG. 10 is a diagram for explaining a first modified example. Referring to FIG. 10, a graph showing the relationship between time and the number of points included in the difference point cloud is shown. As shown in FIG. 10, as time passes, the number of points included in the difference point cloud decreases, then becomes constant, and then increases. As time passes, the state of the train transitions from not being present on the track to approaching, then after approaching but before separating, then while separating, and then back to not being present on the track. For example, the point cloud included in the difference point cloud at the start of the approach is the first threshold value Th1, and the point cloud included in the difference point cloud at the start of the separation is the second threshold value Th2.

[0106] (Second Modification) Furthermore, the on-track determining unit 180 may determine with higher accuracy whether the train is approaching or leaving.

[0107] 11 and 12 are diagrams for explaining a second modified example. As shown in Fig. 11 and 12, a first range (hereinafter also referred to as a "downstream range E1") and a second range (hereinafter also referred to as an "upstream range E2") located upstream of the first range in the direction of train movement may be set in advance in the measurement range R0 of the sensor 100.

[0108] When the number of points included in the difference point cloud is greater than or equal to the second threshold and less than the first threshold, the on-track determination unit 180 may determine whether the train is approaching or moving away based on the number of points included in the downstream range E1 of the difference point cloud and the number of points included in the upstream range E2 of the difference point cloud.

[0109] When the train is approaching, the number of points included in the point cloud belonging to the downstream range E1 is smaller than the number of points in the point cloud P121 belonging to the upstream range E2, as shown in Fig. 11. On the other hand, when the train is leaving, the number of points included in the point cloud P129 belonging to the downstream range E1 is equal to or greater than the number of points in the point cloud belonging to the upstream range E2, as shown in Fig. 12.

[0110] Therefore, the train presence determination unit 180 may determine that the train is approaching when the number of points included in the point group belonging to the downstream range E1 is smaller than the number of points in the point group P121 belonging to the upstream range E2. On the other hand, as shown in Fig. 12, the train presence determination unit 180 may determine that the train is leaving when the number of points included in the point group P129 belonging to the downstream range E1 is equal to or greater than the number of points in the point group belonging to the upstream range E2.

[0111] (Third Modification) In the above, it is mainly assumed that the background trajectory point cloud P4 is created by superimposing the trajectory point clouds of one train. However, the background trajectory point cloud P4 may be created by superimposing the trajectory point clouds of multiple trains. Alternatively, the background trajectory point cloud P4 may be created by superimposing the trajectory point clouds of multiple types of trains.

[0112] (Fourth Modification) In the above example, the on-track position determination unit 180 determines that a train is not on track when the number of points included in the difference point cloud is equal to or greater than the first threshold, and determines that a train is on track when the number of points included in the difference point cloud is smaller than the first threshold. However, the on-track position determination unit 180 may determine that the state of the train is not on track when the difference between the number of points included in the background trajectory point cloud and the number of points included in the difference point cloud is smaller than the first threshold, and may determine that the state of the train is on track when the difference is equal to or greater than the first threshold.

[0113] Various modifications have been described above.

[0114] <3. Hardware configuration example> Next, an example of the hardware configuration of the information processing device 10 according to the embodiment of the present invention will be described. Below, an example of the hardware configuration of the information processing device 900 will be described as an example of the hardware configuration of the information processing device 10 according to the embodiment of the present invention. Note that the example of the hardware configuration of the information processing device 900 described below is merely one example of the hardware configuration of the information processing device 10. Therefore, the hardware configuration of the information processing device 10 may be such that unnecessary components are deleted from the hardware configuration of the information processing device 900 described below, or new components are added. Note that the hardware of the information processing device 10 can also be realized in a similar manner.

[0115] 13 is a diagram showing a hardware configuration of an information processing device 900 as an example of the information processing device 10 according to an embodiment of the present invention. The information processing device 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, a RAM (Random Access Memory) 903, a host bus 904, a bridge 905, an external bus 906, an interface 907, an input device 908, an output device 909, a storage device 910, and a communication device 911.

[0116] The CPU 901 functions as an arithmetic processing unit and control unit, and controls the overall operation of the information processing device 900 in accordance with various programs. The CPU 901 may also be a microprocessor. The ROM 902 stores programs used by the CPU 901, calculation parameters, etc. The RAM 903 temporarily stores programs used in the execution of the CPU 901, parameters that change as appropriate during the execution, etc. These are interconnected by a host bus 904 that is composed of a CPU bus, etc.

[0117] The host bus 904 is connected to an external bus 906, such as a PCI (Peripheral Component Interconnect / Interface) bus, via a bridge 905. It is not necessary to configure the host bus 904, bridge 905, and external bus 906 separately, and these functions may be implemented on a single bus.

[0118] The input device 908 is composed of input means such as a mouse, keyboard, touch panel, buttons, microphone, switches, and levers that allow the user to input information, and an input control circuit that generates an input signal based on the user's input and outputs it to the CPU 901. By operating this input device 908, the user operating the information processing device 900 can input various data to the information processing device 900 and instruct the information processing device 900 to perform processing operations.

[0119] The output device 909 includes, for example, a display device such as a CRT (Cathode Ray Tube) display device, a liquid crystal display (LCD) device, an OLED (Organic Light Emitting Diode) device, or a lamp, and an audio output device such as a speaker.

[0120] The storage device 910 is a device for storing data. The storage device 910 may include a storage medium, a recording device for recording data on the storage medium, a reading device for reading data from the storage medium, and a deletion device for deleting data recorded on the storage medium. The storage device 910 is configured, for example, with an HDD (Hard Disk Drive). This storage device 910 drives a hard disk and stores programs executed by the CPU 901 and various data.

[0121] The communication device 911 is, for example, a communication interface configured with a communication device for connecting to a network, etc. The communication device 911 may be compatible with either wireless communication or wired communication.

[0122] An example of the hardware configuration of the information processing device 10 according to the embodiment of the present invention has been described above.

[0123] <4. Supplementary Information> Although the preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, the present invention is not limited to these examples. It is clear that a person skilled in the art to which the present invention pertains can conceive of various modifications and alterations within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present invention. [Explanation of symbols]

[0124] 1. Information Processing Systems 10. Information processing equipment 100 sensors 110 Point cloud acquisition part 120 Background trajectory point cloud creation section 135 Judgment section 140 Point cloud extraction part 150 Object detection unit 160 Past frame detected object information storage unit 170 Object Tracking Unit 180 Line location determination section 181 Display control unit 182 Event detection unit 190 Display device 191 Notification device

Claims

1. a point cloud generation unit that acquires a time including a timing when a part or all of a first moving object on a track falls within a measurement range of a sensor, and generates a second point cloud by superimposing a first point cloud of the measurement range acquired by the sensor at the time; a determination unit that determines whether a part or all of a second moving object on the railway is present in the measurement range at a predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time; Equipped with The determination unit a point cloud extraction unit that extracts a difference point cloud by subtracting the third point cloud from the second point cloud; a moving object presence / absence determination unit that determines whether a part or all of the second moving object exists based on the difference point cloud; Equipped with each of the first moving body and the second moving body is a train; Information processing device.

2. the moving object presence / absence determination unit determines that a part or all of the second moving object is present when the number of points included in the difference point cloud is smaller than a first threshold, and determines that a part or all of the second moving object is not present when the number of points included in the difference point cloud is equal to or greater than the first threshold. The information processing device according to claim 1 .

3. the moving object presence determination unit determines that the leading portion or the trailing portion of the second moving object is within the measurement range when the number of points included in the difference point cloud is equal to or greater than a second threshold that is smaller than the first threshold and is smaller than the first threshold, and determines that neither the leading portion nor the trailing portion of the second moving object is within the measurement range when the number of points included in the difference point cloud is smaller than the second threshold. The information processing device according to claim 2 .

4. when the number of points included in the difference point cloud is equal to or greater than a second threshold that is smaller than the first threshold and is smaller than the first threshold, the moving object presence determination unit determines whether the leading portion or the trailing portion of the second moving object is within the measurement range based on the number of points included in a first range of the difference point cloud and the number of points included in a second range of the difference point cloud. The information processing device according to claim 3 .

5. the moving object presence determination unit determines that a part or all of the second moving object does not exist when a difference between the number of points included in the second point cloud and the number of points included in the difference point cloud is smaller than a first threshold, and determines that a part or all of the second moving object exists when the difference is equal to or greater than the first threshold. The information processing device according to claim 1 .

6. The information processing device includes: an object detection unit that detects a first object based on the difference point cloud; The information processing device according to claim 1 .

7. The information processing device includes: a display control unit that controls display of display information according to the position of the first object by a display device; The information processing device according to claim 6 .

8. the display control unit changes the display mode of the display information based on whether or not a part or all of the second moving object is present; The information processing device according to claim 7 .

9. The information processing device includes: an event detection unit that attempts to detect an event that the first object is present within a predetermined range based on the position of the first object; The information processing device according to claim 6 .

10. the event detection unit controls a notification device to notify the detection of the event based on the detection of the event; The information processing device according to claim 9 .

11. the event detection unit changes the predetermined range based on whether or not a part or all of the second moving object is present. The information processing device according to claim 9 .

12. the event detection unit attempts to detect the event based on the determination that a part or all of the second moving object is present; The information processing device according to claim 9 .

13. The information processing device includes: an object tracking unit that performs a tracking process to associate the position of the first object with the position of the second object based on the position of the first object and the position of the second object detected from a past point cloud; the first object and the second object are the same object; The information processing device according to claim 6 .

14. The information processing device includes: a display control unit that controls display, by a display device, of a movement trajectory connecting the position of the first object and the position of the second object; The information processing device according to claim 13.

15. the display control unit changes a display mode of the movement trajectory based on whether or not a part or all of the second moving object is present; The information processing device according to claim 14.

16. acquiring a time including a timing when a part or all of a first moving object falls within a measurement range of a sensor, and generating a second point cloud by superimposing a first point cloud of the measurement range obtained by the sensor at the time; determining whether a part or all of a second moving object is present in the measurement range at a predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time; Including, The determining step comprises: extracting a difference point cloud by subtracting the third point cloud from the second point cloud; determining whether or not a part or all of the second moving object is present based on the difference point cloud; Equipped with each of the first moving body and the second moving body is a train; A computer-implemented information processing method.

17. Computer, a point cloud generation unit that acquires a time including a timing when a part or all of a first moving object on a track falls within a measurement range of a sensor, and generates a second point cloud by superimposing a first point cloud of the measurement range acquired by the sensor at the time; a determination unit that determines whether a part or all of a second moving object on the railway is present in the measurement range at a predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time; It functions as The determination unit a point cloud extraction unit that extracts a difference point cloud by subtracting the third point cloud from the second point cloud; a moving object presence / absence determination unit that determines whether a part or all of the second moving object exists based on the difference point cloud; Equipped with each of the first moving body and the second moving body is a train; program.

18. a sensor for obtaining a point cloud; a point cloud generation unit that acquires a time including a timing when a part or all of a first moving object on a railway falls within a measurement range of the sensor, and generates a second point cloud by superimposing a first point cloud of the measurement range acquired by the sensor at the time; a determination unit that determines whether a part or all of a second moving object on the railway is present in the measurement range at a predetermined time based on the second point cloud and a third point cloud of the measurement range obtained by the sensor at the predetermined time; Equipped with The determination unit a point cloud extraction unit that extracts a difference point cloud by subtracting the third point cloud from the second point cloud; a moving object presence / absence determination unit that determines whether a part or all of the second moving object exists based on the difference point cloud; Equipped with each of the first moving body and the second moving body is a train; Information processing system.

Images