Change detection method, change detection system, and change detection apparatus

Abstract

In one aspect, a change detection apparatus (10) according to the present embodiment includes: a first feature calculation unit (11) that calculates a first feature of first coordinates in a first point cloud by using information about the presence of a point in each of a plurality of first subregions in a first region which is formed from the first subregions and which contains the first coordinates; a second feature calculation unit (12) that calculates a second feature of second coordinates in a second point cloud, the second coordinates corresponding to the first coordinates, using information indicating whether a point is present, a point is absent, or the presence of a point is indeterminate in each of a plurality of second subregions in a second region which is formed from the second subregions and which contains the second coordinates; and a determination unit (13) that uses the first and second features to determine whether or not a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates.

Description

Change detection method, change detection system, and change detection device

[0001] The present invention relates to a change detection method, a change detection system, and a change detection device.

[0002] BACKGROUND ART Research is being conducted into technologies for carrying out surveillance work, such as inspecting various infrastructure facilities and for crime prevention purposes, by using robots such as drones to patrol target locations and photograph predetermined locations.

[0003] For example, Patent Document 1 describes a technology that compares previously captured video with currently captured video to detect whether a change has occurred in a target area. At this time, target data of a changed area caused by work is excluded from the output. Patent Document 2 describes a technology that acquires three-dimensional distance data and uses a three-dimensional polar coordinate grid map to detect the presence or absence of an object based on the acquired three-dimensional distance data.

[0004] JP 2022-063600 A JP 2021-081235 A

[0005] When comparing two point cloud data, if the shooting environment or device used to capture the point cloud data is different, it is expected that the point densities in the point cloud data will be different. That is, the density of one point cloud will be lower than the density of the other point cloud. In this case, it becomes difficult to accurately compare the point clouds, and there is a possibility that the accuracy of change detection will decrease. While the technologies in Patent Documents 1 and 2 disclose the comparison of images and the detection of the presence or absence of objects, they do not address this issue and are unable to solve it.

[0006] The object of the present disclosure is to provide a change detection method, a change detection system, and a change detection device that can improve the accuracy of detecting changes between point groups even when the point groups being compared do not represent accurate information.

[0007] A change detection method according to one aspect of this embodiment is executed by a computer, and calculates a first feature of a first coordinate in a first point cloud using information regarding the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; calculates a second feature of a second coordinate in a second point cloud corresponding to the first coordinate using information indicating whether a point is present, not present, or the presence of a point is unknown in each of the second small regions, the second region being composed of a plurality of second small regions and including the second coordinate; and determines whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate using the first feature and the second feature.

[0008] A change detection system according to one aspect of this embodiment includes a first feature calculation means for calculating a first feature of a first coordinate in a first point cloud using information regarding the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; a second feature calculation means for calculating a second feature of a second coordinate in a second point cloud corresponding to the first coordinate using information indicating that a point exists, that a point does not exist, or that the presence of a point is unknown in each of the second small regions, the second region being composed of a plurality of second small regions and including the second coordinate; and a determination means for determining whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate using the first feature and the second feature.

[0009] A change detection device according to one aspect of this embodiment includes a first feature calculation means for calculating a first feature of a first coordinate in a first point cloud using information regarding the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; a second feature calculation means for calculating a second feature of a second coordinate in a second point cloud corresponding to the first coordinate using information indicating that a point exists, that a point does not exist, or that the presence of a point is unknown in each of the second small regions, the second region being composed of a plurality of second small regions and including the second coordinate; and a determination means for determining whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate using the first feature and the second feature.

[0010] According to the present disclosure, it is possible to provide a change detection method, a change detection system, and a change detection device that can improve the accuracy of detecting changes between point groups even when the point groups being compared do not represent accurate information.

[0011] 9B and 9C . FIG. 9C is a block diagram illustrating an example of a change detection device according to a first embodiment. FIG. 9D is a diagram for explaining calculations by a feature calculation unit according to the first embodiment. FIG. 9E is a flowchart illustrating an example of representative processing of the change detection device according to the first embodiment. FIG. 9F is a block diagram illustrating an example of a change detection system according to the first embodiment. FIG. 9G is a block diagram illustrating an example of a monitoring system according to the second embodiment. FIG. 9H is a block diagram illustrating an example of a center server according to the second embodiment. FIG. 9I shows an example of a reference point cloud according to the second embodiment. FIG. 9I shows an example of an input point cloud according to the second embodiment. FIG. 9I shows an example of a situation in which reference point cloud data is acquired according to the second embodiment. FIG. 9I shows an example of a situation in which input point cloud data is acquired according to the second embodiment. FIG. 9I shows an ideal comparison result between a reference point cloud and an input point cloud according to the second embodiment. FIG. 9I shows an actual comparison result between a reference point cloud and an input point cloud according to the second embodiment. FIG. 9I is a diagram illustrating a polar coordinate system centered on coordinates that are the object of calculation of a reference point cloud according to the second embodiment. FIG. 9I is an example of spatial feature calculated for coordinates of a reference point cloud according to the second embodiment. FIG. 9I is an example of spatial feature calculated for coordinates of an input point cloud according to the second embodiment. FIG. 9I is a diagram for explaining equation (5) in the example of FIGS. 9B and 9C . FIG. 9I is a flowchart illustrating an outline of an example of processing by a center server according to the second embodiment. FIG. 9I is a flowchart illustrating an example of detailed processing by a center server. FIG. 9I is a flowchart illustrating another example of detailed processing by a center server. 1 shows an example of an image when a change is detected by a direct comparison technique. 2 shows an example of an image when a change is detected by the technique of the present disclosure. 3 is a block diagram showing another example of a center server according to the second embodiment. 4 is a flowchart showing an outline of an example of a process by the center server according to the second embodiment. 5 is a flowchart showing an example of detailed processing by the center server. 6 is a flowchart showing another example of detailed processing by the center server. 7 is a block diagram showing another example of a center server according to the second embodiment. 8 is a block diagram showing another example of a center server according to the second embodiment. 9 is a flowchart showing an outline of an example of a process by the center server according to the second embodiment. 10 is a block diagram showing an example of a hardware configuration of an apparatus according to each embodiment.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the following description and drawings in the embodiments have been omitted or simplified as appropriate for clarity of explanation. Furthermore, in this disclosure, unless otherwise specified, when multiple items are defined as "at least one of them," the definition may mean any one item or any multiple items (including all items).

[0013] First Embodiment (1A) Hereinafter, a first embodiment of the present disclosure will be described with reference to the drawings. In this (1A), a change detection device will be described.

[0014] 1 is a block diagram showing an example of a change detection device. The change detection device 10 includes a first feature amount calculation unit 11, a second feature amount calculation unit 12, and a determination unit 13. Each unit (means) of the change detection device 10 is controlled by a control unit (controller) not shown. Each unit will be described below.

[0015] [Configuration Description] The first feature amount calculation unit 11 calculates first feature amounts of first coordinates in the first point cloud. The first point cloud is first data indicating the presence or absence of points at each coordinate in a predetermined area. The first point cloud represents, for example, the shape of an object in three-dimensional space. One example is data obtained using a sensor to visualize an object at a predetermined location. The sensor used may be, for example, a range sensor or an imaging element such as a camera. When a range sensor is used, the first point cloud becomes mapping data obtained by measuring and visualizing an object at a predetermined location. A specific example of a range sensor is LiDAR (Light Detection and Ranging), which uses light detection and ranging. Both 3D LiDAR and 2D LiDAR can be used as LiDAR. An example using 3D LiDAR will be described later in embodiment 2. When a camera is used, the first point cloud becomes mapping data generated based on a two-dimensional image of the predetermined location. The above-described measurements or photographs may be actual or virtual, but the objects represented by the first point cloud are not limited to these. The change detection device 10 may obtain the first point cloud data from an external device or may generate the first point cloud data internally.

[0016] The first feature calculation unit 11 calculates the first feature as follows: For the first point cloud, the first feature calculation unit 11 defines a first region that is composed of a plurality of first small regions and that includes first coordinates. Then, the first feature calculation unit 11 calculates the first feature by using information regarding the presence of a point in each first small region. The information regarding the presence of a point may be, for example, information indicating whether a point is present or absent in each first small region. As another example, the information regarding the presence of a point may be information indicating whether a point is present, absent, or unknown in each first small region. The definition of "unknown presence of a point" will be described later. The information regarding the presence of a point may be generated by the first feature calculation unit 11 analyzing the acquired first point cloud, or may be included in information acquired by the first feature calculation unit 11 from outside.

[0017] 2 is a diagram illustrating the calculations performed by the first feature amount calculation unit 11. In FIG. 2, a first point group is represented as G1, a first coordinate is represented as FC, and a first region including FC is represented as R1. Region R1 can be divided into small regions SR1. In FIG. 2, the presence of a point in point group G1 is represented as a black circle, and the absence of a point is represented as a white circle.

[0018] 2, the first feature amount calculation unit 11 defines a region R1 that includes a coordinate FC and is made up of a plurality of small regions SR1 for the coordinate FC for which the feature amount is to be calculated. The first feature amount calculation unit 11 then calculates the feature amount for the coordinate FC by using information about the presence of points in each small region SR1 of the region R1.

[0019] The calculated first feature may be expressed as a scalar or a vector. When the first feature is expressed as a scalar, a second feature (described later) corresponding to the first feature is also expressed as a scalar. Then, in the determination process of the determination unit 13 described later, the first feature is compared with the corresponding second feature, thereby performing the determination process described later.

[0020] Furthermore, when the first feature quantity is expressed as a vector quantity, the corresponding second feature quantity is also expressed as a vector quantity. Then, in the determination process of the determination unit 13 described below, each element in the first feature quantity is compared with each element in the second feature quantity corresponding to each element in the first feature quantity, thereby performing the determination process described below. A specific example in which the feature quantity is expressed as a vector quantity will be described in detail in embodiment 2.

[0021] 2 shows an example in which information on the presence or absence of one point is associated with one small region SR1. However, by increasing the size of the small region SR1, information on the presence or absence of multiple points may be associated with one small region SR1. The number of small regions SR1 included in the region R1 and the shapes of the region R1 and the small regions SR1 are arbitrary.

[0022] The second feature calculation unit 12 calculates second feature values ​​of second coordinates in the second point cloud, which are different from the first point cloud. The second point cloud is second data indicating the presence or absence of points at each coordinate in a predetermined region, and examples thereof are similar to those of the first point cloud. The change detection device 10 may acquire the second point cloud data from an external device or may generate the second point cloud data internally.

[0023] Here, the first point cloud and the second point cloud are point clouds to be compared, for example, mapping data obtained by measuring the same location. Also, the first coordinates and the second coordinates are compared, and for example, the first coordinates and the second coordinates may indicate the same position, but the relationship between the first coordinates and the second coordinates is not limited to this.

[0024] The change detection device 10 calculates feature amounts for a first coordinate in a first point cloud and a second coordinate in a second point cloud corresponding to the first coordinate by a first feature amount calculation unit 11 and a second feature amount calculation unit 12. By using these feature amounts, it is possible to determine whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate.

[0025] The second feature amount calculation unit 12 calculates the second feature amount as follows. The second feature amount calculation unit 12 defines, for the second point cloud, a second area that is composed of a plurality of second small areas and that includes second coordinates. The definition of this second area and the second small areas is the same as the method executed by the first feature amount calculation unit 11, as described in the example of FIG. 2 .

[0026] The second feature calculation unit 12 calculates the second feature using information indicating whether a point exists, does not exist, or is unknown in each second small region in the second region. "The presence of a point is unknown" indicates that although the presence or absence of a point is defined in the second small region in the second point cloud acquired by the second feature calculation unit 12, it is considered unknown whether the point actually exists. When the presence of a point is unknown in a second small region, the second feature calculation unit 12 calculates the feature for that small region so that it is different from either the case where a point exists or the case where a point does not exist.

[0027] The presence of a point, the absence of a point, or the unknown presence of a point is defined, for example, as follows.

[0028] In one example, the second point cloud is mapping data acquired by measurement using a range sensor. If a point exists in the second small region, the second small region is defined as having a point. On the other hand, if no point exists in the second small region, it is determined whether the second small region is located between the position where the point exists in the second point cloud at the time of measurement and the position of the range sensor. If the second small region is located between the position where the point exists in the second point cloud at the time of measurement and the position of the range sensor, the second small region is defined as having no point. On the other hand, if the second small region is not located between the position where the point exists in the second point cloud and the position of the range sensor, the presence of the point in the second small region is defined as unknown.

[0029] This definition is based on the assumption that when a range sensor acquires mapping data, light should be incident on the range sensor from an object captured in the mapping data, and no object should exist between the object and the range sensor. Ray tracing technology can be applied to this definition. This definition is particularly effective for improving detection accuracy, for example, when the density of points in the second point cloud is sparser than the density of points in the first point cloud. When the density of points in the second point cloud is sparse, a location where there are no points in the second point cloud may not only be a location where there are no points in reality, but also a location where an object was present in measurement but was not recorded as data. In this case, when it is not possible to determine that there are no points in reality, it is preferable to define the location where there are no points as a location where the presence of a point is unknown.

[0030] Another example is as follows. First, let N be the number of points present in the second small region. Furthermore, when assuming a situation in which light is incident on the range measurement sensor from every point present in the second point cloud using the above-mentioned ray tracing technique, let M be the number of times the light passes through the second small region. In this case, it is possible to define whether a point is present or unknown depending on the values ​​of N-M and N+M.

[0031] Specifically, when the value of N+M is less than a threshold Th1 (Th1 is an integer greater than or equal to 0), the presence of a point in the second small region is defined as unknown. This is because the number of points present in the second small region in the second point group is small, and the number of cases in which light from other points present in the second point group passes through the second small region is also small, making it difficult to determine whether a point exists in the second small region. On the other hand, when the value of N+M is greater than or equal to the threshold Th1, the presence or absence of a point in the second small region is defined depending on whether the value of N-M is greater than or equal to a threshold Th2 (Th2 is an integer, for example, 0, but is not limited to this). When the value of N-M is greater than or equal to the threshold Th2, it is considered that there is a high probability that a point exists in the second small region, and therefore it is defined that a point exists. On the other hand, when the value of N-M is less than the threshold (for example, a negative value), it is considered that there is a high probability that a point does not exist in the second small region, and therefore it is defined that there is no point.

[0032] The information on whether a point exists, whether a point does not exist, or whether the existence of a point is unknown, as shown above, may be generated by the second feature calculation unit 12 by analyzing the acquired second point cloud, or may be included in information acquired by the second feature calculation unit 12 from outside.

[0033] The determination unit 13 uses the first feature calculated by the first feature calculation unit 11 and the second feature calculated by the second feature calculation unit 12 to determine whether or not a change in the presence or absence of a point has occurred between the first coordinate and the corresponding second coordinate.

[0034] The determination method of the determination unit 13 may be performed by any calculation process, such as arithmetic operations, using the first feature amount and the second feature amount, or may be performed by an algorithm based on a predefined rule base. For example, if the first feature amount and the second feature amount are scalar amounts, the determination unit 13 may calculate the difference between the first feature amount and the second feature amount and determine whether the difference is equal to or greater than a threshold. If the difference is equal to or greater than the threshold, the determination unit 13 determines that a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates. If the difference is less than the threshold, the determination unit 13 determines that a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates.

[0035] As another example, if the first feature quantity and the second feature quantity are vector quantities, the determination unit 13 may compare corresponding elements of each vector quantity and, based on the comparison results for all elements, determine whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate. For example, when comparing elements, the determination unit 13 may determine whether the elements have the same value or whether the difference value of the elements is larger or smaller than a threshold, and calculate a similarity as a comparison result for all elements based on the determination. If this similarity is equal to or greater than a predetermined threshold, it is determined that no change in the presence or absence of a point has occurred between the first coordinate and the second coordinate. This applies when a point exists in both the first coordinate and the second coordinate, or when a point does not exist in either the first coordinate or the second coordinate. If this similarity is less than the predetermined threshold, it is determined that a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate. This applies when a point exists in one of the first coordinate and the second coordinate, but not in the other. An example of determination based on such an algorithm will be described later in embodiment 2.

[0036] Furthermore, the determination method of the determination unit 13 may be performed using a pre-trained AI (Artificial Intelligence) model, such as a neural network. This training is performed by inputting training data to the AI ​​model, including information on sample first and second feature quantities and information (correct answer labels) corresponding to the information, indicating whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate. After the AI ​​model has trained using the training data, the determination unit 13 inputs the first feature quantity calculated by the first feature quantity calculation unit 11 and the second feature quantity calculated by the second feature quantity calculation unit 12 to the AI ​​model. Based on this input information, the AI ​​model outputs information indicating whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate. This method also allows the determination unit 13 to perform the determination process. Any technique, such as logistic regression or a neural network, can be used to train the learning model.

[0037] 3 is a flowchart showing an example of a typical process of the change detection device 10, and this flowchart will be used to explain an overview of the process of the change detection device 10. Note that the details of each process are as described above, and therefore will not be explained again.

[0038] First, the first feature amount calculation unit 11 calculates a first feature amount of a first coordinate in the first point cloud (step S11; first feature amount calculation step). The second feature amount calculation unit 12 calculates a second feature amount of a second coordinate in the second point cloud (step S12; second feature amount calculation step). The determination unit 13 uses the first feature amount and the second feature amount to determine whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate (step S13; determination step). Note that either the process of step S11 or the process of S12 may be executed first, or both processes may be executed in parallel.

[0039] [Explanation of Effects] As described above, when the existence of a point in a certain location in the second point cloud is unknown, the second feature calculation unit 12 calculates the second feature to reflect that state. The determination unit 13 then makes a determination that reflects the first feature and the second feature. Therefore, for a location where erroneous point information is displayed, such as a point not being present (or present) in the second point cloud even though the point actually exists (or does not exist) in the second point cloud, the determination unit 13 can determine that the existence of that location is unknown. This determination result is estimated to be more accurate than when the erroneous point information is used as is. Therefore, even when the second point cloud to be compared with the first point cloud does not display accurate information, the change detection device 10 can improve the accuracy of detecting changes between the point clouds.

[0040] The determination unit 13 may perform the above determination at multiple corresponding coordinates in the first point cloud and the second point cloud to detect a change in the presence or absence of points in a predetermined region of the point cloud. For example, the determination unit 13 may perform the above determination for all coordinates in the first point cloud or all coordinates in the second point cloud. This makes it possible to detect a change in the presence or absence of points throughout the first point cloud or the second point cloud. Therefore, for example, if the first point cloud and the second point cloud are point cloud data measured at the same location, the change detection device 10 can identify a changed location in the two point clouds. A changed location is, for example, a location where an object that existed in one of the two point clouds no longer exists in the other point cloud.

[0041] The change detection device 10 may further include a detection unit that detects a change in the presence or absence of an object between the first point cloud and the second point cloud based on the above-described determination result of the determination unit 13. A specific detection method will be described later in the second embodiment.

[0042] Furthermore, the first feature calculation unit 11 may calculate the first feature using information indicating whether a point exists, whether a point does not exist, or whether the existence of a point is unknown in each first small region in the first region. The definition of whether the existence of a point is unknown is as described above. This allows a state in which the existence of a point in the first point cloud is unknown to be reflected in change detection, thereby further improving the change detection accuracy of the change detection device 10.

[0043] The change detection device 10 may further include an output unit that outputs the determination result of the determination unit 13 to an internal or external location of the change detection device 10. For example, the output unit may visually highlight the location where the determination unit 13 determines that a change in the presence or absence of a point has occurred, and output the data of the determination result to an external device (e.g., a monitor) of the change detection device 10 in a visible format such as an image or point cloud. This process may be performed for all locations where the determination unit 13 determines that a change in the presence or absence of a point has occurred, thereby presenting to the user the locations where the presence or absence of an object has changed in the two point clouds. Examples of "visual highlighting" include, but are not limited to, surrounding the location where the change in the presence or absence of a point (or the location where the presence or absence of an object has changed) with a frame, displaying the outline of the location in a color different from the outline of other objects (e.g., red), flashing the location, or filling the location to display it as a shadow. The output unit may also output the determination result of the determination unit 13 as audio via a speaker. In this way, the output unit can output an alert using an image or audio. The output unit may also output the determination result of the determination unit 13 to another device. Furthermore, the output unit may output the detection result of detecting a change in the presence or absence of an object between the first point cloud and the second point cloud, as described above.

[0044] (1B) Next, in (1B), a change detection system will be described. FIG. 4 is a block diagram showing an example of a change detection system. The change detection system 20 includes a feature amount calculation device 21 and a determination device 22. The feature amount calculation device 21 includes a first feature amount calculation unit 11 and a second feature amount calculation unit 12, and the determination device 22 includes a determination unit 13 and an output unit 14. The first feature amount calculation unit 11 to the determination unit 13 perform the same processing as that shown in (1A). When the feature amount calculation device 21 generates the first and second feature amounts, information on the generated feature amounts is output to the determination device 22. The determination unit 13 of the determination device 22 uses the information on the feature amounts to perform the processing shown in (1A). The output unit 14 of the determination device 22 is the output unit described in (1A), and outputs the determination result of the determination unit 13 to the inside or outside of the determination device 22.

[0045] As described above, the change detection process according to the present disclosure may be implemented by a single device, as shown in (1A), or may be implemented as a system in which the processes are distributed across multiple devices, as shown in (1B). Note that the device configuration shown in (1B) is merely exemplary. As another example, the first device may have a first feature calculation unit 11, and the second device may have a second feature calculation unit 12 and a determination unit 13. The first device may have an acquisition unit that acquires a first point cloud. Alternatively, three different devices may be provided, each having a first feature calculation unit 11, a second feature calculation unit 12, and a determination unit 13. Here, each device may further have an acquisition unit that acquires a first point cloud, an acquisition unit that acquires a second point cloud, and an output unit 14.

[0046] As yet another example, part or all of the change detection system 20 may be provided on a cloud server built on a cloud, or on another type of virtualized server created using virtualization technology, etc. Functions other than those provided on such servers are placed on edges. For example, in a system that monitors video captured on-site via a network, edges are devices placed on or near the site, and are also devices that are close to the terminal in terms of the network hierarchy.

[0047] In the following embodiment 2, a specific example of the change detection method described in embodiment 1 will be disclosed. However, the specific example of the change detection method described in embodiment 1 is not limited to the one shown below. Furthermore, the configurations and processes described below are merely examples and are not limited to these.

[0048] (2A) [Configuration Description] Fig. 5 is a block diagram showing an example of a monitoring system. The monitoring system 100 includes a plurality of robots 101A, 101B, and 101C (hereinafter collectively referred to as robots 101), a base station 110, a center server 120, and a reference point cloud DB 130. In the example of Fig. 5, the robots 101 are provided on the edge side (site side) of the monitoring system 100, and the center server 120 is located at a location away from the site (cloud side). Each device will be described below.

[0049] The robot 101 functions as a terminal that measures a predetermined location while moving around a monitored site to inspect infrastructure equipment for failures or abnormalities. In other words, the robot 101 is an edge device connected to a network, has a LiDAR 102, and can measure any location. The robot 101 transmits the measured point cloud data to the center server 120 via the base station 110. In this example, the robot 101 transmits the point cloud data via a wireless line. However, the point cloud data may also be transmitted via a wired line. Furthermore, the robot 101 may transmit the point cloud data acquired by measurement using the LiDAR 102 directly to the center server 120, or may transmit the acquired point cloud data to the center server 120 after performing appropriate preprocessing.

[0050] Furthermore, the robot 101 may transmit information indicating the measurement position to the center server 120 together with point cloud data acquired at that position. The robot 101 has functions such as, for example, adaptive Monte Carlo localization (AMCL) or simultaneous localization and mapping (SLAM), estimates its own position using these functions, and transmits the information to the center server 120. As another example, the robot 101 may obtain information indicating the measurement position using a satellite positioning system function such as a global positioning system (GPS), and transmit the information to the center server 120. As yet another example, the movement route and measurement points of the robot 101 may be determined in advance, and the robot 101 and the center server 120 may share this information in advance, so that the center server 120 is aware of the position of the robot 101.

[0051] The robot 101 may be, for example, an AGV (Automatic Guided Vehicle) that runs under the control of the center server 120, an AMR (Autonomous Mobile Robot) that can move autonomously, a drone, etc., but is not limited to these.

[0052] The base station 110 transfers the point cloud transmitted from each robot 101 via a network to the center server 120. For example, the base station 110 is a local 5G (5th Generation) base station, a 5G gNB (next generation Node B), an LTE eNB (evolved Node B), a wireless LAN access point, or the like, but may also be another relay device. The network is, for example, a core network such as a 5GC (5th Generation Core network) or an EPC (Evolved Packet Core), the Internet, or the like.

[0053] Note that a server other than the center server 120 may be connected to the base station 110. For example, a multi-access edge computing (MEC) server may be connected to the base station 110. The MEC server, for example, can control the bit rate of data transmitted by each robot 101 by assigning a bit rate for data transmitted by each robot 101 to the base station 110 and transmitting this information to each robot 101. Furthermore, the MEC server transmits information on the bit rate of each robot 101 to the center server 120, allowing the center server 120 to grasp the bit rate information.

[0054] 6 is a block diagram showing an example of the center server 120. The center server 120 includes a reference point cloud acquisition unit 121, an input point cloud acquisition unit 122, a reference feature amount calculation unit 123, an input feature amount calculation unit 124, a change detection unit 125, and a detection result generation unit 126. The center server 120 can detect changes and generate comparison results by comparing each piece of point cloud data measured and acquired by the robot 101 with reference data. Each unit of the center server 120 will be described below.

[0055] The reference point cloud acquisition unit 121 acquires a reference point cloud to be compared with the point cloud acquired by measurement by the robot 101. The reference point cloud is a point cloud acquired by previously measuring the location measured by the robot 101, and is stored in the reference point cloud DB 130. The reference point cloud is data representing the shape of an object in three-dimensional space.

[0056] For example, when the input point cloud acquisition unit 122 (described later) acquires point cloud data transmitted by the robot 101 (hereinafter also referred to as input point cloud), the center server 120 also acquires information on the measurement positions of the input point cloud. Furthermore, the reference point cloud DB 130 stores the reference point cloud in association with the position information at which the reference point cloud was measured. The reference point cloud acquisition unit 121 compares the measurement position information of the input point cloud with the measurement position information stored in the reference point cloud DB 130. If the comparison results in matching measurement position information stored in the reference point cloud DB 130, the reference point cloud acquisition unit 121 acquires the reference point cloud associated with the matching measurement position information. In this way, the reference point cloud acquisition unit 121 searches the reference point cloud DB 130 to acquire reference point cloud data to be compared with the point cloud measured and acquired by the robot 101.

[0057] Note that data previously measured and acquired may be stored as images rather than as a point cloud in the reference point cloud DB 130. In this case, when matching measurement position information is identified by searching the reference point cloud DB 130, the reference point cloud acquisition unit 121 acquires an image associated with the matching measurement position information and converts the data format of the image into a point cloud, thereby acquiring the reference point cloud.

[0058] The input point cloud acquisition unit 122 acquires the input point cloud acquired by the robot 101 through measurement via the communication unit of the center server 120. This allows the input point cloud acquisition unit 122 to acquire point cloud data measured in real time. However, if the robot 101 takes an image and transmits the image data to the center server 120, the input point cloud acquisition unit 122 can acquire the input point cloud by converting the data format of the image into a point cloud. Like the reference point cloud, the input point cloud is data representing the shape of an object in three-dimensional space. However, there are the following differences between the input point cloud and the reference point cloud.

[0059] When performing a real-time point cloud comparison process using the reference point cloud and input point cloud acquired as described above, the following two problems are expected. The first problem is that the point densities in the reference point cloud and the input point cloud may differ, and the second problem is that the measurement position of the robot 101 may deviate from the measurement position of the reference point cloud. Each problem is explained below.

[0060] (1) FIGS. 7A and 7B show examples of a reference point cloud and an input point cloud. The reference point cloud in FIG. 7A is 3D data measured using a sensor before the robot 101 performs measurements related to the input point cloud. "Before performing measurements related to the input point cloud" may be when the robot is deployed to the site, when the robot is inspecting the site before operating, or before work begins, such as on the morning of an operating day. FIG. 7B shows 3D data acquired by measurement by the robot 101. FIGS. 7A and 7B show data measured inside a warehouse, and a rack L inside the warehouse is captured as a point cloud. Also, in FIG. 7B, an object OB not shown in FIG. 7A is captured as a point cloud.

[0061] Comparing Figures 7A and 7B, the reference point cloud T1 has a denser point density, while the input point cloud T2 has a sparser point density. This is because the input point cloud T2 is data measured in real time, resulting in a smaller data volume. In other words, compared to the reference point cloud T1, the input point cloud T2 may not record point information at coordinates where points should be due to the presence of an object. Furthermore, the point densities of the reference point cloud T1 and the input point cloud T2 may vary depending on the measurement environment of the reference point cloud T1 and the input point cloud T2. As such, in actual use, it is expected that the point densities of the reference point cloud and the input point cloud will differ significantly. Directly comparing these two point clouds may make it difficult to accurately detect changes between the two point clouds. Therefore, it is preferable to improve the accuracy of detecting changes between the point clouds, even when the input point cloud does not provide accurate information compared to the reference point cloud due to changes in point cloud density.

[0062] (2) FIG. 8A shows an example of a situation in which a reference point cloud is acquired by measurement, and FIG. 8B shows an example of a situation in which an input point cloud is acquired by measurement. In FIG. 8A , a location I1 where factory equipment is located is measured by a dedicated sensor S. This dedicated sensor S acquires a point cloud using LiDAR. At this time, the measurement range of the dedicated sensor S is indicated by G11. Gas tanks T1 and T2 are included within the range of G11. On the other hand, in FIG. 8B , the same location I1 is measured by a LiDAR 102 of a robot 101. At this time, the measurement range of the dedicated sensor S is indicated by G12. In addition to gas tanks T1 and T2, object L1, which was not present in FIG. 8A , is included within the range of G12. In such a situation, it is preferable to compare the reference point cloud and the input point cloud to detect object L1 as a change and not detect other objects as changes. In other words, it is preferable to be able to improve the accuracy of detecting changes between point groups even when the input point group does not provide accurate information compared to the reference point group due to changes in the measurement range.

[0063] In FIG. 8B , during measurement, the robot 101 is controlled so that the estimated position of the robot 101 is the same as the position measured by the dedicated sensor S. Ideally, therefore, G11 and G12 are in the same range. However, the error in the self-position estimation of the robot 101 may become large depending on the environment in which the robot 101 is placed. Furthermore, the error may also become large when the position of the robot 101 cannot be corrected by the control of the center server 120. In such cases, as shown in FIG. 8B , a deviation occurs between G11 and G12. This deviation can be of two types: a deviation in the measurement position itself (hereinafter also referred to as a deviation in the translation direction) and a deviation in the measurement direction itself (hereinafter also referred to as a deviation in the rotation direction).

[0064] FIG. 8C shows the ideal comparison result between the reference point cloud and the input point cloud, and FIG. 8D shows the actual comparison result between the reference point cloud and the input point cloud. When G11 and G12 are in the same range, by comparing the reference point cloud and the input point cloud, as shown in FIG. 8C, only object L1 is detected as a change between the reference point cloud and the input point cloud, and other objects are not detected. However, in reality, as described above, a discrepancy occurs between G11 and G12. Therefore, as shown in FIG. 8D, not only object L1 but also gas tanks T1 and T2 may be detected as a change between the reference point cloud and the input point cloud. In this way, a discrepancy in the measurement range may result in a deterioration in change detection accuracy.

[0065] The change detection method proposed in this disclosure makes it possible to solve such problems by the following process.

[0066] Returning to Fig. 6, the description will continue. The reference feature calculation unit 123 corresponds to the first feature calculation unit 11 in the first embodiment, and calculates a feature of each coordinate in the reference point group. This feature is expressed as vectorized information on the presence or absence of points at the coordinates to be calculated and at the coordinates surrounding those coordinates, and will hereinafter also be referred to as a spatial feature. Details of the method for calculating the spatial feature by the reference feature calculation unit 123 will be described below.

[0067] 9A is a diagram showing a polar coordinate system centered on the coordinates that are the object of calculation of the reference point group. When the coordinate F1 that is the object of calculation is placed at the center O, by using the distance r and the angles (θ, φ) from the center O as parameters, it is possible to identify the coordinates that are included in a predetermined region (the first region in the first embodiment) that includes the coordinate F1.

[0068] In this example, the reference feature calculation unit 123 acquires point cloud data in a spherical region S1 of radius ε centered on coordinate F1 in the reference point cloud, and then calculates and finds parameters (r, θ, φ) in the polar coordinate system shown in FIG. 9A for each coordinate in the spherical region S1.

[0069] The reference feature calculation unit 123 divides the spherical region S1 into a plurality of small regions SR1 (first small regions in the first embodiment) so that each coordinate is included within each small region SR1. Here, the spherical region S1 is divided so that the distance r and angle (θ, φ) in each small region SR1 are discretized values. In this example, the angle θ is divided by 2α (rad), the angle φ is divided by α (rad), and the distance r is divided by d. Note that 2α is a value equal to or less than π, and d is a value equal to or less than ε / 2. In this case, the spherical region S1 is Formula 1

[0070] or Formula 2

[0071] The image is divided into a number of small regions SR1. In (1) and (2), the number of small regions SR1 is expressed using a floor function and a ceiling function, respectively. The reference feature calculation unit 123 calculates vectorized spatial features by defining the presence or absence of a point in each divided small region SR1 as an element of the spatial feature. For example, a vector notation is assumed in which 1 is indicated if a point exists at each coordinate, and 2 is indicated if a point does not exist, but examples of vector notation are not limited to this.

[0072] Fig. 9B shows an example of spatial features calculated for the coordinates of the reference point group. In Fig. 9B, in a spherical region S1 whose center is O, the presence of a point in the small region SR1 is indicated by a black circle, and the absence of a point is indicated by a white circle. Fig. 9B also shows a coordinate H1 on the spherical region S1, which will be exemplified when explaining the calculation of spatial features below.

[0073] Furthermore, the reference feature calculation unit 123 sets a neighborhood region δ1 of the coordinate F1 in the reference point cloud. The neighborhood region δ1 is a region including at least one coordinate other than the coordinate F1. For example, the neighborhood region δ1 is set as a region of radius λ1 in a polar coordinate system centered on the coordinate F1, but the setting of the neighborhood region δ1 is not limited to this. The reference feature calculation unit 123 then calculates spatial features for each coordinate other than the coordinate F1 included in the neighborhood region δ1 in the same manner as the calculation of the spatial feature for the coordinate F1 described above. In other words, for each coordinate other than the coordinate F1 included in the neighborhood region δ1, the reference feature calculation unit 123 defines a spherical region (third region) including the coordinates, which is composed of multiple small regions (third small regions), using the same method as the spherical region S1 and small region SR1. The spatial features are then calculated using information on the presence or absence of points in each small region.

[0074] Next, we will explain the calculation method of the input feature amount calculation unit 124. The input feature amount calculation unit 124 corresponds to the second feature amount calculation unit 12 in Embodiment 1, and calculates a spatial feature amount of each coordinate in the input point cloud. This spatial feature amount is expressed as a vector of information indicating whether a point exists, does not exist, or whether the existence of a point is unknown at the coordinates to be calculated and the surrounding coordinates.

[0075] When a coordinate F2 to be calculated in the input point cloud is placed at the center O, the input feature calculation unit 124 identifies a coordinate included in a predetermined region (the second region in the first embodiment) that includes the coordinate F2 by using the distance r and the angle (θ, φ) from the center O as parameters. The coordinate F2 is a coordinate that is the target of comparison with the coordinate F1 in spatial feature amounts, and here indicates the same coordinate in both the reference point cloud and the input point cloud.

[0076] The input feature calculation unit 124 acquires point cloud data in a spherical region S2 of radius ε centered on coordinate F2 from the input point cloud. The size of the spherical region S2 is the same as that of the spherical region S1. Then, the input feature calculation unit 124 calculates and obtains parameters (r, θ, φ) in the polar coordinate system shown in FIG. 9A for each coordinate in the spherical region.

[0077] The input feature amount calculation unit 124 divides the spherical region S2 into a plurality of small regions SR2 (second small regions in the first embodiment) so that each coordinate is included in the small region SR2. The spherical region S1 is divided in the same way as the spherical region S2. That is, the angle θ is divided by 2α (rad), the angle φ is divided by α (rad), and the distance r is divided by d. Note that 2α is a value equal to or less than π, and d is a value equal to or less than ε / 2. Therefore, the spherical region S2 is Formula 3

[0078] or Formula 4

[0079] The image is divided into a number of small regions SR2. (3) and (4) are the same as (1) and (2), respectively. The input feature calculation unit 124 calculates vectorized spatial features by defining, as elements of the spatial feature, information indicating whether a point exists, does not exist, or whether the existence of a point is unknown for each divided small region SR2. For example, a vector notation is assumed in which 1 is indicated if a point exists at each coordinate, 2 is indicated if a point does not exist, and 0 is indicated if the existence of a point is unknown, but examples of vector notation are not limited to this.

[0080] Here, the input feature calculation unit 124 determines information indicating whether a point exists in each small region SR2, whether a point does not exist, or whether the existence of a point is unknown, as follows: If a point exists in a small region SR2, the input feature calculation unit 124 defines that a point exists in that small region SR2. On the other hand, if a point does not exist in a small region SR2, the input feature calculation unit 124 determines whether that small region SR2 is located between the position where a point exists in the input point cloud at the time of measurement and the position of the LiDAR 102. If the small region SR2 is located between the position where a point exists in the input point cloud at the time of measurement and the position of the LiDAR 102, the input feature calculation unit 124 defines that no point exists in the small region SR2. On the other hand, if the small region SR2 is not located between the position where a point exists in the input point cloud at the time of measurement and the position of the LiDAR 102, the input feature calculation unit 124 defines that the existence of a point in the small region SR2 is unknown. Such ray tracing techniques can be applied to the definition.

[0081] As described above, the density of points in the input point cloud is sparser than the density of points in the reference point cloud. Therefore, it is possible that an object captured in the reference point cloud will not be accurately captured in the input point cloud, and some points of the object will not be recorded in the input point cloud. Therefore, when it is not possible to determine that a point does not actually exist, it is preferable to define the existence of a point as unknown for a small region SR2 that does not contain any points.

[0082] 9C is an example of spatial features calculated for the coordinates of the input point cloud. In FIG. 9C, in a spherical region S2 whose center is O at coordinate F2, the presence of a point in the small region SR2 is indicated by a black circle, the absence of a point by a white circle, and the presence of an unknown point by a triangle. Also shown in FIG. 9C is coordinate H2 on the spherical region S2, which will be exemplified when explaining the calculation of spatial features below. Note that coordinates H1 and H2 represent the same coordinates in the reference point cloud and the input point cloud.

[0083] Furthermore, the input feature calculation unit 124 sets a neighborhood region δ2 of the coordinate F2 in the input point cloud. The neighborhood region δ2 is a region including at least one coordinate other than the coordinate F2. For example, the neighborhood region δ2 is set as a region of radius λ2 in a polar coordinate system centered on the coordinate F2, but the setting of the neighborhood region δ2 is not limited to this. The input feature calculation unit 124 then calculates spatial features for each coordinate other than the coordinate F2 included in the neighborhood region δ2 in the same manner as the calculation of spatial features for the coordinate F2 described above. That is, for each coordinate other than the coordinate F2 included in the neighborhood region δ2, the input feature calculation unit 124 defines a spherical region (fourth region) including the coordinates, which is composed of multiple small regions (fourth small regions) using a method similar to that for the spherical region S2 and the small region SR2. The input feature calculation unit 124 then calculates spatial features using information indicating whether a point exists, does not exist, or the presence of a point is unknown in each small region.

[0084] Note that the size of the neighborhood region δ2 in the input point cloud may be the same as or different from the size of the neighborhood region δ1 in the reference point cloud, i.e., in this example, the radius λ1 may be the same length as the radius λ2, or may be different lengths.

[0085] In this manner, the reference feature amount calculation unit 123 and the input feature amount calculation unit 124 calculate the above-mentioned spatial feature amounts at each coordinate of the reference point group and the input point group.

[0086] Returning to Fig. 6, the description will continue. The change detection unit 125 corresponds to the determination unit 13 in the first embodiment. The change detection unit 125 calculates a similarity by comparing spatial feature amounts between a coordinate F1 in the reference point group and a coordinate F2 in the input point group. Then, using the similarity, it determines whether or not the presence or absence of a point at the coordinate F1 has changed at the coordinate F2 when the reference point group is changed to the input point group.

[0087] The change detection unit 125 can perform the following two types of determination processing: (I) Determine whether a point that did not exist in the coordinate F1 of the reference point group now exists in the coordinate F2 of the input point group; (II) Determine whether a point that existed in the coordinate F1 of the reference point group no longer exists in the coordinate F2 of the input point group. Details of processing (I) and (II) are explained below.

[0088] In the process (I), the change detection unit 125 calculates the similarity between the feature of coordinate F1 in the reference point group calculated by the reference feature calculation unit 123 and the feature of coordinate F2 in the input point group calculated by the input feature calculation unit 124. Here, the feature of coordinate F1 is denoted as P1, the feature of coordinate F2 is denoted as P2, and the similarity between the two is denoted as S(P1, P2).

[0089] The change detection unit 125 calculates S(P1, P2) as follows. Formula 5

[0090] Formula 6

[0091] Formula 7

[0092] As mentioned above, φ, θ, and r in (5) to (7) are divided into units of 2α, α, and d, respectively. P1 on the right side of (5) φθｒindicates one element of the feature P1, which is one element of the spatial feature defined in the small region SR1 in the spherical region S1. φθｒ indicates one element of the feature P2, which is one element of the spatial feature defined in the small region SR2 in the spherical region S2. φθｒ and P2 φθｒ are the elements of the feature quantities in the small regions SR1 and SR2 that are at the same position when the reference point group and the input point group are compared.

[0093] Furthermore, ValidNum in (5) is the number of small regions SR2 in the spherical region S2 other than the small region SR2 defined as "point existence unknown." Therefore, (5) is the Score(P1 φθｒ ,P2 φθｒ ) are summed and the sum is normalized by ValidNum. In this case, the small region SR2 defined as "point existence unknown" is not evaluated in the similarity calculation (i.e., it is ignored in the calculation).

[0094] (6) is Score (P1 φθｒ ,P2 φθｒ ) is defined as follows: φθｒ ,P2 φθｒ ) is P1 φθｒ and P2 φθｒ Regarding Same ａｒｏｕｎｄ (P1 φθｒ ,P2 φθｒ ) is present, it is 1, otherwise it is 0.

[0095] (7) is P1(φ+d1)(θ+d2)r, where φ ranges from -ρ1+φ1 to ρ1+φ1 in the reference region and θ ranges from -ρ2+θ2 to ρ2+θ2, and P2 φθｒ When there is a case where the value of ａｒｏｕｎｄ (P1 φθｒ ,P2 φθｒ ) is present. Note that φ1 and θ2 are φθｒThe values ​​of φ and θ of the small region SR1 in the image are also ρ1 and ρ2, which are the allowable deviation values ​​for φ and θ, respectively. φθｒ In this way, in (7), P1 of the small area SR1 is defined to include the adjacent small area SR1. φθｒ Not only the spatial feature elements of the first small region SR1 (elements of spatial features), but also the spatial feature elements of the first small region included in the surrounding area of ​​the small region SR1 are compared with the spatial feature elements of the small region SR2 in the input point cloud corresponding to the small region SR1.

[0096] Also, if the existence of a point is unknown in the small region SR1 or SR2, ａｒｏｕｎｄ (P1 φθｒ ,P2 φθｒ ) is not defined, and in (6), Score(P1 φθｒ ,P2 φθｒ ) becomes 0.

[0097] 9D is a diagram for explaining the above formula (7) in the examples of FIGS. 9B and 9C. This diagram shows a part of the spherical region S1 near the coordinate H1 in FIG. 9B and a part of the spherical region S2 near the coordinate H2 in FIG. 9C. The element in the spatial feature quantity of the small region SR1 including the coordinate H1 is P1. φθｒ The element in the spatial feature of the small region SR2 including the coordinate H2 is P2 φθｒ In addition, in FIG. 9D, the element in the spatial feature amount of the small region SR1 adjacent in the φ direction to the small region SR1 including the coordinate H1 is P1. （φ＋１）θｒ and P1 （φ－１）θｒ Furthermore, in FIG. 9D, the small region SR1 including the coordinate H1, P1 （φ＋１）θｒ The small regions SR1 and P1 （φ－１）θｒ In each of the small regions SR1, adjacent small regions SR1 are defined in the r direction. The elements of the spatial feature of these small regions SR1 are P1 φθ（ｒ－１） , P1(φ+1)θ(r−1), P1(φ−1)θ(r−1).

[0098] In the example of FIG. 9D, P1 φθｒ indicates that the point does not exist, and P2φθｒ indicates that a point exists, so they are not the same element. However, the range of φ indicated by equation (7) is （φ＋１）θｒ and P1 （φ－１）θｒ In this example, P1 （φ－１）θｒ indicates that a point exists, so P1 （φ－１）θｒ and P2 φθｒ Therefore, in the example of FIG. 9D, the Same ａｒｏｕｎｄ (P1 φθｒ ,P2 φθｒ ) exists, and Score(P1 φθｒ ,P2 φθｒ ) becomes 1.

[0099] In this way, the change detection unit 125 calculates Score(P1 φθｒ ,P2 φθｒ ), a matching process is performed to calculate the similarity S(P1, P2) shown in equation (5).

[0100] The change detection unit 125 also calculates similarity S(Pn, P2) for each coordinate other than coordinate F1 included in the neighborhood region δ1 set by the reference feature calculation unit 123, using the same calculation method as for S(P1, P2). Note that Pn is a feature at each coordinate other than coordinate F1, and is calculated by the reference feature calculation unit 123 in the above process. Hereinafter, S(PN, P2) is defined as the similarity S within neighborhood region δ1 including S(P1, P2) and S(Pn, P2).

[0101] After calculating all similarities S(PN, P2) in the neighboring region δ1 in this way, the change detection unit 125 selects the maximum similarity S(PN, P2) among S(PN, P2). ｍａｘ (PN, P2) is identified. ｍａｘ The coordinates of the reference point group corresponding to (PN, P2) are the coordinates that are most similar to the coordinate F2 in the input point group in terms of whether or not a point exists among the coordinates included in the neighboring region δ1 in the reference point group. ｍａｘ (PN, P2) is compared with a predetermined threshold value ThS1 to determine which is larger.

[0102] Sｍａｘ If (PN, P2) is equal to or smaller than ThS1, the change detection unit 125 determines that the coordinates included in the neighborhood region δ1 have a low degree of similarity with the coordinate F2. Then, the change detection unit 125 determines that a point that did not exist at the coordinate F1 of the reference point group now exists at the coordinate F2 of the input point group. On the other hand, S ｍａｘ If (PN, P2) is greater than ThS1, the change detection unit 125 does not make the above determination.

[0103] Next, the process of (II) will be described. The change detection unit 125 executes a matching process to calculate S(P1, P2), which is the similarity between the feature amount of the coordinate F1 and the feature amount of the coordinate F2. This calculation method is the same as that of (I), so its explanation will be omitted.

[0104] The change detection unit 125 also calculates similarity S(P1, Pm) for each coordinate other than coordinate F2 included in the neighborhood region δ2 set by the input feature amount calculation unit 124, using the same calculation method as for S(P1, P2). Note that Pm is a feature at each coordinate other than coordinate F2, and is calculated by the input feature amount calculation unit 124 through the above process. Hereinafter, S(P1, P M) is defined as the similarity S within neighborhood region δ2 including S(P1, P2) and S(P1, P M).

[0105] After calculating all similarities SS(P1, PM) in the neighboring region δ2 in this way, the change detection unit 125 selects the maximum similarity S(P1, PM) among S(P1, PM). ｍａｘ (P1, PM) is identified. ｍａｘ The coordinates of the reference point group corresponding to (P1, PM) are the coordinates that are most similar to the coordinate F1 in the reference point group in terms of whether or not a point exists among the coordinates included in the neighboring region δ2 in the input point group. ｍａｘ The magnitude relationship between (P1, PM) and a predetermined threshold value ThS2 is compared.

[0106] S ｍａｘIf (P1, PM) is equal to or smaller than ThS2, the change detection unit 125 determines that the coordinates included in the neighborhood region δ2 have a low degree of similarity with the coordinate F1. Then, the change detection unit 125 determines that a point that existed at the coordinate F1 of the reference point group no longer exists at the coordinate F2 of the input point group. On the other hand, S ｍａｘ If (P1, PM) is greater than ThS2, the change detection unit 125 does not make the above determination.

[0107] In (I), S ｍａｘ (PN, P2) is determined to be greater than ThS1, and in (II), S ｍａｘ If it is determined that (P1, PM) is greater than ThS2, the change detection unit 125 determines that the presence or absence of a point at the coordinate F1 has not changed at the coordinate F2, i.e., it is determined that there is no change between the coordinate F1 and the coordinate F2.

[0108] The reference feature calculation unit 123 and the input feature calculation unit 124 can change the sizes of the neighborhood regions δ1 and δ2, respectively, depending on the situation. For example, the reference feature calculation unit 123 may increase the size of the neighborhood region δ1 (e.g., the size of the radius λ1) as the distance between the position indicated by the coordinate F1 when measuring the reference point cloud and the position of the dedicated sensor S at the time of measurement increases. This is because the farther the coordinate F1 of the reference point cloud is from the dedicated sensor S at the time of measurement, the wider the influence of the above-mentioned deviation occurs, so it is preferable to set the coordinates of the reference point cloud to be compared over a wider range. For the same reason, the input feature calculation unit 124 may increase the size of the neighborhood region δ2 (e.g., the size of the radius λ2) as the distance between the position indicated by the coordinate F2 when measuring the input point cloud and the position of the LiDAR 102 at the time of measurement increases.

[0109] The change detection unit 125 performs the above processes (I) and (II) for each coordinate in the reference point cloud and each coordinate in the input point cloud corresponding to each coordinate. This allows the change detection unit 125 to detect changes in the presence or absence of points at all coordinates in the reference point cloud and the input point cloud. In (I), coordinates in the input point cloud are selected as the starting point for change detection, and the presence or absence of a change is determined for each coordinate in the input point cloud, thereby detecting whether a point not in the reference point cloud has been newly added to the input point cloud. Meanwhile, in (II), coordinates in the reference point cloud are selected as the starting point for change detection, and the presence or absence of a change is determined for each coordinate in the reference point cloud, thereby detecting whether a point in the reference point cloud is no longer in the input point cloud.

[0110] Returning to FIG. 6 , the description continues. The detection result generation unit 126 generates an image showing the change detected by the change detection unit 125 as a result of the change detection unit 125 performing the process shown in (I). For example, when the reference point cloud and the input point cloud are acquired in the situations shown in FIGS. 8A and 8B , the detection result generation unit 126 can generate the image shown in FIG. 8C as a result of the change detection unit 125 performing the process shown in (I). The detection result generation unit 126 can also generate an image showing the change detected by the change detection unit 125 as a result of the change detection unit 125 performing the process shown in (II). The image generated in this manner does not display areas determined to be unchanged in the comparison between the reference point cloud and the input point cloud, but can display areas of difference between the two. The center server 120 may have an interface, such as a display, that allows the user to view the image generated by the detection result generation unit 126. Instead of an image, the detection result generation unit 126 may generate a point cloud of 3D data showing the change detected by the change detection unit 125. Furthermore, similar to the output unit described in embodiment 1, the detection result generation unit 126 may perform processing to visually emphasize locations on the generated screen or point cloud where a change in the presence or absence of a point has occurred (or locations where a change in the presence or absence of an object has occurred).

[0111] [Explanation of Processing] Figures 10A to 10C are flowcharts showing an example of a typical process of the center server 120, and this flowchart will be used to explain an overview of the process of the center server 120. The details of each process are as described above, so explanations will be omitted where appropriate. Figure 10A is a flowchart showing an overview of an example of a process of the center server 120, and the process flow will be explained first using Figure 10A.

[0112] First, the reference point cloud acquisition unit 121 acquires reference point cloud data from the reference point cloud DB 130 (step S21; acquisition step). Furthermore, the input point cloud acquisition unit 122 acquires input point cloud data by acquiring data transmitted by the robot 101 (step S22; acquisition step). Note that the processing of steps S21 and S22 may be performed in any order, or both processes may be performed in parallel.

[0113] Next, the reference feature calculation unit 123, the input feature calculation unit 124, and the change detection unit 125 execute process (A) (step S23; process (A) step). Process (A) indicates the above process (I) and processes related thereto, and details thereof will be described using FIG. 10B.

[0114] The reference feature calculation unit 123, the input feature calculation unit 124, and the change detection unit 125 also execute process (B) (step S24; process (B) step). Process (B) indicates the above process (II) and related processes, the details of which will be explained using FIG. 10C . Note that either step S23 or S24 may be executed first, or both processes may be executed in parallel.

[0115] The detection result generation unit 126 generates an image showing the change detected in the process (A) based on the information indicating the presence or absence of a change at each coordinate generated by the process (A). Similarly, the detection result generation unit 126 generates an image showing the change detected in the process (B) based on the information indicating the presence or absence of a change at each coordinate generated by the process (B) (step S25; detection result image generation step).

[0116] Next, an example of the detailed processing (A) of the center server will be shown with reference to Fig. 10B. First, the input feature calculation unit 124 calculates spatial feature amounts for coordinates in the input point cloud for which feature amounts have not yet been calculated (step S31; feature amount calculation step). The input feature calculation unit 124 outputs information on the calculated coordinates to the reference feature calculation unit 123.

[0117] Based on the output coordinate information, the reference feature calculation unit 123 identifies coordinates in the reference point cloud that correspond to the output coordinates. Here, it is assumed that the coordinate information output by the input feature calculation unit 124 is the coordinate F2 information in the above example, and the coordinate information identified by the reference feature calculation unit 123 is the coordinate F1 information in the above example. The reference feature calculation unit 123 sets a neighborhood region δ1 that includes the coordinate F1, and calculates the spatial feature of each coordinate included in the neighborhood region δ1 (step S32; feature calculation step). Note that, although the processing of step S31 is performed before the processing of step S32 in the above example, the processing of step S32 may be performed before the processing of step S31, or both processings may be performed in parallel.

[0118] The change detection unit 125 calculates the similarity between the coordinates F1 and F2 using the spatial feature amounts calculated in steps S31 and S32 (step S33; similarity calculation step).The change detection unit 125 compares the maximum value of the calculated similarity with a predetermined threshold to determine whether a change has occurred in the presence or absence of a point at the coordinate F2 (step S34; change detection step).

[0119] The change detection unit 125 determines whether the similarity calculation and change detection determination have been completed for all coordinates in the input point cloud (step S35; completion determination step). If the similarity calculation and change detection determination have not been completed for all coordinates in the input point cloud (No in step S35), the process returns to step S31 and repeats for coordinates in the input point cloud for which similarity has not been calculated. If the similarity calculation and change detection determination have been completed for all coordinates in the input point cloud (Yes in step S35), process (A) ends. Then, as described above, the detection result generation unit 126 generates a change detection image based on the information generated by process (A) indicating the presence or absence of a change at each coordinate.

[0120] Next, an example of the detailed processing (B) of the center server will be shown with reference to Fig. 10C. First, the reference feature calculation unit 123 calculates spatial feature amounts for coordinates in the reference point group for which feature amounts have not yet been calculated (step S41; feature amount calculation step). The reference feature calculation unit 123 outputs information on the calculated coordinates to the input feature calculation unit 124.

[0121] Based on the output coordinate information, the input feature calculation unit 124 identifies coordinates in the input point cloud that correspond to the output coordinates. Here, it is assumed that the coordinate information output by the reference feature calculation unit 123 is the coordinate F1 information in the above example, and the coordinate information identified by the input feature calculation unit 124 is the coordinate F2 information in the above example. The input feature calculation unit 124 sets a neighborhood region δ2 that includes the coordinate F2, and calculates the spatial feature of each coordinate included in the neighborhood region δ2 (step S42; feature calculation step). Note that, although the processing of step S41 is performed before the processing of step S42 in the above example, the processing of step S42 may be performed before the processing of step S41, or both processes may be performed in parallel.

[0122] The change detection unit 125 calculates the similarity between the coordinates F1 and F2 using the spatial feature amounts calculated in steps S41 and S42 (step S43; similarity calculation step).The change detection unit 125 compares the maximum value of the calculated similarity with a predetermined threshold to determine whether a change has occurred in the presence or absence of a point at the coordinates F1 (step S44; change detection step).

[0123] The change detection unit 125 determines whether the similarity calculation and change detection determination have been completed for all coordinates in the reference point group (step S45; completion determination step). If the similarity calculation and change detection determination have not been completed for all coordinates in the reference point group (No in step S45), the process returns to step S41 and repeats for coordinates in the reference point group for which the similarity has not been calculated. If the similarity calculation and change detection determination have been completed for all coordinates in the reference point group (Yes in step S45), process (B) ends. Then, as described above, the detection result generation unit 126 generates an image in which changes have been detected based on the information indicating the presence or absence of changes at each coordinate, generated by process (B).

[0124] 10B, similarities may be calculated for all coordinates in the input point cloud first, and then change detection determination may be performed for each coordinate in the input point cloud. Similarly, in FIG. 10C, similarities may be calculated for all coordinates in the reference point cloud first, and then change detection determination may be performed for each coordinate in the reference point cloud.

[0125] In the above example, both processes (A) and (B) are executed, but only one of processes (A) or (B) may be executed, and an image may be generated for only the executed process by the change detection unit 125. Furthermore, when executing each process in sequence, it goes without saying that when information such as feature amounts and similarities calculated in a previous process is used in a subsequent process, the previously calculated results can be reused without needing to be calculated again.

[0126] Furthermore, when the robot 101 acquires multiple point clouds through measurement, multiple input point clouds are generated in the input point cloud acquisition unit 122. The center server 120 can generate a detection image for each input point cloud by performing the above-described comparison process between each input point cloud and the reference point cloud.

[0127] [Explanation of Effects] As described above, the input feature calculation unit 124 calculates spatial features in the input point cloud by vectorizing information indicating the presence, absence, or unknown existence of a point. The change detection unit 125 can determine whether a change in the presence or absence of a point has occurred between the reference point cloud and the input point cloud by using the calculated spatial features in the reference point cloud and the spatial features in the input point cloud. As described above, the density of points in the input point cloud is sparser than the density of points in the reference point cloud, and there may be cases where an object exists and a point is not recorded at a coordinate where it should be. Even in such cases, by performing the above processing in the center server 120, it is estimated that the accuracy of the determination is higher than when the input point cloud data is used as is. This is particularly effective when performing point cloud comparison processing in real time.

[0128] Furthermore, when calculating the spatial feature values ​​of the calculation target coordinates of the reference point cloud (coordinates F1 in the above example), the reference feature value calculation unit 123 may also calculate the feature values ​​of each coordinate in the neighborhood δ1 of the calculation target coordinates. The change detection unit 125 can determine whether a point that does not exist in the calculation target coordinates exists at the corresponding coordinates in the input point cloud, using the spatial feature values ​​of the calculation target coordinates, the spatial feature values ​​of each coordinate in the neighborhood δ1 of the calculation target coordinates, and the spatial feature values ​​of the calculation target coordinates and the corresponding coordinates in the input point cloud. This makes it possible to calculate the similarity while taking the deviation into account, even if a translational deviation occurs between the measurement of the reference point cloud and the measurement of the input point cloud, thereby suppressing deterioration in change detection accuracy.

[0129] Similarly, when calculating the spatial feature values ​​of the calculation target coordinates (coordinates F2 in the above example) in the input point cloud, the input feature value calculation unit 124 may also calculate the feature values ​​of each coordinate in the neighborhood δ2 of the calculation target coordinates. The change detection unit 125 can determine whether a point existing in the calculation target coordinates does not exist at the corresponding coordinates in the reference point cloud, using the spatial feature values ​​of the calculation target coordinates, the spatial feature values ​​of each coordinate in the neighborhood δ2 of the calculation target coordinates, and the spatial feature values ​​of the coordinates in the reference point cloud that correspond to the calculation target coordinates. This makes it possible to suppress deterioration in change detection accuracy even if a translational deviation occurs between the measurement of the reference point cloud and the measurement of the input point cloud.

[0130] 11A and 11B show images in which changes between the reference point cloud and input point cloud shown in FIGS. 7A and 7B are detected by directly comparing the presence or absence of points, and images in which changes between the two are detected using the method disclosed herein. As described above, FIGS. 7A and 7B show images of a rack L in a warehouse measured using a sensor. It is assumed that the measurement positions of the reference point cloud and the input point cloud are shifted by a predetermined position in the translation direction. Generally, the measurement conditions of the reference point cloud and the input point cloud are not exactly the same and are often different, and this situation reflects this. Furthermore, when the input point cloud is measured, an object OB is present that was not present when the reference point cloud was measured.

[0131] In image C1 of FIG. 11A and image C2 of FIG. 11B, the dotted areas are areas detected as changes between the reference point cloud and the input point cloud. The darker the dots in the images of FIG. 11A and 11B, the more strongly the change is detected. In image C1 of FIG. 11A, not only object OB, which should be detected as a change, but also rack L, which appears differently from the sensor between the reference point cloud and the input point cloud, is detected as a change. However, in image C2 of FIG. 11B, the rack L is prevented from being detected as a change, and object OB is detected as a clear change. In this way, the method disclosed herein can suppress deterioration in change detection accuracy even when a translational misalignment occurs between the measurement of the reference point cloud and the measurement of the input point cloud.

[0132] As a further effect, since deterioration in the accuracy of change detection between point clouds is suppressed, the amount of data that the robot 101 measures and acquires in order to accurately detect changes can be reduced (i.e., the number of measurements can be reduced), and the robot 101 can measure the measurement target from a greater distance. As a result, the distance that the robot 101 travels can be reduced, enabling efficient monitoring or inspection of infrastructure facilities.

[0133] Furthermore, the reference point cloud is data acquired using a sensor, and the reference point cloud acquisition unit 121 may increase the size of the neighborhood region δ1 as the distance between the position indicated by the coordinates targeted for change detection at the time of measurement and the position of the sensor increases. This allows for measuring an area farther away from the sensor, and even if the influence of deviation becomes greater, the neighborhood region can be set to cover the influence of the deviation, thereby preventing a deterioration in change detection accuracy.

[0134] Similarly, the input point cloud is data acquired using a sensor, and the input point cloud acquisition unit 122 may increase the size of the neighborhood region δ2 as the distance between the position indicated by the coordinates subject to change detection at the time of measurement and the position of the sensor increases. This allows the neighborhood region to be set so as to cover the effect of deviation even when measuring an area farther away from the sensor, thereby preventing a deterioration in change detection accuracy.

[0135] Furthermore, when a point does not exist in a small region of the input point cloud, whether to determine that the point does not exist or that the existence of the point is unknown may be determined using the ray tracing technique described above. This allows for determining that the existence of a point is unknown in a location where an object exists when the input point cloud is measured but was not recorded as a point in the input point cloud. Therefore, deterioration in change detection accuracy can be suppressed compared to when such a location is treated as if no point exists.

[0136] Furthermore, when executing process (I), the change detection unit 125 may calculate the similarity between the spatial feature of the calculation target coordinates of the reference point group and the spatial feature of the corresponding coordinates of the input point group that correspond to the calculation target coordinates, and the similarity between the spatial feature of each coordinate in the nearby region δ1 and the spatial feature of the corresponding coordinates of the input point group.

[0137] Here, the similarity between the spatial feature of the coordinates to be calculated and the spatial feature of the corresponding coordinates is calculated using elements of the spatial feature of the coordinates to be calculated in each of a predetermined small region SR1 in the spherical region S1 and a small region SR1 included in the peripheral region of that small region SR1, and elements of a small region SR2 in the spherical region S2 that corresponds to the predetermined small region SR1.

[0138] Furthermore, the similarity between the spatial features of each coordinate in the nearby region δ1 and the spatial features of the corresponding coordinate is calculated using the elements of the spatial features of a predetermined small region SR1 in the spherical region S1 of each coordinate and a small region SR1 included in the surrounding region of that small region SR1, as well as the elements of the small region SR2 in the spherical region S2 that corresponds to the predetermined small region SR1.

[0139] Similarly, when performing process (II), the change detection unit 125 may calculate the similarity between the spatial feature of the calculation target coordinates of the input point group and the spatial feature of the corresponding coordinates of the reference point group that correspond to the calculation target coordinates, and the similarity between the spatial feature of each coordinate in the nearby region δ2 and the spatial feature of the corresponding coordinates of the reference point group.

[0140] Here, the similarity between the spatial features of the coordinates to be calculated and the spatial features of the corresponding coordinates is calculated using elements of the spatial features of the coordinates to be calculated in each of a predetermined small region SR2 in the spherical region S2 and a small region SR2 included in the peripheral region of that small region SR2, and elements of a small region SR1 in the spherical region S1 that corresponds to the predetermined small region SR2.

[0141] Furthermore, the similarity between the spatial features of each coordinate in the nearby region δ2 and the spatial features of the corresponding coordinate is calculated using the elements of the spatial features of a predetermined small region SR2 in the spherical region S2 of each coordinate and a small region SR2 included in the peripheral region of that small region SR2, as well as the elements of the small region SR1 in the spherical region S1 that corresponds to the predetermined small region SR2.

[0142] By this processing, even if a rotational misalignment occurs between the measurement of the reference point group and the measurement of the input point group, the similarity calculation can take that misalignment into account, thereby preventing a deterioration in change detection accuracy. In the above processing, the change detection unit 125 may also appropriately change the size of at least one of the surrounding areas of small region SR1 and small region SR2. This allows the tolerance for rotational misalignment to be changed.

[0143] Note that the reference feature calculation unit 123 may also calculate spatial features in the input point cloud by vectorizing information indicating the presence, absence, or unknown existence of a point in the same manner as the input feature calculation unit 124. The ray tracing technology described in the description of the input feature calculation unit 124 can be applied to the definition of the information indicating the unknown existence of a point.

[0144] The change detection unit 125 calculates the similarity using the spatial feature calculated in this manner and the spatial feature calculated by the input feature calculation unit 124, using the method described above. In this case, in the process of (II), ValidNum in equation (5) is the number of small regions SR1 in the spherical region S1 other than the small region SR1 defined as "where the presence of a point is unknown." Therefore, the small regions SR1 defined as "where the presence of a point is unknown" are not evaluated in the calculation of the similarity. This allows the state in which the presence of a point in the reference point cloud is unknown to be reflected in the change detection, thereby further improving the accuracy of change detection between the reference point cloud and the input point cloud.

[0145] The reference point cloud and the input point cloud may be mapping data generated based on two-dimensional images of a specified location captured by a camera or a positioning sensor. In addition to the above example, the following cases are also considered as cases in which a misalignment may occur between the captured images of the reference point cloud and the captured images of the input point cloud. For example, a misalignment may occur when images of the same location are captured at different times using a mobile camera carried by a person and the reference point cloud and the input point cloud are generated based on those images. Furthermore, even if images of the same location are captured at different times by a fixed camera attached to a facility, a misalignment may occur due to a shift in the position or capturing direction of the fixed camera due to an earthquake or facility deterioration. Even in such cases, the above processing can be performed to prevent a deterioration in change detection accuracy.

[0146] The change detection unit 125 may perform process (I) on a plurality of coordinates in the input point cloud that are not all of the coordinates and a plurality of coordinates in the reference point cloud that correspond to those coordinates. Similarly, the change detection unit 125 may perform process (II) on a plurality of coordinates in the reference point cloud that are not all of the coordinates and a plurality of coordinates in the input point cloud that correspond to those coordinates. The detection result generation unit 126 generates an image showing the changes detected by the change detection unit 125 based on this determination result. In this way, if there is an area in the point cloud that does not require determination, the center server 120 can detect changes in the coordinates excluding that area.

[0147] (2B) [Configuration Description] Fig. 12 is a block diagram showing another example of the center server according to the second embodiment. In addition to the components shown in Fig. 6, the center server 120 further includes an extraction unit 127. Below, the processing of the center server 120 will be described in (2A) and only the points specific to this example will be explained.

[0148] The extraction unit 127 directly compares the presence or absence of points at corresponding coordinates between the reference point group acquired by the reference point group acquisition unit 121 and the input point group acquired by the input point group acquisition unit 122. As a result, information on coordinates where the presence or absence of points at corresponding coordinates differs between the reference point group and the input point group (information on changed coordinates) is extracted, and other coordinates are not extracted.

[0149] The reference feature calculation unit 123 and the input feature calculation unit 124 perform the process shown in (2A) for the coordinates of the reference point group and the coordinates of the input point group extracted in this way to calculate spatial features. Furthermore, the change detection unit 125 and the detection result generation unit 126 perform the process shown in (2A) using the calculated spatial features.

[0150] 13A to 13C are flowcharts showing an example of a typical process of the center server 120 in (2B), corresponding to Figures 10A to 10C, and provide an overview of the process of the center server 120. Note that the same points as in (2A) will be omitted as appropriate.

[0151] 13A , the reference point cloud acquisition unit 121 acquires reference point cloud data from the reference point cloud DB 130 (step S21; acquisition step). Also, the input point cloud acquisition unit 122 acquires input point cloud data by acquiring data transmitted by the robot 101 (step S22; acquisition step).

[0152] Next, the extraction unit 127 directly compares the presence or absence of points at corresponding coordinates between the reference point group acquired by the reference point group acquisition unit 121 and the input point group acquired by the input point group acquisition unit 122. This detects information about changed coordinates (step S26; change detection step). The reference feature calculation unit 123, the input feature calculation unit 124, and the change detection unit 125 execute process (A') (step S23'; process (A') step). Details of process (A') will be described using FIG. 13B.

[0153] The reference feature calculation unit 123, the input feature calculation unit 124, and the change detection unit 125 also execute process (B') (step S24'; process (B') step). Process (B') represents the above process (II) and related processes, the details of which will be described with reference to FIG. 13C. Note that either step S23' or S24' may be executed first, or both may be executed in parallel.

[0154] The detection result generation unit 126 generates an image showing the change detected in the process (A') based on the information indicating the presence or absence of a change at each coordinate generated by the process (A'). Similarly, the detection result generation unit 126 generates an image showing the change detected in the process (B') based on the information indicating the presence or absence of a change at each coordinate generated by the process (B') (step S25; detection result image generation step).

[0155] Next, an example of the detailed processing (A') of the center server will be shown with reference to Fig. 13B. First, the input feature calculation unit 124 calculates spatial feature amounts for coordinates in the input point cloud extracted by the extraction unit 127, for which feature amounts have not yet been calculated (step S31; feature amount calculation step). The input feature calculation unit 124 outputs information on the calculated coordinates to the reference feature calculation unit 123.

[0156] Based on the output coordinate information, the reference feature calculation unit 123 identifies a coordinate F1 in the reference point group that corresponds to the coordinate. The reference feature calculation unit 123 sets a neighborhood region δ1 that includes the coordinate F1, and calculates the spatial feature of each coordinate included in the neighborhood region δ1 (step S32; feature calculation step). As described above, the process of step S31 or the process of step S32 may be performed in any order, or both processes may be performed in parallel.

[0157] The change detection unit 125 calculates the similarity between the coordinates F1 and F2 using the spatial feature amounts calculated in steps S31 and S32 (step S33; similarity calculation step).The change detection unit 125 compares the maximum value of the calculated similarity with a predetermined threshold to determine whether a change has occurred in the presence or absence of a point at the coordinate F2 (step S34; change detection step).

[0158] The change detection unit 125 determines whether the similarity calculation and change detection determination have been completed for all coordinates extracted from the input point cloud (step S36; completion determination step). If the similarity calculation and change detection determination have not been completed for all extracted coordinates (No in step S36), the process returns to step S31 and repeats for coordinates for which similarity has not been calculated. If the similarity calculation and change detection determination have been completed for all extracted coordinates (Yes in step S36), process (A') ends. Then, as described above, the detection result generation unit 126 generates a change detection image based on the information indicating the presence or absence of a change at each coordinate, generated by process (A').

[0159] Next, an example of the detailed processing (B') of the center server is shown using Fig. 13C. First, the reference feature calculation unit 123 calculates spatial feature amounts for coordinates in the reference point group extracted by the extraction unit 127 for which feature amounts have not yet been calculated (step S41; feature amount calculation step). The reference feature calculation unit 123 outputs information on the calculated coordinates to the input feature calculation unit 124.

[0160] Based on the output coordinate information, the input feature calculation unit 124 identifies coordinate F2 in the input point cloud that corresponds to the coordinate. The input feature calculation unit 124 sets a neighborhood region δ2 that includes coordinate F2, and calculates the spatial feature of each coordinate included in neighborhood region δ2 (step S42; feature calculation step). As described above, the process of step S41 or the process of step S42 may be performed in any order, or both processes may be performed in parallel.

[0161] The change detection unit 125 calculates the similarity between the coordinates F1 and F2 using the spatial feature amounts calculated in steps S41 and S42 (step S43; similarity calculation step).The change detection unit 125 compares the maximum value of the calculated similarity with a predetermined threshold to determine whether a change has occurred in the presence or absence of a point at the coordinates F1 (step S44; change detection step).

[0162] The change detection unit 125 determines whether the similarity calculation and change detection determination have been completed for all coordinates extracted from the reference point cloud (step S46; completion determination step). If the similarity calculation and change detection determination have not been completed for all extracted coordinates (No in step S46), the process returns to step S41 and repeats for coordinates for which similarity has not been calculated. If the similarity calculation and change detection determination have been completed for all extracted coordinates (Yes in step S46), process (B') ends. Then, as described above, the detection result generation unit 126 generates an image in which changes have been detected based on the information indicating the presence or absence of changes at each coordinate generated by process (B').

[0163] As described above, the center server 120 can extract information on the changed coordinates using the extraction unit 127 and execute the change detection process shown in (2A) for the extracted coordinates. This reduces the calculation costs required for all steps of the process compared to executing the change detection process shown in (2A) for all coordinates of the input point cloud or reference point cloud.

[0164] (2C) Fig. 14A is a block diagram showing another example of the center server according to the second embodiment. In addition to the components shown in Fig. 12, the center server 120 further includes an object identification unit 128. Below, with regard to the processing of the center server 120, the points explained in (2A) and (2B) will be omitted, and only points specific to this example will be explained.

[0165] The object identification unit 128 determines whether the change in the presence or absence of a point shown in the image generated by the detection result generation unit 126 as a result of executing the processing shown in (I) or the image generated by the detection result generation unit 126 as a result of executing the processing shown in (II) corresponds to any object (e.g., a container, a vehicle, construction materials, etc.).

[0166] For example, the object identification unit 128 determines whether the changed point cloud portion is equal to or larger than a predetermined size, and if it is equal to or larger than the predetermined size, determines that the changed point cloud portion corresponds to some kind of object.If the changed point cloud portion is smaller than the predetermined size, the object identification unit 128 determines that the changed point cloud portion is noise.

[0167] As another example, the object identification unit 128 may compare the location of the changed point cloud with pre-stored point cloud data of various objects for determination, such as containers, vehicles, construction materials, etc. If the location of the changed point cloud matches the point cloud data of any object, or matches it within an error of, for example, a few percent, the object identification unit 128 identifies the matching object as an object whose presence or absence has changed between the input point cloud and the reference point cloud.

[0168] Alternatively, the object identification unit 128 may perform object determination using a pre-trained AI model. This training is performed by inputting training data into the AI ​​model, including image information indicating a changed point cloud as a sample and information indicating various objects corresponding to that information (correct labels). After the AI ​​model has been trained using the training data, the object identification unit 128 inputs the image generated by the detection result generation unit 126 into the AI ​​model. Based on this input image, the AI ​​model outputs information indicating the object shown in the input image. This also allows the object identification unit 128 to perform object identification processing. Note that any technique, such as logistic regression or a neural network, can be used to train the learning model.

[0169] Even when the detection result generation unit 126 generates a point cloud of 3D data as the detection result, the object identification unit 128 determines whether the change in the presence or absence of points shown in the point cloud, which is the detection result, corresponds to some kind of object. For example, the detection result generation unit 126 can perform object determination using a pre-trained AI model. This learning is performed by inputting training data including a sample changed point cloud and information (correct labels) indicating various objects corresponding to that information into the AI ​​model. After the AI ​​model has been trained using the training data, the object identification unit 128 inputs the point cloud generated by the detection result generation unit 126 into the AI ​​model. Based on this point cloud, the AI ​​model can output information indicating the object indicated by the point cloud. Other determination methods that the detection result generation unit 126 can perform are as described above.

[0170] [Explanation of Effects] In this way, the center server 120 can detect whether there has been a change in the presence or absence of an object between the reference point cloud and the input point cloud. More preferably, the center server 120 can also identify an object whose presence or absence has changed between the reference point cloud and the input point cloud. The object identification unit 128 may generate and output a screen on which a process for visually highlighting the identified object has been performed. The visual highlighting process is as described in the first embodiment. Furthermore, when an object is identified, the object identification unit 128 may output an audio alert via a speaker. Furthermore, the object identification unit 128 may output the determination result of the determination unit 13 to another device.

[0171] (2D) Fig. 14B is a block diagram showing another example of the center server according to the second embodiment. In addition to the components shown in Fig. 14A, the center server 120 further includes a mobility control unit 129. Below, with regard to the processing of the center server 120, the points explained in (2A) to (2C) will be omitted, and only points specific to this example will be explained.

[0172] The movement control unit 129 controls the movement of the robot 101. For example, when the object identification unit 128 determines that a location of the changed point cloud corresponds to some kind of object, the movement control unit 129 can instruct the robot 101 to approach the location of the changed point cloud and then further measure the location. This instruction is given in order to obtain and analyze more detailed point cloud data of the location. Alternatively, the movement control unit 129 may control the robot 101 as described above when the object identification unit 128 identifies a specific type of object.

[0173] This instruction is output in at least one of the following cases: (I) When it is determined that an object does not exist in the reference point cloud but an object now exists in the input point cloud, or (II) When it is determined that an object exists in the reference point cloud but an object no longer exists in the input point cloud, in order to perform a detailed analysis of the change location, however, it is preferable that the instruction be output at least when an object does not exist in the reference point cloud but an object now exists in the input point cloud.

[0174] 15A and 15B are flowcharts showing an example of a typical process of the center server 120 in (2D), corresponding to Fig. 13A, and provide an overview of the process of the center server 120. Note that the same points as in (2A) and (2B) will be omitted as appropriate.

[0175] Steps S21 to S25 are the same as those in FIG. 13A , and therefore will not be described here. After step S25, the object identification unit 128 executes the process shown in (2C) and determines whether the changed point cloud corresponds to an object in the image generated by the detection result generation unit 126 through the process in (I) (step S27; object detection determination step). If the changed point cloud does not correspond to an object (No in step S27), the movement control unit 129 does not execute any special control. In this case, the robot 101 moves, for example, along a previously set movement route. On the other hand, if the changed point cloud corresponds to an object (Yes in step S27), the movement control unit 129 controls the robot 101 to approach the changed point cloud and then further measure the changed point (step S28; robot movement step). In this case, the robot 101 temporarily deviates from the previously set movement route and moves.

[0176] It should be noted that the processes shown in steps S27 and S28 can also be performed on the image generated by the detection result generating unit 126 through the process (II). Furthermore, the center server 120 can also perform the processes shown in (2A) to (2D) on the image acquired by the robot approaching and measuring as a result of the process in step S28.

[0177] [Explanation of Effect] As described above, when an object is detected in an image generated by the detection result generation unit 126, the center server 120 controls the robot 101 to acquire detailed point cloud data. This enables more detailed inspection of infrastructure equipment for failures or abnormalities. Note that even when the detection result generation unit 126 generates a point cloud, the movement control unit 129 can execute similar control based on the object determination result of the object identification unit 128.

[0178] As another example, when the movement control unit 129 determines that the object detected by the object identification unit 128 is present on a previously set movement route, the movement control unit 129 may set the movement route to a route that avoids the location where the object is present. The movement control unit 129 controls the movement of the robot 101 so that the robot 101 moves on the newly set route.

[0179] The present disclosure is not limited to the above-described embodiments and can be modified as appropriate without departing from the spirit of the present disclosure. For example, the configurations or processes described in the above-described embodiments can be combined as desired. Furthermore, the order of processes described in each flowchart may be changed as appropriate, rather than as shown in the drawings, and multiple processes may be executed in parallel.

[0180] For example, in Fig. 10B, the processes of steps S31 to S34 are repeatedly executed for each point in the input point cloud. However, instead of this, the process of calculating spatial features for all coordinates to be processed in the input point cloud and the process of calculating spatial features for an area including coordinates of the reference point cloud corresponding to each coordinate of the input point cloud may be executed first. Then, the processes of steps S33 to S34 are executed for all coordinates, thereby detecting changes in the presence or absence of points in the area to be processed. Similar process order changes are possible in other flowcharts as well.

[0181] In the second embodiment, polar coordinates are used to define the spherical region S1, which is the region surrounding the coordinate F1, and the spherical region S2, which is the region surrounding the coordinate F2. However, it is possible to define the regions surrounding the coordinates F1 and F2 by using other types of coordinate systems, such as cylindrical coordinates, instead of polar coordinates. However, using polar coordinates has the effect of facilitating the calculations of the reference feature calculation unit 123 and the input feature calculation unit 124.

[0182] The similarity calculation methods shown in (5) to (7) in the second embodiment can be modified as appropriate. For example, in (5), the change detection unit 125 calculates the Score(P1 φθｒ ,P2 φθｒ) was simply added to calculate S(P1, P2). However, in other ways, ａｒｏｕｎｄ (P1 φθｒ ,P2 φθｒ ) exists in the entire range of φ, θ, and r, S(P1, P2) may be calculated so that S(P1, P2) monotonically increases. Also, in (5), the change detection unit 125 calculates S(P1, P2) as proportional to the reciprocal of ValidNum. However, S(P1, P2) may be calculated by other methods so that S(P1, P2) monotonically decreases with respect to ValidNum.

[0183] In the second embodiment, any part of the above-described processing executed by each unit of the center server 120 may be executed by at least one of the robot 101 and a different server. In other words, the processing of the center server 120 may be realized by a distributed system.

[0184] As another example, the center server 120 may not be provided, and the robot 101 may perform the above-described processing of the center server 120 on a standalone basis. In this case, a reference point cloud and position information at which the reference point cloud was measured are stored in association with each other in a memory unit within the robot 101. The robot 101 performs measurements using its own LiDAR 102 and acquires an input point cloud. The robot 101 searches the memory unit using the position information at which the input point cloud was measured and acquires data on a reference point cloud to be compared with the input point cloud. Details of this are similar to the processing of the reference point cloud acquisition unit 121 described above. The robot 101 can then execute processing related to the reference feature calculation unit 123 to the detection result generation unit 126 described above. The robot 101 may also execute processing related to the object identification unit 128. Furthermore, when the object identification unit 128 detects an object, the robot 101 can control its own movement unit to approach a location of the changed point cloud and then further measure that location.

[0185] In the above-described embodiments, this disclosure has been described as a hardware configuration, but this disclosure is not limited to this. This disclosure can also be realized by having a processor in a computer execute a computer program to perform the processes (steps) of each device or center server in the change detection device and change detection system described in the above-described embodiments.

[0186] 16 is a block diagram showing an example of the hardware configuration of an information processing device 90 that executes the processes of the above-described embodiments. Referring to FIG. 16, the information processing device 90 includes a signal processing circuit 91, a processor 92, and a memory 93.

[0187] The signal processing circuit 91 is a circuit for processing signals in accordance with the control of the processor 92. The signal processing circuit 91 may include a communication circuit for receiving signals from a transmitting device.

[0188] The processor 92 is connected (coupled) to the memory 93, and performs the processing of the device described in the above embodiment by reading and executing software (computer programs) from the memory 93. Examples of the processor 92 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Demand-Side Platform), and an ASIC (Application Specific Integrated Circuit). A single processor may be used as the processor 92, or multiple processors may be used in cooperation with each other.

[0189] The memory 93 may be a volatile memory, a nonvolatile memory, or a combination thereof. The volatile memory may be, for example, a random access memory (RAM) such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The nonvolatile memory may be, for example, a read only memory (ROM) such as a programmable random only memory (PROM) or an erasable programmable read only memory (EPROM), a flash memory, or a solid state drive (SSD). The memory 93 may be a single memory or a combination of multiple memories.

[0190] The memory 93 is used to store one or more instructions. Here, the one or more instructions are stored as a group of software modules in the memory 93. The processor 92 can perform the processes described in the above embodiments by reading and executing the group of software modules from the memory 93.

[0191] The memory 93 may include a memory provided outside the processor 92, as well as a memory built into the processor 92. The memory 93 may also include a storage device located away from the processors constituting the processor 92. In this case, the processor 92 can access the memory 93 via an I / O (Input / Output) interface.

[0192] As described above, one or more processors included in each device in the above-described embodiments execute one or more programs including instructions for causing a computer to execute the algorithms described using the drawings. This processing enables the information processing described in each embodiment to be realized.

[0193] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0194] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes: (Supplementary Note 1) A change detection method executed by a computer, which calculates a first feature amount of a first coordinate in a first point cloud using information regarding the presence of a point in each of the first small regions, in a first region consisting of a plurality of first small regions and including the first coordinate; calculates a second feature amount of a second coordinate in a second point cloud corresponding to the first coordinate using information indicating, in each of the second small regions, that a point is present, that a point is not present, or that the presence of a point is unknown, in a second region consisting of a plurality of second small regions and including the second coordinate; and determines whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate using the first feature amount and the second feature amount. (Supplementary Note 2) The change detection method according to Supplementary Note 1, in which the first feature amount is calculated using information indicating, in each of the first small regions, that a point is present, that a point is not present, or that the presence of a point is unknown. (Supplementary Note 3) The change detection method according to Supplementary Note 1 or 2, further calculating a feature amount of each coordinate in a first neighborhood of the first coordinate in the first point cloud, in a third neighborhood composed of a plurality of third small neighborhoods and including the coordinate, using information on the presence of points in each of the third small neighborhoods, and determining whether a point that does not exist in the first coordinate is present in the second coordinate using the feature amount of each coordinate in the first neighborhood, the first feature amount, and the second feature amount. (Supplementary Note 4) The change detection method according to Supplementary Note 3, wherein the first point cloud data is data acquired using a first sensor, and the size of the first neighborhood is increased as the distance between the position indicated by the first coordinate and the position of the first sensor when the first point cloud data is acquired increases.(Supplementary Note 5) The change detection method according to any one of Supplementary Notes 1 to 4, further calculating a feature amount of each coordinate in a second neighboring region of the second coordinate in the second point cloud using information indicating whether a point is present, not present, or the presence of a point is unknown in each of the fourth small regions, in a fourth region consisting of a plurality of fourth small regions and including the coordinate, and determining whether a point present at the first coordinate is not present at the second coordinate using the feature amount of each coordinate in the second neighboring region, the first feature amount, and the second feature amount. (Supplementary Note 6) The change detection method according to any one of claims 1 to 5, (Supplementary Note 7) The change detection method according to any one of Supplementary Notes 1 to 6, wherein the second point cloud data is data acquired by a second sensor, and when the second small region in which no point exists in the second point cloud is located between a position where a point exists in the second point cloud at the time of data acquisition and the position of the second sensor, the second small region is defined as having no point, and when the second small region is not located between a position where a point exists in the second point cloud and the position of the second sensor, the presence of a point in the second small region is defined as being unknown. (Supplementary Note 8) The change detection method according to any one of Supplementary Notes 1 to 7, wherein the first point cloud and the second point cloud are compared, and coordinates where the presence or absence of a point differs between the first point cloud and the second point cloud are extracted, and at least one of the first coordinates or the second coordinates is extracted as coordinates where the presence or absence of a point differs.(Supplementary Note 9) The change detection method described in Supplementary Note 2, wherein the first point cloud data is data acquired by a first sensor, and for the first small area in which no point exists in the first point cloud, if the first small area is located between a position in the first point cloud where a point exists at the time of acquiring the data and the position of the first sensor, the first small area is defined as having no point, and if the first small area is not located between a position in the first point cloud where a point exists and the position of the first sensor, the presence of a point in the first small area is defined as being unknown. (Supplementary Note 10) The change detection method according to Supplementary Note 3 or 4, wherein the similarity between the first feature and the second feature and, for each coordinate in the first neighboring region, the similarity between the feature of the coordinate and the second feature are calculated to determine whether a point that does not exist at the first coordinate exists at the second coordinate; the similarity between the first feature and the second feature is calculated using elements of the first feature in each of the first small region in the first region and a first small region included in a peripheral region of the first small region, and elements of the second small region in the second region corresponding to the first small region; and the similarity between the feature of each coordinate in the first neighboring region and the second feature is calculated using elements of the first feature in each of the third small region in the third region and a small region included in a peripheral region of the third small region, and elements of the second small region in the second region corresponding to the third small region. (Supplementary Note 11) The change detection method according to Supplementary Note 5, wherein the second point cloud data is data acquired using a second sensor, and the size of the second neighborhood area is increased as the distance between the position indicated by the second coordinates and the position of the second sensor increases when the second point cloud data is acquired.(Supplementary Note 12) The change detection method according to Supplementary Note 5 or 11, wherein the similarity between the first feature and the second feature and, for each coordinate in the second neighboring region, the similarity between the feature of the coordinate and the first feature are calculated to determine whether a point present at the first coordinate does not exist at the second coordinate; the similarity between the first feature and the second feature is calculated using an element of the second feature in each of the second small region in the second region and a second small region included in a peripheral region of the second small region, and an element of the first small region in the first region corresponding to the second small region; and the similarity between the feature of each coordinate in the second neighboring region and the first feature is calculated using an element of the fourth feature in each of the fourth small region in the fourth region and a small region included in a peripheral region of the fourth small region, and an element of the first small region in the first region corresponding to the fourth small region. (Supplementary Note 13) A change detection system comprising: a first feature amount calculation means for calculating a first feature amount of a first coordinate in a first point cloud using information regarding the presence of a point in each of the first small regions, in a first region consisting of a plurality of first small regions and including the first coordinate; a second feature amount calculation means for calculating a second feature amount of a second coordinate in a second point cloud corresponding to the first coordinate using information indicating, in each of the second small regions and including the second coordinate, that a point is present, that a point is not present, or that the presence of a point is unknown; and a determination means for determining whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate using the first feature amount and the second feature amount. (Supplementary Note 14) The change detection system according to Supplementary Note 13, wherein the first feature amount calculation means calculates the first feature amount using information indicating, in each of the first small regions, that a point is present, that a point is not present, or that the presence of a point is unknown.(Supplementary Note 15) The change detection system according to Supplementary Note 13 or 14, wherein the first feature amount calculation means further calculates feature amounts of each coordinate in a first neighborhood of the first coordinate in the first point cloud, in a third neighborhood composed of a plurality of third small neighborhoods and including the coordinate, using information on the presence of points in each of the third small neighborhoods, and the determination means determines whether a point that does not exist at the first coordinate is present at the second coordinate using the feature amount of each coordinate in the first neighborhood, the first feature amount, and the second feature amount. (Supplementary Note 16) The change detection system according to Supplementary Note 15, wherein the first point cloud data is data acquired using a first sensor, and the first feature amount calculation means increases the size of the first neighborhood as the distance between the position indicated by the first coordinate and the position of the first sensor when the first point cloud data is acquired increases. (Supplementary Note 17) The change detection system according to any one of Supplementary Notes 13 to 16, wherein the second feature amount calculation means further calculates feature amounts of each coordinate in a second neighboring region of the second coordinate in the second point cloud using information indicating that a point is present, that a point is not present, or that the presence of a point is unknown in each of the fourth small regions, in a fourth region consisting of a plurality of fourth small regions and including the coordinate, and the determination means determines whether a point present at the first coordinate is not present at the second coordinate using the feature amount of each coordinate in the second neighboring region, the first feature amount, and the second feature amount. (Supplementary Note 18) The change detection system according to any one of Supplementary Notes 13 to 17, further comprising: a detection unit that detects a change in the presence or absence of an object between the first point cloud and the second point cloud by performing the determination at a plurality of corresponding coordinates in the first point cloud and the second point cloud; and an output unit that outputs a result of the detection.(Supplementary Note 19) The change detection system according to any one of Supplementary Notes 13 to 18, wherein the second point cloud data is data acquired by a second sensor, and wherein, for the second small region where no point is present in the second point cloud, if the second small region is located between a position where a point was present in the second point cloud at the time of acquiring the data and the position of the second sensor, the second small region is defined as having no point, and if the second small region is not located between a position where a point was present in the second point cloud and the position of the second sensor, the presence of a point in the second small region is defined as being unknown. (Supplementary Note 20) The change detection system according to any one of Supplementary Notes 13 to 19, further comprising an extraction unit that compares the first point cloud with the second point cloud and extracts coordinates where the presence or absence of a point differs between the first point cloud and the second point cloud, and at least one of the first coordinates or the second coordinates is coordinates extracted by the extraction unit. (Supplementary Note 21) The change detection system described in Supplementary Note 14, wherein the first point cloud data is data acquired by a first sensor, and for the first small area in which no point exists in the first point cloud, if the first small area is located between a position in the first point cloud where a point exists at the time of acquiring the data and the position of the first sensor, the first small area is defined as having no point, and if the first small area is not located between a position in the first point cloud where a point exists and the position of the first sensor, the presence of a point in the first small area is defined as being unknown.(Supplementary Note 22) The change detection system described in Supplementary Note 15 or 16, wherein the determination means determines whether a point that does not exist at the first coordinate exists at the second coordinate by calculating a similarity between the first feature and the second feature and, for each coordinate in the first neighboring region, a similarity between the feature of the coordinate and the second feature; the similarity between the first feature and the second feature is calculated using an element of the first feature in each of the first small region in the first region and a first small region included in a peripheral region of the first small region, and an element of the second small region in the second region corresponding to the first small region; and the similarity between the feature of each coordinate in the first neighboring region and the second feature is calculated using an element of the first feature in each of the third small region in the third region and a small region included in a peripheral region of the third small region, and an element of the second small region in the second region corresponding to the third small region. (Supplementary Note 23) The change detection system described in Supplementary Note 17, wherein the second point cloud data is data acquired using a second sensor, and the second feature amount calculation means increases the size of the second neighborhood area as the distance between the position indicated by the second coordinates and the position of the second sensor at the time of acquiring the second point cloud data increases.(Supplementary Note 24) The change detection system described in Supplementary Note 17 or 23, wherein the determination means determines whether a point present at the first coordinate does not exist at the second coordinate by calculating a similarity between the first feature and the second feature and, for each coordinate in the second neighboring region, a similarity between the feature of the coordinate and the first feature; the similarity between the first feature and the second feature is calculated using an element of the second feature in each of the second small region in the second region and a second small region included in a peripheral region of the second small region, and an element of the first small region in the first region corresponding to the second small region; and the similarity between the feature of each coordinate in the second neighboring region and the first feature is calculated using an element of the fourth feature in each of the fourth small region in the fourth region and a small region included in a peripheral region of the fourth small region, and an element of the first small region in the first region corresponding to the fourth small region. (Supplementary Note 25) A change detection device comprising: a first feature amount calculation means for calculating a first feature amount of a first coordinate in a first point cloud using information regarding the presence of a point in each of the first small regions, in a first region consisting of a plurality of first small regions and including the first coordinate; a second feature amount calculation means for calculating a second feature amount of a second coordinate in a second point cloud corresponding to the first coordinate using information indicating, in each of the second small regions and including the second coordinate, that a point is present, that a point is not present, or that the presence of a point is unknown; and a determination means for determining, using the first feature amount and the second feature amount, whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate. (Supplementary Note 26) The change detection device according to Supplementary Note 25, wherein the first feature amount calculation means calculates the first feature amount using information indicating, in each of the first small regions, that a point is present, that a point is not present, or that the presence of a point is unknown.(Supplementary Note 27) The change detection device according to Supplementary Note 25 or 26, wherein the first feature amount calculation means further calculates feature amounts of each coordinate in a first neighborhood area of ​​the first coordinate in the first point cloud, in a third area consisting of a plurality of third small areas and including the coordinate, using information on the presence of points in each of the third small areas, and the determination means determines whether a point that does not exist at the first coordinate is present at the second coordinate using the feature amount of each coordinate in the first neighborhood area, the first feature amount, and the second feature amount. (Supplementary Note 28) The change detection device according to Supplementary Note 27, wherein the first point cloud data is data captured using a first sensor, and the first feature amount calculation means increases the size of the first neighborhood area as the distance between the position indicated by the first coordinate and the position of the first sensor at the time of acquiring the data increases. (Supplementary Note 29) The change detection device according to any one of Supplementary Notes 25 to 28, wherein the second feature amount calculation means further calculates feature amounts of each coordinate in a second neighborhood area of ​​the second coordinate in the second point cloud using information indicating that a point is present, that a point is not present, or that the presence of a point is unknown in each fourth small area, in a fourth area consisting of a plurality of fourth small areas and including the coordinate, and the determination means determines whether a point present at the first coordinate is not present at the second coordinate using the feature amount of each coordinate in the second neighborhood area, the first feature amount, and the second feature amount. (Supplementary Note 30) The change detection device according to any one of Supplementary Notes 25 to 29, further comprising an extraction unit that compares the first point cloud with the second point cloud and extracts coordinates where the presence or absence of a point differs between the first point cloud and the second point cloud, and at least one of the first coordinates or the second coordinates is coordinates extracted by the extraction unit.(Supplementary Note 31) The change detection device according to any one of Supplementary Notes 25 to 30, wherein the second point cloud data is data acquired by a second sensor, and for the second small region in which no point exists in the second point cloud, if the second small region is located between a position in the second point cloud where a point exists at the time of acquiring the data and the position of the second sensor, the second small region is defined as having no point, and if the second small region is not located between a position in the second point cloud where a point exists and the position of the second sensor, the existence of a point in the second small region is defined as being unknown. (Supplementary Note 32) The change detection device according to Supplementary Note 26, wherein the first point cloud data is data acquired by a first sensor, and for the first small region in which no point exists in the first point cloud, if the first small region is located between a position in the first point cloud where a point exists at the time of acquiring the data and the position of the first sensor, the first small region is defined as having no point, and if the first small region is not located between a position in the first point cloud where a point exists and the position of the first sensor, the presence of a point in the first small region is defined as being unknown. (Supplementary Note 33) The change detection device according to Supplementary Note 27 or 28, wherein the determination means determines whether a point that does not exist at the first coordinate exists at the second coordinate by calculating a similarity between the first feature and the second feature and, for each coordinate in the first neighboring region, a similarity between the feature of the coordinate and the second feature; the similarity between the first feature and the second feature is calculated using an element of the first feature in each of the first small region in the first region and a first small region included in a peripheral region of the first small region, and an element of the second small region in the second region corresponding to the first small region; and the similarity between the feature of each coordinate in the first neighboring region and the second feature is calculated using an element of the first feature in each of the third small region in the third region and a small region included in a peripheral region of the third small region, and an element of the second small region in the second region corresponding to the third small region.(Supplementary Note 34) The change detection device according to Supplementary Note 29, wherein the second point cloud data is data acquired using a second sensor, and the second feature amount calculation means increases the size of the second neighborhood area as the distance between the position indicated by the second coordinates and the position of the second sensor increases when the second point cloud data is acquired. (Supplementary Note 35) The change detection device according to Supplementary Note 29 or 34, wherein the determination means determines whether a point present at the first coordinate does not exist at the second coordinate by calculating a similarity between the first feature and the second feature and, for each coordinate in the second neighboring region, a similarity between the feature of the coordinate and the first feature; the similarity between the first feature and the second feature is calculated using an element of the second feature in each of the second small region in the second region and a second small region included in a peripheral region of the second small region, and an element of the first small region in the first region corresponding to the second small region; and the similarity between the feature of each coordinate in the second neighboring region and the first feature is calculated using an element of the fourth feature in each of the fourth small region in the fourth region and a small region included in a peripheral region of the fourth small region, and an element of the first small region in the first region corresponding to the fourth small region. (Supplementary Note 36) A non-transitory computer-readable medium having stored thereon a program that causes a computer to execute the following: calculate a first feature amount of a first coordinate in a first point cloud, in a first region consisting of a plurality of first small regions and including the first coordinate, using information regarding the presence of a point in each of the first small regions; calculate a second feature amount of a second coordinate in a second point cloud corresponding to the first coordinate, in a second region consisting of a plurality of second small regions and including the second coordinate, using information indicating that a point is present, that a point is not present, or that the presence of a point is unknown in each of the second small regions; and determine whether a change in the presence or absence of a point has occurred between the first coordinate and the second coordinate, using the first feature amount and the second feature amount.

[0195] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the disclosure.

[0196] REFERENCE SIGNS LIST 10 Change detection device 11 First feature amount calculation unit 12 Second feature amount calculation unit 13 Determination unit 14 Output unit 20 Change detection system 21 Feature amount calculation device 22 Determination device 100 Monitoring system 101 Robot 102 LiDAR 110 Base station 120 Center server 121 Reference point cloud acquisition unit 122 Input point cloud acquisition unit 123 Reference feature amount calculation unit 124 Input feature amount calculation unit 125 Change detection unit 126 Detection result generation unit 127 Extraction unit 128 Object identification unit 129 Movement control unit 130 Reference point cloud DB

Claims

1. calculating a first feature amount of a first coordinate in a first point cloud using information about the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; calculating a second feature amount of a second coordinate in a second point cloud corresponding to the first coordinate, using information indicating that a point exists, that a point does not exist, or that the existence of a point is unknown in a second region that is composed of a plurality of second small regions and includes the second coordinate; determining whether or not a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates by using the first feature amount and the second feature amount; A computer implemented method for change detection.

2. The first feature amount is calculated using information indicating that a point is present in each of the first small regions, that a point is not present, or that the presence of a point is unknown. The change detection method of claim 1 .

3. Further calculating a feature amount of each coordinate in a first neighborhood of the first coordinate in the first point cloud, in a third neighborhood including the coordinate, the third neighborhood being composed of a plurality of third small regions, using information regarding the presence of a point in each of the third small regions; determining whether or not a point that does not exist in the first coordinates is present in the second coordinates by using a feature amount of each coordinate in the first neighboring region, the first feature amount, and the second feature amount; The change detection method according to claim 1 or 2.

4. the first point cloud data is data acquired using a first sensor, the size of the first neighborhood area is increased as the distance between the position indicated by the first coordinates and the position of the first sensor increases when the first point cloud data is acquired; The change detection method of claim 3.

5. further calculating a feature amount of each coordinate in a second neighborhood area of ​​the second coordinate in the second point cloud using information indicating that a point exists, that a point does not exist, or that the existence of a point is unknown in each of the fourth small areas, the fourth small area being configured of a plurality of fourth small areas and including the coordinate; determining whether or not a point that exists at the first coordinates does not exist at the second coordinates by using a feature amount of each coordinate in the second neighboring region, the first feature amount, and the second feature amount; The change detection method according to claim 1 or 2.

6. detecting a change in the presence or absence of an object between the first cloud of points and the second cloud of points by performing the determination at a plurality of corresponding coordinates in the first cloud of points and the second cloud of points; outputting the result of said detection; The change detection method according to claim 1 or 2.

7. the second point cloud data is data acquired by a second sensor, Regarding the second small region in which there is no point in the second point cloud, if the second small region is located between a position where a point exists in the second point cloud at the time of acquiring the data and the position of the second sensor, it is defined that there is no point in the second small region, and if the second small region is not located between a position where a point exists in the second point cloud and the position of the second sensor, the presence of a point in the second small region is defined as unknown. The change detection method according to claim 1 or 2.

8. a first feature amount calculation means for calculating a first feature amount of a first coordinate in a first point cloud using information on the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; a second feature amount calculation means for calculating a second feature amount of a second coordinate in a second point group corresponding to the first coordinate, using information indicating that a point exists, that a point does not exist, or that the existence of a point is unknown in a second area including the second coordinate, the second area being composed of a plurality of second small areas; a determination means for determining whether or not a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates by using the first feature amount and the second feature amount; A change detection system comprising:

9. the first feature amount calculation means calculates the first feature amount using information indicating that a point is present in each of the first small regions, that a point is not present, or that the presence of a point is unknown; The change detection system of claim 8 .

10. a first feature amount calculation means for calculating a first feature amount of a first coordinate in a first point cloud using information on the presence of a point in each of the first small regions, the first region being composed of a plurality of first small regions and including the first coordinate; a second feature amount calculation means for calculating a second feature amount of a second coordinate in a second point group corresponding to the first coordinate, using information indicating that a point exists, that a point does not exist, or that the existence of a point is unknown in a second area including the second coordinate, the second area being composed of a plurality of second small areas; a determination means for determining whether or not a change in the presence or absence of a point has occurred between the first coordinates and the second coordinates by using the first feature amount and the second feature amount; A change detection device comprising:

Images