Point cloud processing apparatus, point cloud processing system, method for data processing, and program

Abstract

To provide a point cloud processing apparatus capable of indicating the shape of a moving object.SOLUTION: The point cloud processing apparatus according to the present disclosure includes: acquisition means for acquiring first point cloud data generated by measuring, over a predetermined period, a space including a moving object; estimation means for estimating velocity information of the moving object on the basis of second point cloud data constituting the moving object; and a correction unit for correcting the position of the second point cloud data by using the velocity information and difference information between the time at which the second point cloud data is generated and a reference time, the second point cloud data being included in the first point cloud data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a point cloud processing device, a point cloud processing system, a data processing method, and a program. [Background technology]

[0002] LiDAR (Light Detection and Ranging) is a well-known sensing device that creates 3D (Dimensional) maps. LiDAR uses light to measure the distance to surrounding objects. LiDAR generates 3D maps using point cloud data obtained as a result of the measurements.

[0003] Patent Document 1 discloses an example of the configuration of an optical distance measuring device that measures the distance to a measurement object. The optical distance measuring device disclosed in Patent Document 1 measures the distance to the measurement object using an optical coherent method. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 044534 Summary of the Invention [Problem to be solved by the invention]

[0005] Measurement targets for which distances are measured using an optical distance measuring device include moving objects. However, when using the optical distance measuring device disclosed in Patent Document 1, measurement of moving objects whose distance changes over time is not taken into consideration. As a result, there is a problem in that the shape indicated by the point cloud data of a moving object cannot accurately represent the shape of the moving object.

[0006] An object of the present disclosure is to provide a point cloud processing device, a point cloud processing system, a data processing method, and a program that can display the shape of a moving object. [Means for solving the problem]

[0007] The point cloud processing device according to the present disclosure includes an acquisition means for acquiring first point cloud data generated by measuring a space including a moving object for a predetermined period of time, an estimation means for estimating speed information of the moving object based on second point cloud data constituting the moving object, and a correction unit for correcting the position of the second point cloud data using the speed information and difference information between the time when the second point cloud data was generated and a reference time, the second point cloud data being included in the first point cloud data.

[0008] The point cloud processing system according to the present disclosure includes a point cloud generating device that generates first point cloud data by measuring a space including a moving object for a predetermined period of time, and a point cloud processing device that acquires the first point cloud data, estimates speed information of the moving object based on second point cloud data included in the first point cloud data, the second point cloud data constituting the moving object, and corrects the position of the second point cloud data using the speed information and difference information between the time when the second point cloud data was generated and a reference time.

[0009] The data processing method according to the present disclosure acquires first point cloud data generated by measuring a space including a moving object for a predetermined period of time, estimates velocity information of the moving object based on second point cloud data constituting the moving object, and corrects the position of the second point cloud data using the velocity information and difference information between the time when the second point cloud data was generated and a reference time, the second point cloud data being included in the first point cloud data.

[0010] The program according to the present disclosure causes a computer to acquire first point cloud data generated by measuring a space including a moving object for a predetermined period of time, estimate speed information of the moving object based on second point cloud data constituting the moving object, and correct the position of the second point cloud data using the speed information and difference information between the time when the second point cloud data was generated and a reference time, the second point cloud data being included in the first point cloud data. [Effects of the Invention]

[0011] The present disclosure makes it possible to provide a point cloud processing device, a data processing method, and a program that can display the shape of a moving object. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a point cloud processing device. [Figure 2] FIG. 2 shows the flow of the point cloud data correction process executed in the point cloud processing device. [Figure 3] FIG. 3 is a diagram illustrating an example of the configuration of a point cloud processing system. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of a point cloud generating device. [Figure 5] FIG. 5 is a diagram showing the power spectrum of the optical signal transmitted from the transmitting unit and the optical signal received at the receiving unit. [Figure 6] FIG. 6 is a diagram illustrating an example of the configuration of a point cloud processing device. [Figure 7] FIG. 7 shows point cloud data of a stationary object and point cloud data of a moving object. [Figure 8] FIG. 8 shows people and vehicles as moving objects. [Figure 9] FIG. 9 shows that vehicles and people are included in the predetermined area. [Figure 10] FIG. 10 shows point cloud data of a moving object. [Figure 11] FIG. 11 shows an example in which point cloud data of a moving object is generated using a plurality of point cloud generation devices. [Figure 12] FIG. 12 shows point cloud data containing point clouds measured from time t(nk) to time t(n). [Figure 13] FIG. 13 is a diagram showing how a moving object moves. [Figure 14] FIG. 14 is a graph showing the change in the moving speed shown in FIG. [Figure 15] FIG. 15 is a block diagram showing an example of the configuration of a point cloud processing device and the like. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Embodiment 1) The configuration and operation of the present disclosure will be described with reference to the drawings. Fig. 1 is a diagram showing an example configuration of a point cloud processing device 10. The point cloud processing device 10 may be a computer device that operates when a processor executes a program stored in a memory.

[0014] The point cloud processing device 10 has an acquisition unit 11, an estimation unit 12, and a correction unit 13. The acquisition unit 11, the estimation unit 12, and the correction unit 13 may be software or modules that are executed by a processor executing a program stored in a memory, or may be hardware such as a circuit or a chip.

[0015] The acquisition unit 11 may be used as a means for acquiring information, the estimation unit 12 may be used as a means for estimating information, and the correction unit 13 may be used as a means for correcting information.

[0016] The acquisition unit 11, the estimation unit 12, and the correction unit 13 may be provided in one point cloud processing device 10, or may be distributed and arranged in two or more computer devices. In other words, the point cloud processing device 10 may be a point cloud processing system configured by two or more computer devices.

[0017] The acquisition unit 11 acquires first point cloud data generated by measuring a space including moving objects for a predetermined period of time. The moving objects may include, for example, vehicles or pedestrians. The space including the moving objects may be, for example, a space that can be measured by a device that generates the first point cloud data. The device that generates the first point cloud data may be a distance measuring device using optical signals, or may be a sensor device. The first point cloud data has distance information from the device that generates the first point cloud data to objects existing in the space. The first point cloud data may be, for example, three-dimensional data that represents the positions of objects existing in the space using coordinates on predetermined coordinate axes. The first point cloud data is a set of points on the surface of the object. The first point cloud data indicates the shape of the object.

[0018] When the point cloud processing device 10 is equipped with a device that generates first point cloud data, the acquisition unit 11 acquires the first point cloud data without going through a network or the like. When the device that generates the first point cloud data is a device different from the point cloud processing device 10, the acquisition unit 11 acquires the first point cloud data from the device that generates the first point cloud data via a network or the like. Acquiring may be rephrased as receiving.

[0019] The estimation unit 12 estimates the speed information of the moving object based on second point cloud data constituting the moving object. The second point cloud data is point cloud data included in the first point cloud data. The second point cloud data is point cloud data indicating the position of the surface of the moving object. The second point cloud data represents the shape of the moving object.

[0020] The velocity information may be associated with each point constituting the second point cloud data. For example, the estimation unit 12 may estimate the average, maximum, or minimum value of the velocity associated with each point constituting the second point cloud data as the velocity information of the second point cloud data. Alternatively, the estimation unit 12 may estimate the velocity information of the second point cloud data using the amount of movement of the second point cloud data over a predetermined period.

[0021] The correction unit 13 corrects the position of the second point cloud data using velocity information of the second point cloud data and difference information between the time when the second point cloud data was generated and the reference time. Correcting the position of the second point cloud data may mean correcting the position of each point constituting the second point cloud data.

[0022] The second point cloud data includes multiple points measured at different times. Therefore, correcting the positions of the second point cloud data by the correction unit 13 may mean, for example, moving the positions of each point constituting the second point cloud data to positions that would be measured at substantially the same time by the correction unit 13. The substantially same time may have a predetermined time width when a predetermined time is added to or subtracted from a reference time.

[0023] Next, a data processing method executed in the point cloud processing device 10 will be described. FIG. 2 shows the flow of the point cloud data correction process executed in the point cloud processing device 10. First, the acquisition unit 11 acquires first point cloud data generated by measuring a space including a moving object for a predetermined period of time (S11). Next, the estimation unit 12 estimates speed information of the moving object based on second point cloud data constituting the moving object (S12). The second point cloud data is included in the first point cloud data. Next, the correction unit 13 corrects the position of the second point cloud data using the speed information and difference information between the time when the second point cloud data was generated and the reference time (S13).

[0024] As described above, the point cloud processing device 10 corrects the position of the second point cloud data constituting a moving object using the speed information of the moving object and the difference information between the time when the second point cloud data was generated and the reference time. This corrects the position of the second point cloud data measured for a predetermined period to the position at a specific time. As a result, the shape of the second point cloud data representing the moving object is clearly displayed.

[0025] (Embodiment 2) Next, an example of the configuration of a point cloud processing system will be described with reference to Fig. 3. The point cloud processing system of Fig. 3 includes a point cloud generation device 20 and a point cloud processing device 30. The point cloud generation device 20 may be, for example, a distance measuring device that measures the distance to a measurement object. The point cloud generation device 20 may also be, for example, an optical distance measuring device that measures the distance to a measurement object using an optical signal. The point cloud generation device 20 may also be, for example, a LiDAR device.

[0026] The point cloud generating device 20 and the point cloud processing device 30 may transmit or receive data via a network. The network may be, for example, an IP (Internet Protocol) network.

[0027] Next, we will explain the point cloud generation device 20. Fig. 4 shows an example configuration of the point cloud generation device 20. The point cloud generation device 20 may be a computer device that operates when a processor executes a program stored in a memory.

[0028] The point cloud generating device 20 has, as its components, a light source unit 21, a light modulation unit 22, a transmitter unit 23, a receiver unit 24, a coherent receiver unit 25, a time information assigner unit 26, a distance measurement unit 27, a speed calculation unit 28, and an output unit 29. The components constituting the point cloud generating device 20 may be software or modules that perform processing by a processor executing a program stored in a memory. Alternatively, the components constituting the point cloud generating device 20 may be hardware such as a circuit or a chip.

[0029] The light source unit 21 may be used as a means for generating a light source that outputs light. The light modulation unit 22 may be used as a means for modulating light. The transmission unit 23 may be used as a means for transmitting an optical signal. The reception unit 24 may be used as a means for receiving an optical signal. The coherent reception unit 25 may be used as a means for receiving an optical signal using a coherent system. The time information assignment unit 26 may be used as a means for assigning time information. The distance measurement unit 27 may be used as a means for measuring distance. The speed calculation unit 28 may be used as a means for calculating speed. The output unit 29 may be used as a means for outputting information.

[0030] The optical modulation unit 22 modulates the light output from the light source unit 21. The optical modulation unit 22 may generate an optical signal representing an optical pulse, for example, by modulating the phase of the light. The transmission unit 23 transmits the optical signal to the object 40. The optical signal transmitted to the object 40 may be referred to as emitted light.

[0031] The object 40 may be a stationary object that is stationary, or a moving object that is moving. Stationary objects may be buildings, road signs, traffic lights, etc. Moving objects may be vehicles, pedestrians, etc.

[0032] The receiving unit 24 receives the optical signal reflected by the object 40. The reflected optical signal may be referred to as reflected light. Receiving the optical signal may also mean receiving or receiving the reflected light.

[0033] The coherent receiving unit 25 extracts pulse information by causing interference between the optical signal, which is the reflected light, and the light output from the light source unit 21. The pulse information is pulse information used when the optical modulating unit 22 generates an optical signal indicating an optical pulse.

[0034] The time information assigning unit 26 assigns time information to the pulse information extracted by the coherent receiving unit 25. The time information may be, for example, the time when the time information assigning unit 26 receives the pulse information from the coherent receiving unit 25. Alternatively, the time information may be the time when the receiving unit 24 receives the reflected light. If the time information is the time when the reflected light is received by the receiving unit 24, the receiving unit 24 associates information indicating the time when the reflected light was received with the reflected light. The time when the reflected light was received may be associated with the reflected light as timestamp information set by the receiving unit 24, for example.

[0035] The distance measurement unit 27 measures the distance between the point cloud generation device 20 and the object 40. Specifically, the distance measurement unit 27 measures the distance between the point cloud generation device 20 and the position where the optical signal transmitted from the point cloud generation device 20 is reflected on the object 40. For example, the distance measurement unit 27 measures the distance to the object 40 using the time from when the transmitter 23 transmits the optical signal to when the receiver 24 receives the reflected light. Specifically, the time when the transmitter 23 transmits the optical signal is defined as t1, and the time when the receiver 24 receives the reflected light is defined as t2. In this case, the distance D between the point cloud generation device 20 and the object 40 is calculated as D = speed of light × (t2 - t1) / 2. " / " indicates division.

[0036] The velocity calculation unit 28 calculates the velocity of a point (reflection point) indicating the position where the optical signal transmitted from the point cloud generation device 20 is reflected on the object 40. The velocity calculation unit 28 calculates the velocity of the reflection point based on the amount of shift in the frequency of the optical signal transmitted from the transmission unit 23. The amount of shift in frequency may be, for example, a value indicating the difference between the frequency of the optical signal transmitted from the transmission unit 23 and the frequency of the reflected light received by the reception unit 24. The amount of shift in frequency may also be referred to as the amount of Doppler shift.

[0037] Here, the calculation of the velocity using the amount of frequency shift will be explained. First, the frequency fr of the reflected light is calculated by the following equation (1) using the relational expression of the Doppler effect.

[0038] TIFF2025178902000002.tif1596

[0039] Here, c: speed of light v: relative velocity between the point cloud generating device 20 and the object 40 ft: frequency of the optical signal transmitted from the transmitter 23 Let's say.

[0040] Next, the amount of Doppler shift fd is calculated using the following equation (2).

[0041] TIFF2025178902000003.tif1199

[0042] The relative velocity v is sufficiently smaller than the high velocity c. Therefore, the formula (2) may be approximated as the following formula (3).

[0043] TIFF2025178902000004.tif1375

[0044] As a result, the relative velocity v is calculated as shown in the following equation (4).

[0045] TIFF2025178902000005.tif1378

[0046] The frequencies ft and fr may be identified by using the power spectrum of the optical signal transmitted from the transmitter 23 and the optical signal received by the receiver 24. FIG. 5 is a diagram showing the power spectrum of the optical signal transmitted from the transmitter 23 and the optical signal received by the receiver 24. As shown in FIG. 5, the frequency ft of the signal obtained when the relative velocity between the point cloud generation device 20 and the object 40 is 0 shifts to the frequency fr when the relative velocity changes. A change in the relative velocity occurs, for example, when the point cloud generation device 20 is stationary and the object 40 is moving. Alternatively, a change in the relative velocity occurs when the velocity of the point cloud generation device 20 and the velocity of the object 40 differ. Alternatively, a change in the relative velocity occurs when the point cloud generation device 20 is moving and the object 40 is stationary.

[0047] The velocity calculation unit 28 may derive or calculate the power spectrum by performing FFT (Fast Fourier Transform) analysis on the optical signal transmitted from the transmission unit 23 and the reflected light received by the reception unit 24, for example.

[0048] The output unit 29 transmits the three-dimensional information, time information, and velocity information of each point that constitutes the point cloud data to the point cloud processing device 30.

[0049] Next, we will explain the point cloud processing device 30. Fig. 6 shows an example of the configuration of the point cloud processing device 30. The point cloud processing device 30 corresponds to the point cloud processing device 10. In the point cloud processing device 30, detailed explanations of the configuration, functions, etc. that are the same as those of the point cloud processing device 10 will be omitted.

[0050] The point cloud processing device 30 has an acquisition unit 31, an accumulation unit 32, an estimation unit 33, a classification unit 34, and a correction unit 35. The acquisition unit 31 corresponds to the acquisition unit 11 in the point cloud processing device 10. The estimation unit 33 corresponds to the estimation unit 12 in the point cloud processing device 10. The estimation unit 33 corresponds to the correction unit 13 in the point cloud processing device 10. The accumulation unit 32 may be used as a means for accumulating information. The classification unit 34 may be used as a means for classifying information.

[0051] The acquisition unit 31 acquires point cloud data from the point cloud generation device 20. Each point in the point cloud data is associated with time information and velocity information indicating the time when the point was generated. The time when each point was generated may be the time when the point cloud generation device 20 received reflected light, or the time when the point cloud generation device 20 performed coherent reception processing on the reflected light, etc.

[0052] The integrating unit 32 integrates point cloud data generated at different times. "Integrating" may be rephrased as combining, joining, superimposing, adding, or the like. By integrating point cloud data generated at different times, the shape of a stationary object is accurately displayed. "The shape is accurately displayed" may mean that the shape is displayed to an extent that it is recognizable. Alternatively, "the shape is accurately displayed" may mean that the boundaries between one object and another object are clearly defined.

[0053] On the other hand, when generating point cloud data of a moving object, the shape of the moving object is not clearly shown even if point cloud data generated at different times is added together.

[0054] Figure 7 shows point cloud data of a stationary object and point cloud data of a moving object. The stationary object is a person who is stationary, and the moving object is a person who is moving. The point cloud data of the stationary object accurately shows the shape of the stationary object by integrating point cloud data generated at different times.

[0055] On the other hand, the point cloud data of a moving object becomes point cloud data with an elongated shape in the moving direction of the moving object by accumulating point cloud data generated at different times. The point cloud data with an elongated shape may be a rectangle, a rectangle with rounded vertices, an ellipse, a shape with a curve, or the like. The shape of a person shown in the point cloud data of a moving object is shown to illustrate that a person is moving, and in reality, the shape of a person is not accurately shown in the elongated point cloud data.

[0056] The estimation unit 33 estimates the speed of a moving object. For example, the estimation unit 33 estimates the speed using point cloud data included in the elongated shape. Speed ​​information is associated with each point included in the elongated shape. When a moving object is moving at a constant speed, the speed information associated with each point will have substantially the same value. The substantially same value may vary within a predetermined range of values. The estimation unit 33 may estimate the speed information associated with an arbitrary point included in the elongated shape as the speed of the moving object. Alternatively, the estimation unit 33 may estimate the average, maximum, or minimum value of the speeds of an arbitrary number of points included in the elongated shape as the speed of the moving object. Alternatively, the estimation unit 33 may estimate the average, maximum, or minimum value of the speeds of all points included in the elongated shape as the speed of the moving object. Alternatively, the estimation unit 33 may estimate the speed associated with a point located at the center of gravity of the elongated shape as the speed of the moving object.

[0057] The classification unit 34 classifies objects by speed. In other words, the classification unit 34 classifies or identifies objects with different speeds as different objects. The classification unit 34 may also classify point cloud data with the same speed that exist at positions that are more than a predetermined distance apart as different objects. The classification unit 34 may also classify objects moving in different directions as different objects.

[0058] FIG. 8 shows a person and a vehicle as moving objects. The person is shown moving in the upward direction of FIG. 8, and the vehicle is shown moving in the downward direction of FIG. 8. The arrows indicate the direction of movement as well as the speed. For example, the speed increases as the length of the arrow decreases. In FIG. 8, the moving speed of the vehicle is shown to be faster than the moving speed of the person. The moving speeds of the two people are also assumed to be substantially the same.

[0059] When the relative speed between the point cloud generation device 20 and the vehicle and further the relative speed between the point cloud generation device 20 and the person change, the point cloud data generated by the point cloud generation device 20 does not accurately represent the shapes of the vehicle and the person, as shown in FIG. 9. FIG. 9 shows that the vehicle and the person are included in a predetermined area. In other words, FIG. 9 shows that the vehicle and the person have moved within the predetermined area. Although the vehicle and the person in FIG. 9 are shown using dotted lines, in reality, the shapes of the vehicle and the person cannot be identified in the point cloud data shown in FIG. 9. In FIG. 9, the vehicle and the person are shown using dotted lines to illustrate the area that includes the vehicle and the area that includes the person.

[0060] The classification unit 34 classifies the human region and the vehicle region as different objects because the human and the vehicle are moving at different speeds. Also, assume that two people are moving in the same direction at substantially the same speed, as shown in Fig. 8. In this case, if the regions representing people are separated by a predetermined distance or more, the classification unit 34 classifies the respective regions as different objects.

[0061] Furthermore, people who are moving often swing their arms while walking or running. In such cases, the relative speeds of the person's torso, legs, and arms are different from those of the point cloud generating device 20. In such cases, the classification unit 34 may further classify the point cloud data representing a person into parts of the person, such as the torso, legs, and arms.

[0062] The correction unit 35 corrects the position of the point cloud data included in the moving object. Correcting the position of the point cloud data may mean restoring the shape of the point cloud data representing the moving object. Here, correction of the position of the point cloud data will be explained using FIG. 10. FIG. 10 shows the point cloud data of the moving object. Specifically, FIG. 10 shows the time when the points constituting the point cloud data representing the moving object were generated.

[0063] The white arrows in the point cloud data of a moving object indicate the direction of movement of the moving object. In FIG. 10, time t(n) is the most recent time, and t(nk) is the oldest time. n and k are integers equal to or greater than 0. Circles connected with time t(nk), time t(n-2), time t(n-1), and time t(n) by dotted lines indicate points included in the point cloud data of the moving object. In other words, each point indicates a position in the point cloud data at the respective time.

[0064] The moving speed of the moving object is defined as speed V. Speed ​​V may be the relative speed between the point cloud generation device 20 and the moving object. Here, the modification unit 35 modifies the position of the point cloud data of the moving object to represent the shape of the moving object at time t(n). In this case, the modification unit 35 moves, for example, the position of the point generated at time t(nk) in the moving direction of the moving object by a distance indicated by speed V × {t(n) - t(nk)}. Furthermore, the modification unit 35 moves the position of the point generated at time t(n-2) in the moving direction of the moving object by a distance indicated by speed V × {t(n) - t(n-2)}. Furthermore, the modification unit 35 moves the position of the point generated at time t(n-1) in the moving direction of the moving object by a distance indicated by speed V × {t(n) - t(n-1)}. In this way, the modification unit 35 identifies the estimated position of the point generated before time t(n) if it had moved to time t(n). The correction unit 35 corrects the positions of all points associated with the moving speed V.

[0065] By accumulating the points whose positions have been corrected, the shape of the moving object is clearly shown, as shown below the arrow at time t(n) in FIG.

[0066] As described above, the point cloud processing device 30 corrects the position of each point cloud included in a point cloud representing a moving object using speed information and time information. As a result, the point cloud processing device 30 can restore the shape of a moving object that is stretched out and not accurately represented, and generate point cloud data that enables the shape of the moving object to be recognized.

[0067] (Embodiment 3) Next, calculation of the velocity of a moving object according to embodiment 3 will be described. Fig. 11 shows an example in which point cloud data of a moving object is generated using a plurality of point cloud generation devices 20. In Fig. 11, a point cloud generation device 20_1 and a point cloud generation device 20_2 are used as the plurality of point cloud generation devices 20, but three or more point cloud generation devices 20 may be used.

[0068] In velocity estimation using the Doppler shift amount, the point cloud generation device 20_1 and the point cloud generation device 20_2 can estimate velocity components in the line of sight direction, respectively. The line of sight direction may be rephrased as a measurement direction. That is, the line of sight direction may be a direction on a straight line connecting the point cloud generation device 20_1 and the object 40, and may also be a direction on a straight line connecting the point cloud generation device 20_2 and the object 40.

[0069] In FIG. 11, a point cloud generation device 20_1 estimates the velocity in the direction indicated by a velocity vector 1, and a point cloud generation device 20_2 estimates the velocity in the direction indicated by a velocity vector 2.

[0070] The estimation unit 33 may generate the velocity vector 1 and the velocity vector 2 using velocity information received from the point cloud generation device 20_1 and the point cloud generation device 20_2. For example, the estimation unit 33 is assumed to have information regarding the line of sight directions of the point cloud generation device 20_1 and the point cloud generation device 20_2 in advance. Alternatively, the estimation unit 33 may receive information regarding the line of sight directions from the point cloud generation device 20_1 and the point cloud generation device 20_2. The estimation unit 33 may generate the velocity vector 1 and the velocity vector 2 by combining information regarding the line of sight directions of the point cloud generation device 20_1 and the point cloud generation device 20_2 with velocity information in each direction.

[0071] Furthermore, the estimation unit 33 generates the combined vector shown in FIG. 8 by combining the velocity vector 1 and the velocity vector 2.

[0072] As described above, it is possible to accurately estimate the moving direction of the object 40 by combining the velocity information estimated by multiple point cloud generation devices 20. As a result, the accuracy of the shape of the object 40 indicated by the shape of the corrected point cloud data generated by the correction unit 35 is improved compared to when a single point cloud generation device 20 is used.

[0073] (Fourth embodiment) Next, calculation of the velocity of a moving object according to the fourth embodiment will be described. In the fourth embodiment, it is assumed that velocity information is not associated with each point constituting the point cloud data. That is, in the fourth embodiment, a case will be described in which the point cloud generating device 20 does not have a function for estimating the velocity of a moving object.

[0074] 12 shows point cloud data containing point clouds measured from time t(nk) to time t(n). The point cloud data containing point clouds measured from time t(nk) to time t(n) indicates a moving object. The white arrows shown in the point cloud data indicate the direction of movement of the moving object. The direction of movement may be, for example, the direction of a first principal component obtained by performing principal component analysis on each point included in the point cloud data.

[0075] 12 shows point cloud data containing point clouds measured between time t(nk) and Δt, and point cloud data containing point clouds measured during Δt up to time t(n). The estimation unit 33 may estimate the speed of the point cloud data based on the movement distance and the time required for the movement of the point cloud data.

[0076] For example, the estimation unit 33 may identify the center of gravity point in the point cloud data from time t(nk) to Δt and the center of gravity point in the point cloud data from time t(n) to Δt. The estimation unit 33 may estimate the speed of the center of gravity point, calculated based on the moving distance of the center of gravity point and the time required for the movement of the center of gravity point, as the speed of the moving object.

[0077] The distance traveled by the center of gravity is determined from the coordinates of the center of gravity. The time required for the center of gravity to move may be, for example, t(n) - t(nk) - Δt, or may be approximated as t(n) - t(nk) with Δt being a sufficiently small value.

[0078] Instead of using the center of gravity point, the estimation unit 33 may use any point in the point cloud data between time t(nk) and Δt and any point in the point cloud data between Δt and time t(n).

[0079] As explained above, even if the point cloud generating device 20 does not have the function of estimating the speed of each point, the speed of a moving object can be estimated using the distance traveled and the time required for the movement of points included in the point cloud data loaded within a specified time.

[0080] (Embodiment 5) Next, we will explain how to correct the position of point cloud data according to embodiment 5. Here, we will explain how to correct the position of point cloud data when a moving object moves not at a constant speed but at a changing speed when moving from time t(n-3) to time t(n) as shown in Fig. 13 .

[0081] It is assumed that the time intervals of the sections divided into t(n), t(n-1), t(n-2), and t(n-3) in Fig. 13 are the same. In Fig. 13, the travel distance in the section from t(n-2) to t(n-1) is longer than in the other sections, so the speed in the section from t(n-2) to t(n-1) is faster than in the other sections.

[0082] Figure 14 is a graph showing the change in the moving speed shown in Figure 13. In this situation, when the point cloud data generated at t(n-3), t(n-2), and t(n-1) is corrected to the position at t(n), the position is corrected according to the following equation (5).

[0083] TIFF2025178902000006.tif4128

[0084] Here, R(nk): Uncorrected coordinates of point cloud data acquired at time t(nk) R'(n): Corrected coordinates at time t(n) of point cloud data acquired at time t(nk) v(nm): The velocity of the object calculated at time t(nm) Let's say.

[0085] In this way, when correcting the point cloud data, the estimation unit 33 corrects the point cloud data taking into account the speed at which the data changes over time, thereby enabling the estimation unit 33 to more accurately estimate the position of the corrected point cloud data.

[0086] FIG. 15 is a block diagram showing an example configuration of a point cloud processing device 10, a point cloud generation device 20, and a point cloud processing device 30 (hereinafter referred to as the point cloud processing device 10, etc.). Referring to FIG. 15, the point cloud processing device 10, etc. includes a network interface 1201, a processor 1202, and a memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) conforming to the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0087] The processor 1202 reads and executes software (computer programs) from the memory 1203 to perform the processing of the point cloud processing device 10 described using the flowcharts. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

[0088] The memory 1203 is configured by a combination of volatile memory and non-volatile memory. The memory 1203 may include storage located remotely from the processor 1202. In this case, the processor 1202 may access the memory 1203 via an I / O (Input / Output) interface (not shown).

[0089] 15, the memory 1203 is used to store a group of software modules. The processor 1202 can perform processing of the point cloud processing device 10, etc. by reading and executing the group of software modules from the memory 1203.

[0090] As described with reference to FIG. 15, each of the processors included in the point cloud processing device 10 etc. executes one or more programs including a group of instructions for causing a computer to perform the algorithm described with reference to the drawings.

[0091] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0092] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0093] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0094] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) an acquisition means for acquiring first point cloud data generated by measuring a space including a moving object for a predetermined period of time; an estimation means for estimating velocity information of the moving object based on second point cloud data constituting the moving object, the second point cloud data being included in the first point cloud data; a correction unit that corrects a position of the second point cloud data by using the velocity information and difference information between the time when the second point cloud data was generated and a reference time. (Appendix 2) the first point cloud data includes distance information to an object identified based on an optical signal transmitted into the space and a reflected light when the optical signal is reflected by an object present in the space; and The estimation means 2. The point cloud processing device according to claim 1, wherein the velocity information is estimated based on frequency information of a first reflected light when the optical signal is reflected by the moving object. (Appendix 3) The estimation means 3. The point cloud processing device according to claim 2, wherein the velocity information is estimated based on frequency information of the optical signal transmitted into the space and frequency information of the first reflected light. (Appendix 4) The estimation means 4. The point cloud processing device according to claim 3, wherein the velocity information is estimated based on a frequency shift amount calculated from frequency information of the optical signal and frequency information of the first reflected optical signal. (Appendix 5) The estimation means 2. The point cloud processing device according to claim 1, wherein the velocity information is estimated based on a distribution of the second point cloud data in the predetermined period. (Appendix 6) The estimation means 6. The point cloud processing device according to any one of appendices 1 to 5, wherein the second point cloud data is associated with the velocity information. (Appendix 7) The acquisition means acquiring a plurality of first point cloud data generated by measuring the space at different positions; The estimation means 7. The point cloud processing device according to claim 1, wherein a plurality of pieces of velocity information of the moving objects estimated based on the respective first point cloud data are synthesized. (Appendix 8) The point cloud processing device according to any one of appendices 1 to 7, further comprising a classification means for classifying each of the moving objects based on the velocity information when a plurality of the moving objects exist in the space. (Appendix 9) The correction unit 9. The point cloud processing device according to claim 1, wherein the position of the second point cloud data at the time when the second point cloud data was generated is corrected to the position of the second point cloud data at the reference time. (Appendix 10) a point cloud generating device that generates first point cloud data by measuring a space including a moving object for a predetermined period of time; a point cloud processing device that acquires the first point cloud data, estimates speed information of the moving object based on second point cloud data included in the first point cloud data, the second point cloud data constituting the moving object, and corrects the position of the second point cloud data using the speed information and difference information between the time when the second point cloud data was generated and a reference time. (Appendix 11) Acquiring first point cloud data generated by measuring a space including a moving object for a predetermined period of time; estimating velocity information of the moving object based on second point cloud data constituting the moving object, the second point cloud data being included in the first point cloud data; A data processing method comprising: correcting a position of the second point cloud data using the velocity information and difference information between the time when the second point cloud data was generated and a reference time. (Appendix 12) Acquiring first point cloud data generated by measuring a space including a moving object for a predetermined period of time; estimating velocity information of the moving object based on second point cloud data constituting the moving object, the second point cloud data being included in the first point cloud data; A program that causes a computer to execute the following: correcting the position of the second point cloud data using the velocity information and difference information between the time when the second point cloud data was generated and a reference time.

[0095] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 9 that are dependent on Supplementary Notes 1 may also be dependent on Supplementary Notes 10 to 12 in the same dependency relationship as Supplementary Notes 2 to 9. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0096] 10 Point Cloud Processing Device 11 Acquisition Department 12 Estimation part 13 Correction section 20 point cloud generator 21 Light source section 22 Optical modulation section 23 Transmitter 24 Receiving unit 25 Coherent receiver 26 Time information adding section 27 Distance measurement unit 28 Speed ​​calculation section 29 Output section 30 Point Cloud Processing Device 31 Acquisition Department 32 Integration section 33 Estimation part 34 Classification Department 35 Correction section 40 Objects

Claims

1. an acquisition means for acquiring first point cloud data generated by measuring a space including a moving object for a predetermined period of time; an estimation means for estimating velocity information of the moving object based on second point cloud data constituting the moving object, the second point cloud data being included in the first point cloud data; a correction unit that corrects a position of the second point cloud data using the velocity information and difference information between the time when the second point cloud data was generated and a reference time.

2. the first point cloud data includes distance information to an object identified based on an optical signal transmitted into the space and a reflected light of the optical signal reflected by the object existing in the space; and The estimation means The point cloud processing device according to claim 1 , wherein the velocity information is estimated based on frequency information of a first reflected light when the optical signal is reflected by the moving object.

3. The estimation means The point cloud processing device according to claim 2 , wherein the velocity information is estimated based on frequency information of the optical signal transmitted into the space and frequency information of the first reflected light.

4. The estimation means The point cloud processing apparatus according to claim 3 , wherein the velocity information is estimated based on a frequency shift amount calculated from frequency information of the optical signal and frequency information of the first reflected light.

5. The estimation means The point cloud processing apparatus according to claim 1 , wherein the velocity information is estimated based on a distribution of the second point cloud data during the predetermined period.

6. The estimation means The point cloud processing device according to claim 1 , wherein the second point cloud data is associated with the velocity information.

7. The acquisition means acquiring a plurality of first point cloud data generated by measuring the space at different positions; The estimation means The point cloud processing apparatus according to claim 1 , wherein a plurality of pieces of the velocity information of the moving object estimated based on each of the first point cloud data are synthesized.

8. a point cloud generating device that generates first point cloud data by measuring a space including a moving object for a predetermined period of time; a point cloud processing device that acquires the first point cloud data, estimates velocity information of the moving object based on second point cloud data included in the first point cloud data, the second point cloud data constituting the moving object, and corrects the position of the second point cloud data using the velocity information and difference information between the time when the second point cloud data was generated and a reference time.

9. acquiring first point cloud data generated by measuring a space including a moving object for a predetermined period of time; estimating velocity information of the moving object based on second point cloud data constituting the moving object, the second point cloud data being included in the first point cloud data; a data processing method for correcting a position of the second point cloud data using the velocity information and difference information between the time when the second point cloud data was generated and a reference time.

10. acquiring first point cloud data generated by measuring a space including a moving object for a predetermined period of time; estimating velocity information of the moving object based on second point cloud data constituting the moving object, the second point cloud data being included in the first point cloud data; A program that causes a computer to execute the following: correcting the position of the second point cloud data using the velocity information and difference information between the time when the second point cloud data was generated and a reference time.

Images