Signal processing system and signal processing method

Abstract

To provide a signal processing system capable of improving accuracy of processing when integrating point group data, which are measured by a plurality of sensors, and a signal processing method.SOLUTION: A signal processing system comprises: an acquisition section for acquiring point group data, which are measured by a plurality of sensors, in a predetermined cycle; an object detection section for detecting an object for the unit of a sensor on the basis of the point group data; a synchronization processing section for estimating detection results of the object at future synchronous times coming at fixed time intervals based on a plurality of past time-sequential detection results of the object for the unit of a sensor by means of extrapolation and at time intervals; and an integration processing section for executing predetermined processing by integrating estimation results of the detection results of the object which are estimated on a sensor basis.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a signal processing system and a signal processing method. [Background technology]

[0002] 5 of Patent Document 1 describes a signal processing system including a first slave unit having a first laser sensor, a first point cloud acquisition device, and an object detection unit, a second slave unit having a second laser sensor, a second point cloud acquisition device, and an object detection unit, and a master unit that combines point cloud data and calculates a representative position based on the object detection results transmitted from the first and second slave units. Also, as described in paragraph 0030 of Patent Document 1, the signal processing system of Patent Document 1 combines point cloud data measured by two sensors by combining point clouds that are close in time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-121505 Summary of the Invention [Problem to be solved by the invention]

[0004] In the signal processing system described in Patent Document 1, point cloud data that are close in time are combined as described above, so there was a problem that depending on the magnitude of the difference in measurement timing between multiple sensors, the accuracy of processing when integrating point cloud data measured by multiple sensors may decrease.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a signal processing system and a signal processing method that can improve the accuracy of processing when integrating point cloud data measured by multiple sensors. [Means for solving the problem]

[0006] In order to solve the above problem, the signal processing system according to the present disclosure includes an acquisition unit that acquires point cloud data measured by a plurality of sensors at a predetermined cycle; an object detection unit that detects an object for each of the sensors based on the point cloud data; a synchronization processing unit that estimates the object detection results at future synchronization times that arrive at regular time intervals by extrapolation for each of the sensors based on past time series detection results of the objects; and an integration processing unit that integrates the estimated object detection results for each of the sensors and performs predetermined processing.

[0007] The signal processing method according to the present disclosure includes the steps of acquiring point cloud data measured by a plurality of sensors at a predetermined cycle, detecting an object for each of the sensors based on the point cloud data, estimating, for each of the sensors, the detection results of the object at a future synchronized time that arrives at a certain time interval by extrapolation based on the past time series detection results of the object, and integrating the estimated results of the object detection results estimated for each of the sensors to perform a predetermined process. [Effects of the Invention]

[0008] According to the signal processing system and signal processing method of the present disclosure, it is possible to improve the accuracy of processing when integrating point cloud data measured by a plurality of sensors. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example configuration of a signal processing system according to a first embodiment of the present disclosure. [Figure 2] 4 is a timing chart for explaining an example of operation of a synchronization processing unit according to the first embodiment of the present disclosure. [Figure 3] 5 is a flowchart illustrating an example of the operation of a data acquisition unit and an object detection unit according to the first embodiment of the present disclosure. [Figure 4] 10 is a flowchart showing an example of operation of a synchronization processing unit according to the first embodiment of the present disclosure. [Figure 5]10 is a flowchart showing an example of operation of an integration processing unit according to the first embodiment of the present disclosure. [Figure 6] FIG. 10 is a block diagram showing an example configuration of a signal processing system according to a second embodiment of the present disclosure. [Figure 7] FIG. 10 is a block diagram showing an example configuration of a signal processing system according to a third embodiment of the present disclosure. [Figure 8] FIG. 10 is a block diagram showing an example configuration of a signal processing system according to a fourth embodiment of the present disclosure. [Figure 9] FIG. 1 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, a signal processing system and a signal processing method according to an embodiment of the present disclosure will be described with reference to the drawings. Note that the same or corresponding components in each drawing are designated by the same reference numerals and descriptions thereof will be omitted as appropriate.

[0011] First Embodiment FIG. 1 is a block diagram illustrating an example configuration of a signal processing system 1 according to a first embodiment of the present disclosure. The signal processing system 1 illustrated in FIG. 1 uses multiple sensors SN-1 to SN-N installed on the roadside of a roadway 40 to measure, for example, the position and speed of a vehicle 50 traveling on the roadway 40. The signal processing system 1 illustrated in FIG. 1 includes two or more sensors SN-1 to SN-N, a signal processing device 2, an integrated processing device 3, a time synchronization server 4, and a signal line CL. Each of the sensors SN-1 to SN-N is, for example, a LiDAR (Light Detection and Ranging) sensor. It irradiates an object (a vehicle 50, which is an example of a moving object in this embodiment) located within a predetermined measurement range with light while changing the irradiation direction at a predetermined interval, and captures the light reflected from the object with an optical sensor to measure three-dimensional point cloud data. The point cloud data includes, for example, three-dimensional position coordinate information, the intensity of the reflected light at each point, and color information of each point captured by an optical camera. 1, for example, sensor SN-1 irradiates light from the upstream side to the downstream side and measures point cloud data etc. behind the vehicle 50 based on light reflected from the rear of the vehicle 50. Also, for example, sensor SN-2 irradiates light from the downstream side to the upstream side and measures point cloud data etc. ahead of the vehicle 50 based on light reflected from the front of the vehicle 50. Note that in the example shown in FIG. 1, the traveling direction of the vehicle 50 is indicated as the Y direction, the width direction of the roadway 40 as the X direction, and the upward direction as the Z direction.

[0012] The signal processing device 2 can be configured using a computer such as a personal computer, and includes a data acquisition unit 5 and an object detection unit 6 as functional blocks configured by a combination of hardware and software. The data acquisition unit 5 acquires point cloud data measured by multiple sensors SN-1 to SN-N at a predetermined cycle. The data acquisition unit 5 is an example configuration of an "acquisition unit" according to the present disclosure, and includes, for example, data acquisition units (1) 5-1 to (N) 5-N. The data acquisition unit 5 acquires point cloud data for each of sensors SN-1 to SN-N through parallel processing of the data acquisition units (1) 5-1 to (N) 5-N. In this case, for example, the data acquisition unit (1) 5-1 acquires time-series point cloud data measured by sensor SN-1 at a predetermined cycle and stores it in a predetermined memory provided in the signal processing device 2. Furthermore, for example, the data acquisition unit (2) 5-2 acquires time-series point cloud data measured by sensor SN-2 at a predetermined cycle and stores it in a predetermined memory provided in the signal processing device 2.

[0013] The object detection unit 6 detects objects for each sensor SN-1 to SN-N based on each point cloud data acquired by the data acquisition unit 5. There is no limitation on the method for detecting objects based on point cloud data, and a known method such as that described in Patent Document 1 can be used. The object detection unit 6 calculates and outputs, as an object detection result, one or more pieces of position information indicating the detected position of the object. The object detection unit 6 is an example configuration of the "object detection unit" according to the present disclosure and includes, for example, object detection units (1) 6-1 to object detection units (N) 6-N. The object detection unit 6 receives, through parallel processing by the object detection units (1) 6-1 to object detection units (N) 6-N, the point cloud data acquired by the data acquisition units (1) 5-1 to data acquisition units (N) 5-N at the acquisition period of each point cloud data, detects objects for each sensor SN-1 to SN-N, and outputs information indicating the object detection result to the integrated processing device 3, for example, at the acquisition period of each point cloud data. In this case, for example, the object detection unit (1) 6-1 detects an object based on the time-series point cloud data acquired by the data acquisition unit (1) 5-1 and stores the detection result in a predetermined memory provided in the signal processing device 2. Also, for example, the object detection unit (2) 6-2 detects an object based on the time-series point cloud data acquired by the data acquisition unit (2) 5-2 and stores the detection result in a predetermined memory provided in the signal processing device 2.

[0014] The integrated processing device 3 can be configured using a computer such as a personal computer, and includes a synchronization processing unit 7 and an integrated processing unit 8 as functional blocks configured by a combination of hardware and software. The synchronization processing unit 7 estimates object detection results at future synchronization times arriving at regular time intervals by extrapolating the results based on past time-series detection results of multiple objects by the object detection unit 6 for each of sensors SN-1 to SN-N. FIG. 2 is a timing chart for explaining an example of the operation of the synchronization processing unit 7 according to the first embodiment of the present disclosure. The horizontal axis represents time and the vertical axis represents the Y coordinate of the object, showing the change over time in the Y coordinate of the object. Each of times t1 to t6 is a synchronization time arriving at a regular time interval Ts.

[0015] Data Udv4, Udv5, Udv7, and Udv8 indicated by dashed circles are data calculated as detection results by the object detection unit (1) 6-1 based on the point cloud data measured by sensor SN-1, while data Ddv4, Ddv5, Ddv7, and Ddv8 indicated by dashed rectangles are data calculated as detection results by the object detection unit (2) 6-2 based on the point cloud data measured by sensor SN-2.

[0016] Furthermore, data Uev3 indicated by a solid circle is data estimated by extrapolation based on the two pieces of data Udv4 and Udv5 indicated by dashed circles as the object detection result based on the measurement results of sensor SN-1 at synchronization time t3. Furthermore, data Uev4 indicated by a solid circle is data estimated by extrapolation based on the two pieces of data Udv7 and Udv8 indicated by dashed circles as the object detection result based on the measurement results of sensor SN-1 at synchronization time t4.

[0017] Furthermore, data Dev3 shown by a solid-line rectangle is data estimated by extrapolation based on two pieces of data Ddv4 and Ddv5 shown by dashed-line rectangles as the object detection result based on the measurement results of sensor SN-2 at synchronization time t3. Furthermore, data Dev4 shown by a solid-line rectangle is data estimated by extrapolation based on two pieces of data Ddv7 and Ddv8 shown by dashed-line rectangles as the object detection result based on the measurement results of sensor SN-2 at synchronization time t4.

[0018] The synchronization processing unit 7 is a configuration example of a "synchronization processing unit" according to the present disclosure and includes synchronization processing units (1) 7-1 to (N) 7-N. The synchronization processing unit 7 estimates object detection results for each of sensors SN-1 to SN-N at time intervals Ts for each of synchronization times t1 to t6 by extrapolation through parallel processing by synchronization processing units (1) 7-1 to (N) 7-N. For example, the synchronization processing unit (1) 7-1 extrapolates object detection results (data Uev3 and Uev4) at future synchronization times t3 and t4, which arrive at a fixed time interval Ts, at the fixed time interval Ts based on past time-series object detection results (data Udv4, Udv5, Udv7, Udv8, etc.) by the object detection unit (1) 6-1. Furthermore, for example, the synchronization processing unit (2) 7-2 extrapolates the object detection results (data Dev3 and Dev4) at future synchronization times t3 and t4, which arrive at a fixed time interval Ts, based on the past time series detection results of multiple objects (data Ddv4, Ddv5, Ddv7, Ddv8, etc.) by the object detection unit (2) 6-2 at the fixed time interval Ts.

[0019] In the example shown in FIG. 2, one future data item is estimated based on two past time-series data items, but future data items may be estimated based on three or more past time-series data items.

[0020] The integration processing unit 8 integrates the estimation results of the object detection results estimated for each of the sensors SN-1 to SN-N and performs a predetermined process. Here, the predetermined process includes, for example, a process of calculating the position and speed of the vehicle 50, which is a moving body. The predetermined process also includes, for example, a process of calculating the length and width of the vehicle 50. The predetermined process may also be, for example, a process of providing information about the vehicle 50 traveling on the roadway 40 via wireless communication to a vehicle 50 attempting to merge onto the roadway 40, based on the calculated data. For example, in the example shown in FIG. 2, the object detection result based on the point cloud data measured by the sensor SN-2 and the object detection result based on the point cloud data measured by the sensor SN-1 are used as detection results corresponding to the front and rear ends of the vehicle 50. Based on the estimation results of the object detection results at each of the synchronization times t1 to t6, the integration processing unit 8 can estimate the vehicle length (the distance between the front and rear ends) with higher accuracy than when synchronization processing is not performed. In addition, integrating the estimation results estimated for each sensor SN-1 to SN-N means combining and using a single estimation result (for multiple sensors) at the synchronized time of object detection results based on point cloud data measured by multiple sensors.

[0021] In addition, the time synchronization server 4 uses the signal line CL to supply a signal that serves as a reference when synchronizing operations to the signal processing device 2 and the integrated processing device 3. The signal supplied by the time synchronization server 4 can be used as a timing reference for each of the synchronization times t1 to t6 shown in FIG.

[0022] FIG. 3 is a flowchart showing an example of the operation of the data acquisition unit 5 and the object detection unit 6 according to the first embodiment of the present disclosure. The process shown in FIG. 3 is repeatedly executed at the sampling period of the sensors SN-1 to SN-N. In the process shown in FIG. 3, the data acquisition unit 5 acquires each point cloud data measured by the sensors SN-1 to SN-N (step S11). Next, the data acquisition unit 5 stores each acquired point cloud data in a predetermined memory (step S12). Next, the object detection unit 6 detects an object based on each stored point cloud data (step S13). Next, The object detection unit 6 stores the detection result of the detected object in a predetermined memory (step S14), and the process shown in FIG. 3 ends.

[0023] 4 is a flowchart showing an example of the operation of the synchronization processing unit 7 according to the first embodiment of the present disclosure. The process shown in FIG. 4 is repeatedly executed at a synchronization interval of the detection result (the synchronization interval is the time interval Ts in the example shown in FIG. 2). In the process shown in FIG. 4, the synchronization processing unit 7 estimates the detection result at the next synchronization time by extrapolation based on the past object detection result (step S21). Next, the synchronization processing unit 7 stores the estimated detection result in a predetermined memory (step S22), and ends the process shown in FIG. 4.

[0024] Fig. 5 is a flowchart showing an example of the operation of the integration processing unit 8 according to the first embodiment of the present disclosure. The processing shown in Fig. 5 is repeatedly executed at an execution cycle of the integration processing (for example, the time interval Ts shown in Fig. 2 (however, not limited to this time)). In the processing shown in Fig. 5, the integration processing unit 8 executes predetermined processing by integrating each estimated value at each synchronization time of each detection result of an object based on each measurement value of the point cloud data of each sensor SN-1 to SN-N (step S31), and then ends the processing shown in Fig. 5.

[0025] As described above, the signal processing system 1 of this embodiment includes a data acquisition unit 5 that acquires point cloud data measured by multiple sensors SN-1 to SN-N at a predetermined cycle, an object detection unit 6 that detects objects for each sensor based on each point cloud data, a synchronization processing unit 7 that estimates object detection results at future synchronization times arriving at regular time intervals Ts by extrapolation for each sensor based on past time-series object detection results, and an integration processing unit 8 that integrates the object detection results estimated for each sensor and performs predetermined processing. This configuration can minimize the impact of differences in measurement timing between the multiple sensors, thereby improving the accuracy of processing when integrating point cloud data measured by the multiple sensors.

[0026] In addition, in this embodiment, the object to be detected is a moving body such as a vehicle 50, and the predetermined processing performed by the integrated processing unit 8 can include processing to calculate the position and speed of the moving body such as a vehicle 50.

[0027] In this embodiment, the data acquisition unit 5 acquires point cloud data for each sensor through parallel processing, the object detection unit 6 detects objects for each sensor through parallel processing, and the synchronization processing unit 7 estimates the object detection results for each sensor through parallel processing at time intervals Ts by extrapolation. With this configuration, even when there are a large number of sensors, it is easy to synchronize the acquisition of point cloud data, object detection, and synchronization processing between sensors.

[0028] Second Embodiment Fig. 6 is a block diagram showing an example configuration of a signal processing system 1a according to a second embodiment of the present disclosure. In the signal processing system 1a of the second embodiment, the signal processing device 2 and the integrated processing device 3 shown in Fig. 1 are configured as a single processing device 2a (computer). In the second embodiment, it is possible to omit the external signal exchange between the signal processing device 2 and the integrated processing device 3 that was necessary in the first embodiment.

[0029] <Third embodiment> 7 is a block diagram showing a configuration example of a signal processing system 1b according to a third embodiment of the present disclosure. In the signal processing system 1b of the third embodiment, the signal processing device 2 shown in FIG. 1 is configured using a plurality of signal processing devices (1) 2-1 to (N) 2-N. In the third embodiment, the load of the signal processing device 2 of the first embodiment can be distributed to the plurality of signal processing devices (1) 2-1 to (N) 2-N.

[0030] <Fourth embodiment> FIG. 8 is a block diagram showing a configuration example of a signal processing system 1c according to the fourth embodiment of the present disclosure. In the signal processing system 1c of the fourth embodiment, the signal processing device 2 shown in FIG. 1 is configured using a plurality of signal processing devices (1) 2c-1 to signal processing devices (K) 2c-K (where K < N). For example, the signal processing device (1) 2c-1 includes a data acquisition unit 5c-1 and an object detection unit 6c-1. The data acquisition unit 5c-1 includes a data acquisition unit (1) 5-1, a data acquisition unit (2) 5-2, and a data acquisition unit (3) 5-3. The object detection unit 6c-1 includes an object detection unit (1) 6-1, an object detection unit (2) 6-2, and an object detection unit (3) 6-3. Further, the signal processing device (K) 2c-K includes a data acquisition unit 5-N and an object detection unit 6-N. In the fourth embodiment, the load of the signal processing device 2 of the first embodiment can be distributed to a plurality of signal processing devices (1) 2c-1 to signal processing devices (K) 2c-K. Also, compared with the third embodiment, the number of a plurality of signal processing devices (1) 2c-1 to signal processing devices (K) 2c-K can be reduced.

[0031] (Function and effect) In the signal processing system and the signal processing method having the above configuration, each point cloud data measured by a plurality of sensors SN-1 to SN-N is acquired at a predetermined cycle, an object is detected for each sensor based on each point cloud data, and based on the detection results of a plurality of objects in the past time series for each sensor, the detection result of an object at a future synchronization time arriving at a fixed time interval Ts is estimated at the time interval Ts by extrapolation. Further, each estimation result of the detection result of the object estimated for each sensor is integrated to execute a predetermined process. According to this configuration, the influence due to the difference in measurement timings among a plurality of sensors can be suppressed to a small level, so that the accuracy of the process when integrating the point cloud data measured by a plurality of sensors can be improved.

[0032] (Other embodiments) As described above, the embodiments of the present disclosure have been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and design changes and the like within the scope not departing from the gist of the present disclosure are also included.

[0033] 〈Computer configuration〉 FIG. 9 is a schematic block diagram illustrating the configuration of a computer according to at least one embodiment. The computer 90 includes a processor 91 , a main memory 92 , a storage 93 , and an interface 94 . The above-mentioned signal processing devices 2, 2-1 to 2-N, 2c-1 to 2c-K, integrated processing device 3, processing device 2a, etc. are implemented in a computer 90. The operations of the above-mentioned processing units are stored in the form of a program in a storage 93. A processor 91 reads the program from the storage 93, loads it into a main memory 92, and executes the above-mentioned processing in accordance with the program. The processor 91 also allocates storage areas in the main memory 92 corresponding to the above-mentioned storage units in accordance with the program.

[0034] The program may be for realizing some of the functions to be performed by the computer 90. For example, the program may be combined with other programs already stored in storage or other programs implemented in other devices to perform the functions. In other embodiments, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to or instead of the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), and FPGA (Field Programmable Gate Array). In this case, some or all of the functions realized by the processor may be realized by the integrated circuit.

[0035] Examples of storage 93 include a hard disk drive (HDD), a solid state drive (SSD), a magnetic disk, a magneto-optical disk, a compact disc read-only memory (CD-ROM), a digital versatile disc read-only memory (DVD-ROM), and a semiconductor memory. Storage 93 may be an internal medium directly connected to the bus of computer 90, or an external medium connected to computer 90 via interface 94 or a communication line. Furthermore, when this program is distributed to computer 90 via a communication line, computer 90 that receives the program may load the program into main memory 92 and execute the above-described processing. In at least one embodiment, storage 93 is a non-transitory tangible storage medium.

[0036] <Additional Notes> The signal processing system described in each embodiment can be understood, for example, as follows.

[0037] (1) A signal processing system according to a first aspect includes an acquisition unit that acquires point cloud data measured by a plurality of sensors at a predetermined cycle, an object detection unit that detects an object for each of the sensors based on the point cloud data, a synchronization processing unit that estimates, for each of the sensors, detection results of the object at future synchronization times that arrive at regular time intervals by extrapolation based on past time-series detection results of the object, and an integration processing unit that integrates the estimated object detection results for each of the sensors and performs predetermined processing. According to this aspect and each of the following aspects, it is possible to improve the accuracy of processing when integrating point cloud data measured by a plurality of sensors.

[0038] (2) A signal processing system according to a second aspect is the signal processing system of (1), in which the object is a moving object, and the predetermined processing includes processing for calculating the position and velocity of the moving object.

[0039] (3) A signal processing system according to a second aspect is a signal processing system according to (1) or (2), wherein the acquisition unit acquires each point cloud data for each sensor by parallel processing, the object detection unit detects the object for each sensor by parallel processing, and the synchronization processing unit estimates the detection result of the object for each sensor by extrapolation at the time interval by parallel processing. [Explanation of symbols]

[0040] 1...Signal processing system 2...Signal processing device 3...Integrated processing equipment 5...Data acquisition section 6...Object detection unit 7...Synchronization processing section 8...Integrated processing section SN-1 to SN-N...Sensors

Claims

1. an acquisition unit that acquires point cloud data measured by a plurality of sensors at a predetermined cycle; an object detection unit that detects an object for each of the sensors based on the point cloud data; a synchronization processing unit that estimates, for each sensor, a detection result of the object at a future synchronization time that arrives at a certain time interval by extrapolation based on a plurality of detection results of the object in a past time series; an integration processing unit that integrates the estimation results of the object detection results estimated for each of the sensors and executes a predetermined process; A signal processing system comprising:

2. the object is a moving object, The predetermined process includes a process for calculating the position and velocity of the moving object.

2. The signal processing system of claim 1.

3. the acquisition unit acquires the point cloud data for each of the sensors by parallel processing; the object detection unit detects the object on a sensor-by-sensor basis by parallel processing; The synchronization processing unit estimates the detection result of the object for each sensor at the time interval by extrapolation through parallel processing.

3. A signal processing system according to claim 1 or 2.

4. acquiring point cloud data measured by a plurality of sensors at a predetermined interval; detecting an object for each of the sensors based on the point cloud data; a step of extrapolating, for each of the sensors, a detection result of the object at a future synchronized time that arrives at a certain time interval based on a plurality of past time-series detection results of the object at the time intervals; a step of integrating the estimation results of the object detection results estimated for each of the sensors and executing a predetermined process; A signal processing method comprising:

Images