Object detection system, CSI measurement device, object detection method, and CSI measurement program

Abstract

An object detection system according to an embodiment of the present invention is characterized by comprising: a capture unit that acquires CSI by capturing radio waves between a base station and a terminal; a phase difference calculation unit that calculates a phase difference between predetermined resource elements with respect to the CSI acquired by the capture unit; a variance calculation unit that calculates the variance or the moving variance of the phase difference calculated by the phase difference calculation unit; and a detection processing unit that detects a person or an object on the basis of the variance or the moving variance of the phase difference calculated by the variance calculation unit.

Description

Object detection system, CSI measurement device, object detection method, and CSI measurement program

[0001] The present invention relates to an object detection system, a CSI measurement device, an object detection method, and a CSI measurement program that enable detection and movement detection of people and objects.

[0002] A cellular base station (base station) transmits broadcast signals, downlink control signals (PDSCH), downlink data signals (PDCCH), downlink reference signals (LTE: Cell-specific RS, 5G: CSI-RS, DM-RS, TRS), etc. (See, for example, Non-Patent Document 1). Furthermore, a user equipment (UE) transmits uplink control signals (PUCCH), uplink data signals (PUSCH), etc. Furthermore, LTE / 5G reference signals include RSSI, CSI, etc.

[0003] In wireless LANs, wireless sensing techniques that utilize received power such as RSSI and channel state information (CSI) are being studied (see, for example, Non-Patent Document 2).

[0004] In addition, by using LTE / 5G OSS (open source software) such as Open Air Interface, it is possible to emulate the operation of the LTE / 5G protocol stack, making it possible to capture radio waves transmitted by base stations using OSS and calculate CSI without using specialized equipment.

[0005] Kazuki Takeda and five others, "NR Physical Layer Specifications for 5G," NTT DOCOMO Technical Journal, Vol. 26, No. 3 (Nov. 2018). Tomoki Murakami and four others, "Bird and Animal Detection System Using Wireless LAN Signals," NTT Technical Journal, April 2019.

[0006] However, in the past, even if the transmitted and received radio waves between a base station and a UE were captured and CSI was obtained, the phase and amplitude information contained in the CSI contained noise, and the values ​​fluctuated significantly even with slight changes in the environment, so the obtained CSI data could not be used as is to detect people and objects or to detect movement.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an object detection system, a CSI measurement device, an object detection method, and a CSI measurement program that enable accurate detection of people and objects.

[0008] An object detection system according to one embodiment of the present invention is characterized by having a capture unit that acquires CSI by capturing radio waves between a base station and a terminal, a phase difference calculation unit that calculates the phase difference between predetermined resource elements based on the CSI acquired by the capture unit, a variance calculation unit that calculates the variance or moving variance of the phase difference calculated by the phase difference calculation unit, and a detection processing unit that detects people or objects based on the variance or moving variance of the phase difference calculated by the variance calculation unit.

[0009] Furthermore, a CSI measurement device according to one embodiment of the present invention is characterized by having a capture unit that acquires CSI by capturing radio waves between a base station and a terminal, a phase difference calculation unit that calculates a phase difference between predetermined resource elements based on the CSI acquired by the capture unit, a variance calculation unit that calculates a variance or moving variance of the phase difference calculated by the phase difference calculation unit, and an output processing unit that performs predetermined processing on the variance or moving variance of the phase difference calculated by the variance calculation unit and outputs the variance or moving variance to an external device.

[0010] In addition, an object detection method according to one embodiment of the present invention is characterized by including a capture process for acquiring CSI by capturing radio waves between a base station and a terminal, a phase difference calculation process for calculating a phase difference between predetermined resource elements based on the CSI acquired by the capture process, a variance calculation process for calculating a variance or moving variance of the phase difference calculated by the phase difference calculation process, a sensing process for detecting data related to the surrounding environment using a sensor, and a detection processing process for detecting a person or object based on the variance or moving variance of the phase difference calculated by the variance calculation process and the data detected by the sensing process.

[0011] Furthermore, a CSI measurement program according to one embodiment of the present invention is a CSI measurement program for causing a computer to function as each part of a CSI measurement device having a capture unit that acquires CSI by capturing radio waves between a base station and a terminal, a phase difference calculation unit that calculates a phase difference between predetermined resource elements based on the CSI acquired by the capture unit, a variance calculation unit that calculates the variance or moving variance of the phase difference calculated by the phase difference calculation unit, and an output processing unit that performs predetermined processing on the variance or moving variance of the phase difference calculated by the variance calculation unit and outputs it to an external device.

[0012] According to the present invention, it is possible to detect people and objects with high accuracy.

[0013] 1 is a diagram illustrating an environment in which an object detection system according to an embodiment is used; FIG. 2 is a functional block diagram illustrating functions of the object detection system; FIG. 3 is a diagram illustrating resource blocks used by a phase difference calculation unit; FIG. 4 is a graph illustrating results of calculation of the variance of phase difference over time for each frequency by a variance calculation unit; FIG. 5 is a graph illustrating results of calculation of the variance of phase difference over frequency for each predetermined time by a variance calculation unit; FIG. 6 is a diagram illustrating an example hardware configuration of a CSI measurement device according to an embodiment; FIG. 7 is a diagram illustrating an environment in which an object detection system is used around multiple base stations; FIG. 8 is a diagram illustrating an overview of an environment in which multiple object detection systems detect users and objects; FIG. 9 is a diagram illustrating an example system configuration in which multiple object detection systems detect users and objects; FIG. 10 is a diagram illustrating an example system configuration in which multiple object detection systems are connected via a network; FIG. 11 is a diagram illustrating an environment in which a first modified example of an object detection system is used; FIG. 12 is a diagram illustrating a second modified example of an object detection system; and FIG. 13 is a diagram illustrating a modified example of an object detection system.

[0014] An object detection system 10 according to an embodiment will be described below with reference to the drawings. FIG. 1 is a diagram illustrating an example environment in which the object detection system 10 according to an embodiment is used. The object detection system 10 detects a user (person) 3 and an object (object) 4 located around a base station 1 and a UE (user equipment: terminal) 2 that wirelessly communicate with each other. The object 4 is assumed to be a moving object such as an AVG (Automatic Guided Vehicle) or an AMR (Autonomous Mobile Robot).

[0015] The object detection system 10 includes, for example, a CSI measurement device 20 and an information processing device (external terminal) 30, and has the function of receiving radio waves transmitted by the base station 1 and the UE 2 and arriving after being reflected, diffracted, transmitted, etc., and capturing the CSI.

[0016] The object detection system 10 then uses the captured CSI to detect the user 3, the object 4, etc. The object detection system 10 will be described in more detail below.

[0017] 2 is a functional block diagram illustrating functions of the object detection system 10. As shown in FIG. 2, the object detection system 10 has a configuration in which an information processing device 30 is connected to a CSI measurement device 20, for example.

[0018] The CSI measuring device 20 includes, for example, a CSI measuring unit 22 and a CSI evaluating unit 24, captures radio waves from the base station 1, and estimates the amplitude and phase of the CSI. Then, the CSI measuring device 20 outputs the estimation result to the information processing device 30.

[0019] For example, the CSI measurement unit 22 includes a capture unit 220. The capture unit 220 acquires CSI by capturing radio waves between the base station 1 and the UE 2, and outputs the CSI to the CSI evaluation unit 24.

[0020] The CSI measurement unit 22 may measure the RSSI and the RSRP simultaneously. Also, a plurality of CSI measurement units 22 may be provided in the CSI measurement device 20.

[0021] The CSI evaluation unit 24 has a phase difference calculation unit 240, a variance calculation unit 242, and an output processing unit 244, and calculates the moving variance of the CSI phase difference based on the information output by the CSI measurement unit 22, and outputs the calculated results to the information processing device 30.

[0022] For example, the phase difference calculation unit 240 calculates a phase difference between predetermined resource elements for the CSI acquired by the capture unit 220, and outputs the calculation result to the variance calculation unit 242. For example, the phase difference calculation unit 240 calculates a phase difference (CSI phase difference) between resource elements at different frequencies at the same time, or at different times at the same frequency, for the CSI acquired by the capture unit 220.

[0023] 3 is a diagram illustrating resource blocks used by the phase difference calculation unit 240. When the wireless communication between the base station 1 and the UE 2 is LTE, the phase difference calculation unit 240 may focus on a specific resource block and calculate the phase difference of the CSI of resource elements between cell-specific RSs in adjacent frequency domains. For example, in FIG. 3, the phase difference calculation unit 240 may calculate the phase difference of the CSI of two resource elements marked with dashed circles.

[0024] Alternatively, the phase difference calculation unit 240 may calculate the phase difference between adjacent CSI-RSs for all resource blocks, and then calculate the average or median. Alternatively, the phase difference calculation unit 240 may calculate the phase difference between CSI-RSs that are a specific number of CSI-RSs apart, or the phase difference between CSI-RSs in the frequency domain that are the most distant.

[0025] Furthermore, the phase difference calculation unit 240 may calculate the phase difference of the CSI of resource elements between adjacent cell-specific RSs in the time domain. For example, in Fig. 3, the phase difference calculation unit 240 may calculate the phase difference of the CSI of two resource elements marked with solid circles.

[0026] Furthermore, the phase difference calculation unit 240 may calculate the phase difference of the CSI for resource blocks spaced apart by a specified value. Also, the phase difference calculation unit 240 may calculate the phase difference of the CSI between CRSs of antenna ports of adjacent antennas, or may calculate the phase difference of the CSI between CSI-RSs of the antenna port of the farthest antenna.

[0027] Furthermore, even when the wireless communication between the base station 1 and the UE 2 is 5G NR, the phase difference calculation unit 240 may focus on a specific resource block as described above and calculate the phase difference between the CSI of CSI-RSs that are adjacent in the frequency domain or time domain. Furthermore, the phase difference calculation unit 240 may calculate the phase difference between the CSI of resource elements that are most distant in the frequency domain (or time domain) or that are separated by a predetermined value. Alternatively, the phase difference calculation unit 240 may refer to another reference signal that includes the CSI.

[0028] The variance calculation unit 242 (FIG. 2) calculates the variance (or moving variance) of the phase difference calculated by the phase difference calculation unit 240 and outputs the calculation result to the output processing unit 244 .

[0029] Fig. 4 is a graph illustrating the results of calculating the variance (moving variance) of phase difference over time for each frequency (Resource Block; RB) by the variance calculation unit 242. Fig. 5 is a graph illustrating the results of calculating the variance of phase difference for each frequency (Resource Block; RB) for each predetermined time (for example, every 5 minutes) by the variance calculation unit 242.

[0030] The output processing unit 244 performs a predetermined process on the variance of the phase difference (or the moving variance) calculated by the variance calculation unit 242, and outputs the result to the information processing device 30. For example, the output processing unit 244 performs a process of converting the variance of the phase difference into a numerical format such as a CSV file or into a graph format, and outputs the processing result to the information processing device 30.

[0031] For example, the output processing unit 244 may output a CSV file or graph, etc., targeting a specific subcarrier or multiple subcarriers, with time on the X axis and the following values ​​on the Y axis: 1) CSI amplitude or amplitude variance (or standard deviation) 2) CSI phase, phase difference, phase difference variance (or standard deviation) Or, calculated from 1) and 2): a) average b) moving average (variance or standard deviation) c) exponential moving average (variance or standard deviation) d) weighted moving average (variance or standard deviation)

[0032] The output processing unit 244 may output the data at a constant time interval, or may change the time interval depending on fluctuations in the calculated average value or variance. For example, the output processing unit 244 may output the data at a shorter time interval as the increase or decrease in the variance value increases.

[0033] The output processing unit 244 may also perform a Fourier transform or calculate a power spectrum using data such as the amplitude, moving average, and moving variance of the CSI, and output the peak value. The output processing unit 244 may also perform a wavelet transform or a short-term Fourier transform, and output a graph or data file of the transformation results. The output processing unit 244 may also output CSI data that has been smoothed using a Kalman filter, a particle filter, or the like.

[0034] In addition, the CSI measurement device 20 can evaluate and output, for example, by using the results shown in Figures 4 and 5, that N users 3 were present in the area being sensed over a predetermined period of time.

[0035] In addition, the CSI measuring device 20 may associate the variance of the phase difference calculated by the variance calculation unit 242 with the number of times users 3 or objects 4 travel, their speed, their direction, distance, and the density and density changes of users 3 or objects 4 in the target area, and output only the result of the association.

[0036] For example, if the object 4 to be sensed moves periodically and the variance of the phase difference of the CSI takes a periodic value, the CSI measuring device 20 may associate the number of repetitions and direction of the movement of the object 4 with the variance of the phase difference calculated by the variance calculation unit 242, and output only the result.

[0037] Furthermore, the CSI measuring device 20 may estimate the congestion rate of the user 3 or the object 4 from the degree of variance of the amplitude or phase difference of the CSI. For example, the CSI measuring device 20 may calculate the variance of the phase difference of the CSI at regular time intervals and estimate the congestion rate on the condition that the variance value falls within a predetermined interval.

[0038] At this time, the CSI measuring device 20 may estimate the congestion rate using machine learning (deep learning), etc. For example, the CSI measuring device 20 may learn the variance and the congestion rate in advance using training data, estimate the congestion rate based on the data acquired by the capture unit 220, and output only the estimation result.

[0039] The information processing device 30 is a computer equipped with an analysis unit 32 that analyzes the CSI. The analysis unit 32 includes, for example, a detection processing unit 320. The detection processing unit 320 performs processing to detect a person or an object based on the variance (or movement variance) of the phase difference calculated by the variance calculation unit 242.

[0040] That is, the information processing device 30 can detect the user 3 or the object 4 and their movements using the results output by the CSI measurement device 20. The output processing unit 244 or the information processing device 30 may associate the detected movement of a person or object with the processing result.

[0041] In this way, the object detection system 10 calculates the phase difference between predetermined resource elements for the CSI acquired by the capture unit and calculates the variance of the phase difference, thereby enabling accurate detection of people and objects. In other words, the object detection system 10 can extract and output phase difference information from the CSI, remove noise from the CSI itself, and output only data that correlates with fluctuations in the object or environment being sensed. Furthermore, the object detection system 10 can reduce the amount of output data by outputting only the processed data rather than the original CSI data.

[0042] Note that each function possessed by the CSI measurement device 20 and the information processing device 30 may be partially or entirely configured by hardware such as a programmable logic device (PLD) or a field programmable gate array (FPGA), or may be configured as a program executed by a processor such as a CPU.

[0043] For example, the CSI measuring device 20 and the information processing device 30 can be realized using a computer and a program, and the program can be recorded on a storage medium or provided via a network.

[0044] Fig. 6 is a diagram showing an example of the hardware configuration of a CSI measurement device 20 according to one embodiment. As shown in Fig. 6, for example, the CSI measurement device 20 has an input unit 50, an output unit 51, a communication unit 52, a CPU 53, a memory 54, and an HDD 55 connected via a bus 56, and has the functionality of a computer. The CSI measurement device 20 is also configured to be able to input and output data to and from a computer-readable storage medium 57.

[0045] The input unit 50 is, for example, a keyboard and a mouse. The output unit 51 is, for example, a display device such as a display that outputs images. The communication unit 52 is, for example, a wired or wireless network interface, and may have a function as an output unit that outputs data to the outside.

[0046] As described above, the CPU 53 controls each component of the CSI measurement device 20 and performs predetermined processing, etc. The memory 54 and the HDD 55 are storage units that store data, etc.

[0047] The storage medium 57 is capable of storing programs and the like that cause the CSI measurement device 20 to execute the functions of the CSI measurement device 20. Note that the architecture that configures the CSI measurement device 20 is not limited to the example shown in FIG.

[0048] Next, a description will be given of other examples and modifications of the object detection system 10 and the CSI measurement device 20. Fig. 7 is a diagram illustrating an example of an environment in which the object detection system 10 is used around a plurality of base stations 1.

[0049] 7 , the object detection system 10 may acquire CSI from two or more different base stations 1 and detect a user 3, an object 4, etc. from all of the data. In this case, the object detection system 10 may acquire CSI from the data of the base station 1 with the highest received power.

[0050] Alternatively, the object detection system 10 may focus on a specific UE 2 and acquire only CSI from base stations 1 directed to the same UE 2. Alternatively, the object detection system 10 may calculate a correlation coefficient of the analysis results of the acquired CSI, and if the correlation coefficient is large, may analyze only the data from the base station with the highest received power among the base stations with the highest correlation.

[0051] 8 is a diagram showing an overview of an environment in which multiple object detection systems 10 detect users 3 and objects 4. For example, two object detection systems 10 may acquire CSI from a single base station 1 to detect users 3 and objects 4. In this case, the two object detection systems 10 may be configured as an integrated unit or independently.

[0052] 9 is a diagram showing an example system configuration in which multiple object detection systems 10 detect a user 3 and an object 4. As shown in FIG. 9, multiple object detection systems 10 may be connected to an information processing device 30. Furthermore, the information processing device 30 may be provided with the same number of analysis units 32 (see FIG. 2) as the number of connected object detection systems 10. Furthermore, the information processing device 30 may be provided with a configuration in which the number of analysis units 32 is smaller than the number of connected object detection systems 10.

[0053] 10 is a diagram showing an example of a system configuration in which a plurality of object detection systems 10 are connected via a network. As shown in FIG. 10, a plurality of object detection systems 10 may be connected to a network 100 including the Internet, a cloud, or the like.

[0054] Furthermore, the object detection system 10 may be configured to include a CSI evaluator 24 (see FIG. 2 ) on the Internet, the cloud, or the like included in the network 100. In this case, the number of CSI evaluators 24 may be the same as the number of connected object detection systems 10, or a number of CSI evaluators 24 fewer than the number of connected object detection systems 10 may be provided.

[0055] 11 is a diagram illustrating an example environment in which the first modified example of the object detection system 10 is used. In addition to the configuration described above, the first modified example of the object detection system 10 has a sensor 40 that detects data related to the surrounding environment. The sensor 40 is a sensor that detects (senses), for example, images, sound, light, infrared rays, ultraviolet rays, vibrations, etc.

[0056] For example, when the sensor 40 detects an image or vibration, the object detection system 10 may evaluate the correlation between the detected image or vibration and the CSI (phase difference variance or movement variance) to detect movement of the user 3 or object 4 being sensed. Furthermore, the object detection system 10 may measure and output the CSI and perform movement detection, for example, when a detected image contains a specific number or more of a specific person or object, or when a specific movement is included. Furthermore, the object detection system 10 may use the sensor 40 to detect sounds such as the volume of murmurs or footsteps, and associate the sounds with the variance of the amplitude of the CSI.

[0057] In addition, if there is sufficient correlation between the data detected by the sensor 40 and the CSI, the object detection system 10 may complement the analysis results of the CSI using data acquired by the sensor 40 other than the CSI when the base station 1 is not transmitting a signal.

[0058] Furthermore, when the sensor 40 is a camera, the object detection system 10 may use data obtained from the camera as training data to learn a movement detection model based on the CSI. In this case, the object detection system 10 may extract features from the CSI to reduce unnecessary data.

[0059] 12 is a diagram showing a second modified example (object detection system 10a) of the object detection system 10. The object detection system 10a includes a CSI measuring device 20a, a camera sensor 42, and an information processing device 30. A plurality of camera sensors 42 may be provided in the object detection system 10a. For example, the CSI measuring device 20a includes a CSI measurement unit 22, a CSI evaluation unit 24, and a camera sensor 42. The camera sensor 42 may be provided only within the CSI measuring device 20a, or only outside the CSI measuring device 20a.

[0060] The camera sensor 42 outputs, for example, the detection result of the surrounding environment to the CSI evaluation unit 24. Then, the CSI evaluation unit 24 may associate the CSI with fluctuations in the environmental information based on the CSI obtained from the CSI measurement unit 22 and the environmental data obtained from the camera sensor 42, and output the result to the information processing device 30.

[0061] 13 is a diagram showing a modification (object detection system 10b) of object detection system 10a. Object detection system 10b does not include information processing device 30, but instead includes a function equivalent to information processing device 30 in a cloud or the like included in network 100.

[0062] The functions performed by the components described herein may be implemented in circuitry or processing circuitry, including general purpose processors, application specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to perform the described functions.

[0063] A processor includes transistors and other circuits and is considered to be circuitry or processing circuitry. A processor may be a programmed processor that executes programs stored in memory.

[0064] In this specification, a circuitry, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0065] If the hardware is a processor considered to be a type of circuitry, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0066] 1...base station, 2...UE, 3...user, 4...object, 10, 10a, 10b...object detection system, 20, 20a...CSI measurement device, 22...CSI measurement unit, 24...CSI evaluation unit, 30...information processing device, 32...analysis unit, 40...sensor, 42...camera sensor, 50...input unit, 51...output unit, 52...communication unit, 53...CPU, 54...memory, 55...HDD, 56...bus, 57...storage medium, 100...network, 220...capture unit, 240...phase difference calculation unit, 242...variance calculation unit, 244...output processing unit, 320...detection processing unit

Claims

1. An object detection system comprising: a capture unit that acquires CSI by capturing radio waves between a base station and a terminal; a phase difference calculation unit that calculates the phase difference between predetermined resource elements based on the CSI acquired by the capture unit; a variance calculation unit that calculates the variance or moving variance of the phase difference calculated by the phase difference calculation unit; and a detection processing unit that detects a person or object based on the variance or moving variance of the phase difference calculated by the variance calculation unit.

2. The object detection system according to claim 1, wherein the phase difference calculation unit calculates a phase difference between resource elements at different frequencies at the same time, or at different times at the same frequency, for the CSI acquired by the capture unit.

3. A CSI measurement device comprising: a capture unit that acquires CSI by capturing radio waves between a base station and a terminal; a phase difference calculation unit that calculates a phase difference between predetermined resource elements based on the CSI acquired by the capture unit; a variance calculation unit that calculates the variance or moving variance of the phase difference calculated by the phase difference calculation unit; and an output processing unit that performs predetermined processing on the variance or moving variance of the phase difference calculated by the variance calculation unit and outputs the result to an external device.

4. The CSI measurement device according to claim 3, wherein the phase difference calculation unit calculates a phase difference between resource elements at different frequencies at the same time, or at different times at the same frequency, for the CSI acquired by the capture unit.

5. An object detection method comprising: a capture step of acquiring CSI by capturing radio waves between a base station and a terminal; a phase difference calculation step of calculating a phase difference between predetermined resource elements based on the CSI acquired by the capture step; a variance calculation step of calculating a variance or moving variance of the phase difference calculated by the phase difference calculation step; a sensing step of detecting data related to the surrounding environment using a sensor; and a detection processing step of detecting a person or object based on the variance or moving variance of the phase difference calculated by the variance calculation step and the data detected by the sensing step.

6. The object detection method according to claim 5, wherein the phase difference calculation step calculates a phase difference between resource elements at different frequencies at the same time, or between resource elements at different times at the same frequency, for the CSI acquired in the capture step.

7. A CSI measurement program for causing a computer to function as each part of a CSI measurement device having: a capture unit that acquires CSI by capturing radio waves between a base station and a terminal; a phase difference calculation unit that calculates a phase difference between predetermined resource elements based on the CSI acquired by the capture unit; a variance calculation unit that calculates the variance or moving variance of the phase difference calculated by the phase difference calculation unit; and an output processing unit that performs predetermined processing on the variance or moving variance of the phase difference calculated by the variance calculation unit and outputs the result to an external device.

8. The CSI measurement program according to claim 7, wherein the phase difference calculation unit calculates a phase difference between resource elements at different frequencies at the same time, or at different times at the same frequency, for the CSI acquired by the capture unit.

Images