Electronic device, method for controlling electronic device, and program

Abstract

The present invention provides an electronic device, a control method for the electronic device, and a program that can efficiently detect a location with good accuracy by transmitting and receiving radio waves. [Solution] An electronic device that estimates the position of a sensor equipped with an antenna calculates the average of the product of the object's position and the received intensity within a predetermined region (a voxel) and the variance of the product of the object's position and the received intensity within the voxel, based on the position of the object and the received intensity based on the reflected wave obtained from the transmitted wave sent from the antenna and the reflected wave when the transmitted wave is reflected by an object. An evaluation function is calculated based on the position of the object and a function indicating the density of the point cloud within the voxel, obtained from the transmitted wave sent from the antenna and the reflected wave when the transmitted wave is reflected by an object. The position of the sensor is then estimated based on the evaluation function.

Description

[Technical Field]

[0001] This disclosure relates to electronic equipment, methods for controlling electronic equipment, and programs. [Background technology]

[0002] For example, in fields such as the automotive industry, technologies for measuring the distance between a vehicle and a designated object are considered important. In particular, in recent years, various radar (RADAR (Radio Detecting and Ranging)) technologies have been researched, which measure the distance to an object by transmitting radio waves such as millimeter waves and receiving the reflected waves reflected from obstacles and other objects. The importance of such distance measurement technologies is expected to increase even further in the future with the development of technologies that assist drivers and technologies related to autonomous driving that automate part or all of the driving. Hereafter, radar sensors that use millimeter waves will also be referred to as "millimeter wave sensors."

[0003] Furthermore, techniques are known that simultaneously estimate the self-position of a moving object and create an environmental map, such as SLAM (Simultaneous Localization and Mapping). Research is also progressing on techniques that utilize the output of, for example, millimeter-wave sensors, LiDAR (Light Detection and Ranging), and / or cameras that capture images, in order to estimate the self-position of a moving object with good accuracy. For example, Patent Document 1 proposes adjusting the detection results by a millimeter-wave sensor according to the three-dimensional shape of each region of the three-dimensional map generated from images captured by an imaging device. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2021-86298 [Overview of the project] [Problems that the invention aims to solve]

[0005] For example, if the position of an aircraft or other object can be detected efficiently and with good accuracy by transmitting and receiving radio waves such as millimeter waves, it is expected to be useful in a wide variety of fields.

[0006] The purpose of this disclosure is to provide an electronic device, a control method for the electronic device, and a program that can efficiently detect a position with good accuracy by transmitting and receiving radio waves. [Means for solving the problem]

[0007] An electronic device according to one embodiment is An electronic device that estimates the position of a sensor equipped with an antenna, Let N be an integer greater than or equal to 1. Based on the position X(N-1) of the object and the received intensity P(N-1) based on the reflected wave, obtained from the transmitted wave sent from the antenna and the reflected wave reflected by the object in the N-1th frame, which is a predetermined time interval, The average of the product of the position X(N-1) of the object within a predetermined region of the voxel and the received intensity P(N-1) is A(N-1), The variance V(N-1) of the product of the position X(N-1) of the object within the voxel and the received intensity P(N-1) is calculated. The above average A(N-1) and, The aforementioned variance V(N-1) and, The position X(N) of the object is obtained from the transmitted wave sent from the antenna and the reflected wave reflected by the object in the Nth frame following the (N-1)th frame, The function V(N-1) represents the density of the point cloud within the voxel, Based on this, the evaluation function ρ(N) is calculated, The position of the sensor is estimated based on the evaluation function ρ(N).

[0008] A control method according to one embodiment is: A control method for electronic equipment that estimates the position of a sensor equipped with an antenna, Let N be an integer greater than or equal to 1. Based on the position X(N-1) of the object and the received intensity P(N-1) based on the reflected wave, obtained from the transmitted wave sent from the antenna and the reflected wave reflected by the object in the N-1th frame, which is a predetermined time interval, The average of the product of the position X(N-1) of the object within a predetermined region of the voxel and the received intensity P(N-1) is A(N-1), A step of calculating the variance V(N-1) of the product of the position X(N-1) of the object within the voxel and the received intensity P(N-1), The above average A(N-1) and, The aforementioned variance V(N-1) and, The position X(N) of the object is obtained from the transmitted wave sent from the antenna and the reflected wave reflected by the object in the Nth frame following the (N-1)th frame, The function V(N-1) represents the density of the point cloud within the voxel, The steps include: calculating the evaluation function ρ(N) based on this; The steps include: estimating the position of the sensor based on the evaluation function ρ(N); Includes.

[0009] A program according to one embodiment is: In electronic devices that estimate the position of a sensor equipped with an antenna, Let N be an integer greater than or equal to 1. Based on the position X(N-1) of the object and the received intensity P(N-1) based on the reflected wave, obtained from the transmitted wave sent from the antenna and the reflected wave reflected by the object in the N-1th frame, which is a predetermined time interval, The average of the product of the position X(N-1) of the object within a predetermined region of the voxel and the received intensity P(N-1) is A(N-1), A step of calculating the variance V(N-1) of the product of the position X(N-1) of the object within the voxel and the received intensity P(N-1), The above average A(N-1) and, The aforementioned variance V(N-1) and, In the N-th frame following the (N-1)-th frame, the position X(N) of the object obtained from the transmitted wave sent from the antenna and the reflected wave obtained by reflecting the transmitted wave from the object, and a function V(N-1) indicating the density of the point cloud in the voxel, a step of calculating an evaluation function ρ(N) based on the above; a step of estimating the position of the sensor based on the evaluation function ρ(N); is executed.

Advantages of the Invention

[0010] According to one embodiment, it is possible to provide an electronic device, a control method for an electronic device, and a program that can efficiently detect a position with good accuracy by transmitting and receiving radio waves.

Brief Description of the Drawings

[0011] <0000​​​​​​​​​​​​​​​​​​​​​​​​​​ [Figure 10] This is a block diagram showing the functional configuration of an electronic device according to another embodiment. [Figure 11] This is a flowchart illustrating the operation of an electronic device according to another embodiment. [Figure 12] This is a block diagram showing the functional configuration of an electronic device according to another embodiment. [Modes for carrying out the invention]

[0012] One embodiment will be described below with reference to the drawings.

[0013] In this disclosure, “electronic device” may mean an electric-powered device. In this disclosure, “system” may mean a device that is electric-powered. In this disclosure, “user” may mean a person (typically a human) who uses a system and / or electronic device according to one embodiment. A user may include a person who, by using a system and / or electronic device according to one embodiment, enjoys the benefit of detecting the position of such electronic device or other device.

[0014] An electronic device according to one embodiment can be mounted on a vehicle (mobile object) such as an automobile, and can detect a predetermined object (target) present in the vicinity of the mobile object as a target. To this end, the electronic device according to one embodiment can transmit a wave (radio wave) to the vicinity of the mobile object from a transmitting antenna (transmitting antenna array) installed on the mobile object. Furthermore, the electronic device according to one embodiment can receive reflected waves from the transmitted wave from a receiving antenna (receiving antenna array) installed on the mobile object. At least one of the transmitting antenna and the receiving antenna may be provided with a radar sensor such as a millimeter-wave sensor installed on the mobile object.

[0015] The following describes a typical example in which an electronic device according to one embodiment is mounted on a mobile vehicle, such as an automobile. However, the electronic device according to one embodiment is not limited to being mounted on an automobile. The electronic device according to one embodiment may be mounted on various mobile vehicles, such as autonomous vehicles, buses, taxis, trucks, motorcycles, bicycles, ships, aircraft, helicopters, agricultural equipment such as tractors, snowplows, street sweepers, police cars, ambulances, drones, and rockets. Furthermore, the electronic device according to one embodiment is not necessarily limited to being mounted on a mobile vehicle that moves under its own power. For example, the mobile vehicle on which the electronic device according to one embodiment is mounted may be a trailer towed by a tractor. Also, the mobile vehicle (e.g., automobile) in this disclosure is not limited by length, width, height, engine displacement, passenger capacity, or load capacity. For example, the automobile in this disclosure may include automobiles with an engine displacement greater than 660cc and automobiles with an engine displacement of 660cc or less, so-called kei cars. Furthermore, the automobile in this disclosure may also include automobiles that use electricity for part or all of their energy and use a motor. Also, the electronic device according to one embodiment does not necessarily have to be mounted on a mobile vehicle. For example, the electronic device according to one embodiment may be attached to or built into other equipment fixed to the ground. Alternatively, the electronic device according to one embodiment may be fixed to the ground, for example.

[0016] An electronic device according to one embodiment can measure the distance between a sensor and an object in a situation where at least one of the sensor and a predetermined object may be moving. Furthermore, an electronic device according to one embodiment can measure the distance between a sensor and an object even when both the sensor and the object are stationary.

[0017] The electronic device according to one embodiment described below can efficiently detect the position of itself or other objects with good accuracy by transmitting and receiving radio waves, such as a millimeter-wave sensor. According to the electronic device according to one embodiment, for example, in a technique such as SLAM, the self-position of a moving object can be estimated efficiently with good accuracy.

[0018] (The process leading to the conception of an electronic device according to one embodiment) In describing an electronic device according to one embodiment, we will first discuss the detection of the surrounding environment using a millimeter-wave sensor and the detection of the surrounding environment using sensors other than millimeter-wave sensors. Here, we will use a LiDAR sensor as an example of a sensor other than a millimeter-wave sensor.

[0019] Figure 1 shows the interior of a typical room, which was prepared as the ambient environment to be detected by the sensors. The results of detecting this ambient environment using LiDAR and millimeter-wave sensors are described below.

[0020] Figure 2 shows a plot of point cloud data obtained by detecting the surrounding environment shown in Figure 1 using a typical LiDAR sensor. The point cloud plot in Figure 2 provides an overview of the surrounding environment shown in Figure 1. Figure 2 demonstrates that the walls of the surrounding environment shown in Figure 1 are detected with good accuracy by using a typical LiDAR sensor.

[0021] Figure 3 shows a plot of the probability density distribution obtained by detecting the surrounding environment shown in Figure 1 using a typical millimeter-wave sensor. The plot in Figure 3, like in Figure 2, shows an overview of the surrounding environment shown in Figure 1. Figure 3 shows that when using a typical millimeter-wave sensor, noise can be detected in areas other than the walls of the surrounding environment shown in Figure 1.

[0022] Furthermore, comparing the results shown in Figure 2 and Figure 3, it can be seen that the two differ in the following respects. First, the detection results from the millimeter-wave sensor are noisier than the detection results from the LiDAR sensor. On the other hand, the detection results from the LiDAR sensor are known to be susceptible to light reflection and other factors. Also, the detection results from the millimeter-wave sensor consist of fewer point clouds than the detection results from the LiDAR sensor, meaning they have lower resolution (or resolution). In general, the detection results from the millimeter-wave sensor have lower resolution (or resolution) than the detection results from sensors that capture images, such as cameras. Conversely, detection from the LiDAR sensor yields a larger number of point clouds in one frame. For this reason, in order to use a method such as NDT (Normal Distribution Transform) with a LiDAR sensor, it may be necessary to perform some kind of processing to reduce the number of point clouds. NDT is a method of normalizing the point cloud of an object by dividing the environment map (search space) into voxels (in a grid), calculating a normal distribution, and matching it with the point cloud data obtained from the LiDAR sensor.

[0023] Here, we will further discuss NDT. NDT is known as a method used for matching point cloud data. NDT efficiently represents the three-dimensional structure of the environment and is considered effective when estimating location and / or creating maps. In NDT, point cloud data obtained from sensors is modeled as a normal distribution.

[0024] The basic steps of NDT can be performed as follows: First, the environment is divided into voxels. Next, the distribution of point cloud data contained within each voxel is calculated. Specifically, the mean and variance are determined from the location of the point cloud and modeled as a normal distribution. This allows the point cloud data of the entire environment to be statistically represented using the normal distribution obtained for each voxel. Then, the current point cloud data (data obtained after the modeled data) is compared with this modeled data, and the position and orientation are optimized. Specifically, the position and orientation are adjusted so that the point cloud data best fits the modeled normal distribution.

[0025] The detection results from the millimeter-wave sensor are not output as a point cloud, but rather as a representation of the surrounding terrain in the form of a probability density function. Typical millimeter-wave sensors calculate the center value of the output obtained in the form of a probability density function and output that center as the measurement point. The probability distribution information that is removed when the output from the millimeter-wave sensor is converted into a point cloud includes information about the surrounding terrain (environment). In other words, when the output from the millimeter-wave sensor is expressed in the form of a point cloud, the probability distribution extends in the direction of obstacles in the surrounding environment. Through demonstration experiments, the applicant has confirmed that by utilizing the probability distribution information that is removed when the output from the millimeter-wave sensor is converted into a point cloud, the lower resolution and accuracy compared to LiDAR sensors can be effectively compensated for.

[0026] Figure 4 is a conceptual diagram illustrating an example of applying NDT to the output of a LiDAR sensor.

[0027] To estimate self-position based on the output of a LiDAR sensor, a method is known that approximates the probability density function by applying NDT to the point cloud detected by the LiDAR sensor. For example, suppose a point cloud is detected by a LiDAR sensor as shown on the left side of Figure 4. By applying NDT to the point cloud detected in this way, the point cloud can be matched. Specifically, as shown on the right side of Figure 4, first, the point cloud is divided into voxels (in a grid). Then, as shown on the right side of Figure 4, matching is performed by assigning a normal distribution to the point cloud within each voxel.

[0028] As shown in Figure 4, applying NDT to the output of a LiDAR sensor results in a large number of points in a single frame, as mentioned above. Therefore, it may be necessary to reduce the number of points using some method. In such a method, NDT is applied to each point for each frame of the LiDAR sensor output. Consequently, methods that apply NDT to the output of a LiDAR sensor tend to be computationally intensive.

[0029] Figure 5 shows a model of the data obtained after applying signal processing, such as FFT (Fast Fourier Transform), to the output of a millimeter-wave sensor, representing it as a probability density.

[0030] As shown in Figure 5, the data obtained after signal processing such as FFT on the output of a millimeter-wave sensor can be considered equivalent to NDT data without the need for matching of each point cloud. Therefore, data obtained after signal processing such as FFT on the output of a millimeter-wave sensor allows for efficient inter-frame matching using a small amount of point cloud data. With methods that estimate the self-position of a moving object based on the output of a millimeter-wave sensor, the probability density function can be directly obtained from the millimeter-wave sensor even with a relatively small number of point clouds. For this reason, such methods can reduce the computational cost of the conversion process. Furthermore, since such methods can reduce the data size, calculations can be performed even when the implemented memory is relatively small.

[0031] The frames of this disclosure are described below. A frame of this disclosure refers to distance, velocity, and / or angle data acquired by a millimeter-wave sensor in one measurement cycle. A higher frame rate (number of frames obtained per second) improves real-time performance and / or accuracy. A frame consists of distance data, velocity data, and / or angle data. Distance data is obtained by measuring the distance from the sensor to an object. This can be calculated based on the time it takes for radio waves to reflect off the object and return. Velocity data is obtained by measuring the relative velocity of an object using the Doppler effect. This allows for the determination of whether the object is moving towards or away from the sensor. Angle data is obtained by measuring the direction of an object. This is obtained by calculating the angle based on the phase difference of the incoming radio waves. Here, the frame collection process can consist of transmitting radio waves, receiving reflected waves, data analysis, and data aggregation. By transmitting and receiving radio waves for a number of antennas equal to the number of antennas, and then performing data analysis and data aggregation, distance, velocity, and / or angle data can be obtained.

[0032] The following describes a method for estimating self-position by applying NDT to the output of a millimeter-wave sensor using an electronic device according to one embodiment. This method improves the accuracy of self-position estimation even with a small number of detected point clouds (points) by combining information on the error distribution, which was not previously used. The electronic device according to one embodiment efficiently utilizes less point cloud information than when using a LiDAR sensor, that is, it utilizes the raw data of the millimeter-wave sensor. Here, raw data may refer to information read by an image sensor, such as a millimeter-wave sensor, recorded as is. According to the electronic device according to one embodiment, signal processing (e.g., CFAR processing and / or direction of arrival estimation processing) performed during normal point cloud acquisition is not required for the output of the millimeter-wave sensor. For example, the electronic device according to one embodiment may use the raw data output from the millimeter-wave sensor as is. Furthermore, the electronic device according to one embodiment may perform only CFAR processing on the millimeter-wave sensor and not direction of arrival estimation processing. According to the electronic device according to one embodiment, SLAM using a millimeter-wave sensor can be realized.

[0033] (Configuration of an electronic device according to one embodiment) Next, the functional configuration of an electronic device according to one embodiment will be described.

[0034] Figure 6 is a block diagram showing the functional configuration of an electronic device according to one embodiment. As shown in Figure 6, the electronic device according to one embodiment may include a physical layer 1, a signal processing layer 10, and an application layer 20.

[0035] The physical layer 1 shown in Figure 1 may be, for example, the hardware that constitutes a millimeter-wave sensor. As shown in Figure 1, the physical layer 1 may include, for example, a transmitting antenna 2, a receiving antenna 3, an RF unit 4, a relay unit 5, a DSP (Digital Signal Processor) 6, and an FPGA (Field Programmable Gate Array) 7.

[0036] The transmitting antenna 2 may be, for example, an antenna array (array antenna) as an antenna that transmits radio waves. The receiving antenna 3 may be, for example, an antenna array (array antenna) as an antenna that receives reflected waves that have been reflected by an object or the like from the transmitting antenna 2. The RF unit 4 may be a circuit that provides various functions necessary for transmitting radio waves by the transmitting antenna 2 and / or receiving radio waves by the receiving antenna 3. The relay unit 5 may be a circuit that provides functions for relaying signals, such as converting between digital signals and analog signals. The DSP 6 may be a processor to which setting values ​​set by the antenna setting unit 12 of the signal processing layer 10 are supplied. The DSP 6 has the function of making various settings when transmitting radio waves from the transmitting antenna 2 of the physical layer 1 based on the setting values ​​set by the antenna setting unit 12 of the signal processing layer 10. The FPGA 7 may be a device that has the function of supplying a signal based on radio waves received from the receiving antenna 3 as RAW data to the FFT processing unit 14 of the signal processing layer 10. Each functional unit constituting the physical layer 1 may be based on the same or similar concepts as each functional unit constituting a known millimeter-wave sensor. Therefore, a more detailed explanation of each functional component constituting the physical layer 1 will be omitted.

[0037] The signal processing layer 10 shown in Figure 2 may be composed of any circuit, CPU (Central Processing Unit), or DSP, for example, that has the function of performing various processing on the signals transmitted and received by the millimeter-wave sensor. The signal processing layer 10 may also be composed of specific means by which software and hardware resources cooperate. As shown in Figure 2, the signal processing layer 10 may include, for example, an antenna setting unit 12, an FFT (Fast Fourier Transform) processing unit 14, a CFAR (Constant False Alarm Rate) processing unit 16, and an arrival direction estimation unit 18.

[0038] The antenna setting unit 12 sets setting values ​​for various settings when transmitting radio waves from the transmitting antenna 2 of the physical layer 1. The setting values ​​set by the antenna setting unit 12 may be supplied to the DSP 6 of the physical layer 1. The FFT processing unit 14 has the function of performing FFT processing on the RAW data supplied from the physical layer 1. The data processed by the FFT processing unit 14 may be supplied to the CFAR processing unit 16. The CFAR processing unit 16 has the function of performing CFAR processing on the data processed by the FFT processing unit 14. The data processed by the CFAR processing unit 16 may be supplied to the direction of arrival estimation unit 18. The direction of arrival estimation unit 18 has the function of estimating the direction of arrival of the radio waves (received waves) based on the data processed by the CFAR processing unit 16.

[0039] As a result of signal processing by each functional unit of the signal processing layer 10, the signal processing layer 10 outputs information such as the object's position coordinates, signal strength (received radio wave strength), and / or the object's velocity. The information output from the signal processing layer 10 may be supplied to the application layer 20. Each functional unit constituting the signal processing layer 10 may be based on the same or similar concepts as each functional unit constituting a known millimeter-wave sensor. Therefore, a more detailed explanation of each functional unit constituting the signal processing layer 10 is omitted.

[0040] The application layer 20 shown in Figure 2 may be application software executed in, for example, a signal processing unit or any control unit that constitutes a millimeter-wave sensor. Alternatively, the application layer 20 may be configured by specific means in which software and hardware resources cooperate. The application layer 20 may include, for example, a front-end 22 and a back-end 24. The front-end 22 may include an NDT (Normal Distribution Transform) processing unit 222 and a Doppler processing unit 224. The back-end 24 may include an odometry processing unit 242, a loop closing processing unit 244, and a pose graph optimization processing unit 246.

[0041] In the front-end 22 of the application layer 20, the NDT processing unit 222 has the function of performing NDT processing based on information supplied from the signal processing layer 10. The Doppler processing unit 224 also has the function of performing Doppler processing based on information supplied from the signal processing layer 10. The information processed in the front-end 22 of the application layer 20 may be supplied to the back-end 24. Various algorithms may be used as SLAM algorithms in this disclosure.

[0042] In the backend 24 of the application layer 20, the odometry processing unit 242 performs odometry processing based on information supplied from the frontend. The information processed by the odometry processing unit 242 may be supplied to the pose graph optimization processing unit 246. In addition, the loop closure processing unit 244 performs loop closure processing based on information supplied from the frontend. The information processed by the loop closure processing unit 244 may be supplied to the pose graph optimization processing unit 246. The pose graph optimization processing unit 246 performs pose graph optimization processing based on the information processed by the odometry processing unit 242 and the information processed by the loop closure processing unit 244. As a result of processing by each functional unit of the application layer 20, the estimated position information of the object is output from the application layer 20.

[0043] An electronic device according to one embodiment may not include at least some of the functional components shown in Figure 6, and may also include other functional components other than those shown in Figure 6.

[0044] Furthermore, the electronic device according to one embodiment may comprise the physical layer 1, the signal processing layer 10, and the application layer 20 as an integrated unit, or at least a part of them may be provided separately. For example, the electronic device according to one embodiment may comprise all of the physical layer 1, the signal processing layer 10, and the application layer 20 as an integrated unit. Alternatively, for example, the electronic device according to one embodiment may comprise the physical layer 1 and the signal processing layer 10 in the housing of the millimeter-wave sensor, and the application layer 20 in other external equipment such as a server. Alternatively, for example, the electronic device according to one embodiment may comprise the physical layer 1 in the housing of the millimeter-wave sensor, and the signal processing layer 10 and the application layer 20 in other external equipment such as a server.

[0045] (Operation of an electronic device according to one embodiment) Figure 7 is a diagram conceptually illustrating the operation of an electronic device according to one embodiment. The operation of the electronic device according to one embodiment will be described below.

[0046] In Figure 7, only the physical layer 1 of the electronic device shown in Figure 6 is schematically represented. That is, the physical layer 1 shown in Figure 7 may represent the hardware of a millimeter-wave sensor equipped with a transmitting antenna and / or a receiving antenna. The area A2 shown by light shading in Figure 7 represents the detection range of the millimeter-wave sensor shown as physical layer 1. The area A1 shown by dark shading in Figure 7 represents the detection range of one element of the antenna of the millimeter-wave sensor shown as physical layer 1. Figure 7 shows an overview of the detection range of the millimeter-wave sensor shown as physical layer 1.

[0047] As described above, the electronic device according to one embodiment estimates its own position by applying NDT to the output of a millimeter-wave sensor. Here, in order to improve the shape of the NDT and the accuracy of the matching, which can greatly affect the accuracy of SLAM, the electronic device according to one embodiment operates as follows. [1] Fitting is performed using not only the coordinates of the point cloud obtained as output from the millimeter-wave sensor, but also its received intensity. [2] When matching the point cloud obtained as output from a millimeter-wave sensor with the NDT, the density of the point cloud according to distance is taken into consideration (i.e., voxels closer to the self-position where the point cloud is denser are given more weight).

[0048] To achieve the above-described operation, an electronic device according to one embodiment may employ an algorithm that includes the following steps [1] to [3].

[0049] Step 1: Divide the point cloud obtained as output from the millimeter-wave sensor into voxels of a fixed size.

[0050] A voxel of a certain size may be, for example, a grid as shown in Figure 7. In Figure 7, the received intensity of the point cloud constituting the voxel is shown according to the intensity of gray in the grayscale. That is, darker areas in the voxel indicate high received intensity of the point cloud, while lighter areas indicate low received intensity of the point cloud.

[0051] Step 2: When calculating the mean and variance from the point cloud within each voxel, apply the received intensity information.

[0052] The mean q of the voxels shown in Figure 7 can be expressed by the following equation (1).

number

[0053] Furthermore, the variance C of the voxels shown in Figure 7 can be expressed as follows: Equation (2). In Equation (2), T represents the transpose of the matrix.

number

[0054] Step 3: Matching is performed by reflecting the ratio of the point cloud obtained as output from the millimeter-wave sensor to the number of voxels, according to the distance from the antenna in physical layer 1.

[0055] The evaluation function used when performing the matching described above, i.e., the matching evaluation function (normal distribution) ρ, can be expressed as shown in equation (3) below. In equation (3), T represents the transpose of the matrix.

number

number

[0056] Figure 8 is a flowchart illustrating the operation of an electronic device according to one embodiment. Hereinafter, with reference to Figure 8, we will first describe the general operation of the electronic device according to one embodiment.

[0057] When the operation shown in Figure 8 begins, the application layer 20 of the electronic device according to one embodiment acquires a point (measurement point) detected by the physical layer 1 and signal processing layer 10 that constitute the millimeter-wave sensor (step S1). In step S1, the application layer 20 may acquire a measurement point up to the direction of arrival estimation stage, as shown in Figure 6. Alternatively, in step S1, the application layer 20 may acquire a measurement point up to the CFAR processing stage, as described later, or it may acquire RAW data up to the FFT processing stage.

[0058] When the application layer 20 acquires a measurement point, the NDT processing unit 222 calculates the NDT based on the acquired measurement point (step S2). In step S2, the NDT processing unit 222 may calculate the mean q shown in equation (1) and the variance C shown in equation (2) above. Also in step S2, the NDT processing unit 222 may apply the received signal strength information when calculating the mean q and the variance C. That is, when the NDT processing unit 222 calculates the mean q and the variance C, x k and p k The calculation is performed including the following. In short, the NDT processing unit 222 calculates the mean q and variance C based not only on the received signal strength of the point cloud but also on the position information.

[0059] In step S2, once the NDT is calculated, the odometry processing unit 242 calculates the evaluation function ρ as shown in equation (3) above (step S3). In step S3, the odometry processing unit 242 calculates the evaluation function ρ using the density distribution V in the voxels and the received radio wave intensity p. That is, in step S3, the odometry processing unit 242 calculates the evaluation function ρ by giving importance to relatively close voxels.

[0060] In the above processing, the NDT processing unit 222 may perform the processing up to equations (1) and (2). Equation (3) represents the odometry (calculation of the self-position trajectory) processing. Therefore, the odometry processing unit 242 may perform the processing of equation (3).

[0061] In step S3, once the evaluation function ρ is calculated, the odometry processing unit 242 calculates (estimates) the trajectory of the millimeter-wave sensor's own position using the evaluation function ρ (step S4). In step S4, the odometry processing unit 242 may calculate (estimate) the trajectory of the millimeter-wave sensor's own position by, for example, calculating R and t that maximize the evaluation function ρ. Alternatively, in step S4, depending on the algorithm used, the odometry processing unit 242 may calculate (estimate) the trajectory of the millimeter-wave sensor's own position by calculating R and t that minimize the evaluation function ρ. The processing in step S4 may be performed based on conventionally known knowledge. Here, R and t represent estimated values ​​of the attitude of the robot or mobile body on which the millimeter-wave sensor is installed. Specifically, R represents rotation and t represents translational movement. By using R and t, the current point cloud can be converted to the point cloud of the previous frame.

[0062] Figure 9 is a flowchart that specifically illustrates the operation of the electronic device according to one embodiment shown in Figure 6. Hereinafter, the general operation of the electronic device according to one embodiment shown in Figure 6 will be specifically described with reference to Figure 9.

[0063] When the operation shown in Figure 9 begins, the direction of arrival estimation unit 18 estimates the direction of arrival of the received wave (reflected wave) (step S11). The process in step S11 may correspond to the process in step S1 shown in Figure 8. In the operation shown in Figure 9, the application layer 20 can obtain the point cloud output after performing the direction of arrival estimation process.

[0064] In step S11, once the direction of arrival of the received wave (reflected wave) is estimated, the NDT processing unit 222 divides the obtained point cloud into voxels of a fixed size (step S12). In step S12, the NDT processing unit 222 may divide the obtained point cloud into voxels of a fixed size as shown in Figure 7.

[0065] In step S12, once the point cloud is divided into voxels of a certain size, the NDT processing unit 222 fits a normal distribution to the distribution within each voxel (step S13). In step S13, the NDT processing unit 222 calculates the mean (equation (1) above) and variance (equation (2) above) for each voxel as information about the normal distribution.

[0066] In step S13, after fitting a normal distribution to the distribution within each voxel, the NDT processing unit 222 reflects the received intensity of the point cloud (measurement points) (step S14). In step S14, the NDT processing unit 222 reflects the received intensity of the measurement points as shown in the mean of equation (1) and the variance of equation (2) above. The processing from step S12 to step S14 may correspond to the processing of step S2 shown in Figure 8.

[0067] In step S14, after reflecting the received intensity at the measurement point, the pause graph optimization processing unit 246 reflects the influence of voxels (step S15). In step S15, the pause graph optimization processing unit 246 calculates the Voxel influence as shown in the normal distribution of equation (3) above. j By using this method, the effects of voxels may be reflected. The process in step S15 may correspond to the process in step S3 shown in Figure 8.

[0068] After step S15, the application layer 20 can estimate the self-position of the millimeter-wave sensor after undergoing optimization processing.

[0069] According to one embodiment of the electronic device, the position can be detected efficiently with good accuracy by transmitting and receiving radio waves. Therefore, according to one embodiment of the electronic device, it is expected to be useful in a wide variety of situations, such as self-position estimation in SLAM.

[0070] (Other Embodiment 1) Other embodiments of the electronic device according to the above-described embodiment will be explained below.

[0071] Figure 10 is a block diagram showing the functional configuration of an electronic device according to another embodiment. Hereinafter, only the differences from the electronic device according to one embodiment shown in Figure 6 will be described.

[0072] As shown in Figure 10, in the electronic device according to another embodiment, the signal processed by the FFT processing unit 14 of the signal processing layer 10 may be supplied to the application layer 20. That is, in the electronic device according to another embodiment shown in Figure 10, the signal processed by the FFT processing unit 14 of the signal processing layer 10 may be supplied to the application layer 20 without performing CFAR processing and direction of arrival estimation. In this case, in step S1 shown in Figure 8, the application layer 20 may acquire RAW data up to the FFT processing stage.

[0073] Figure 11 is a flowchart that specifically illustrates the operation of the electronic device according to another embodiment shown in Figure 10. Hereinafter, the general operation of the electronic device according to the other embodiment shown in Figure 10 will be specifically described with reference to Figure 11.

[0074] When the operation shown in Figure 11 begins, the FFT processing unit 14 performs FFT processing on the signal based on the received wave (reflected wave) (step S21). In the operation shown in Figure 11, the application layer 20 can obtain a heatmap by performing FFT processing on the RAW data. Here, the heatmap obtained in step S21 may be, for example, as shown in Figure 5. In Figure 5, for example, the vertical axis may represent distance and the horizontal axis may represent angle. In Figure 5, a two-dimensional heatmap is shown for simplification, but a three-dimensional heatmap may also be obtained.

[0075] In step S21, once a heatmap is obtained by the FFT process, the NDT processing unit 222 divides the heatmap into cells (step S22). In variations of other embodiments, this step S22 may be skipped.

[0076] In step S22, once the heatmap is divided into cells, the NDT processing unit 222 fits a distribution, such as a normal distribution, to the output of the heatmap using a method such as a Gaussian Mixture Model (GMM) (step S23). In variations of other embodiments, this step S23 may be skipped.

[0077] In step S23, once the normal distribution has been fitted, the NDT processing unit 222 divides the obtained normal distribution into voxels of a fixed size (step S24). In step S24, the NDT processing unit 222 may divide the obtained point cloud into voxels of a fixed size, as shown in Figure 7.

[0078] In step S24, if the normal distribution is divided into voxels of a certain size, the NDT processing unit 222 fits the normal distribution from the distribution within each voxel (step S25). In step S25, the NDT processing unit 222 calculates the mean (the above formula (1)) and variance (the above formula (2)) in each voxel as information on the normal distribution. In a modification of another embodiment, when performing the process of step S25, without using p k and V j , the evaluation function ρ of the above formula (3) may be calculated only with the mean and variance of the position x.

[0079] In step S25, when the normal distribution is fitted from the distribution within each voxel, the NDT processing unit 222 reflects the reception intensity of the normal distribution (step S26). In step S26, the NDT processing unit 222 may reflect the reception intensity of the normal distribution, such as the mean of the above formula (1) and the variance of the above formula (2).

[0080] In step S26, when the reception intensity of the normal distribution is reflected, the pose graph optimization processing unit 246 reflects the influence of the voxel (step S27). In step S27, the pose graph optimization processing unit 246 may reflect the influence of the voxel by using V j as in the normal distribution of the above formula (3).

[0081] Thus, also in the electronic device according to another embodiment shown in FIG. 10, by the operation shown in FIG. 11, the position can be efficiently detected with good accuracy by radio wave transmission and reception.

[0082] (Another Embodiment 2) Hereinafter, still another embodiment of the electronic device according to the above-described embodiment will be described.

[0083] FIG. 12 is a block diagram showing a functional configuration of an electronic device according to another embodiment. Hereinafter, only differences from the electronic device according to the embodiment shown in FIG. illustrating a first embodiment will be described.

[0084] As shown in Figure 12, in the electronic device according to another embodiment, the signal processed by the FFT processing unit 14 of the signal processing layer 10 may be supplied to the application layer 20 after being processed by the CFAR processing unit 16. That is, in the electronic device according to another embodiment shown in Figure 12, the signal processed by the FFT processing unit 14 of the signal processing layer 10 may be supplied to the application layer 20 without direction estimation. In this case, in step S1 shown in Figure 8, the application layer 20 may acquire the measurement point up to the CFAR processing stage.

[0085] Thus, in the electronic device according to the other embodiment shown in Figure 12, the position can be detected efficiently with good accuracy by transmitting and receiving radio waves through an operation similar to the operation shown in Figure 11.

[0086] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art will find it easy to make various modifications or alterations based on this disclosure. Therefore, it should be noted that these modifications or alterations are within the scope of this disclosure. For example, the functions included in each functional part can be rearranged in a logically consistent manner. Multiple functional parts may be combined into one or divided. The embodiments relating to this disclosure described above are not limited to being implemented strictly according to the respective embodiments, but can be implemented by combining features or omitting parts as appropriate. In other words, the contents of this disclosure can be modified and altered in various ways based on this disclosure by those skilled in the art. Therefore, these modifications and alterations are within the scope of this disclosure. For example, in each embodiment, each functional part, each means, each step, etc. can be added to other embodiments in a logically consistent manner, or replaced with each functional part, each means, each step, etc. from other embodiments. Also, in each embodiment, multiple functional parts, each means, each step, etc. can be combined into one or divided. Furthermore, the embodiments of this disclosure described above are not limited to being implemented strictly according to the respective embodiments described, but can also be implemented by combining or omitting some of the features as appropriate.

[0087] The embodiments described above are not limited to implementation as electronic devices. For example, the embodiments described above may be implemented as a control method for devices such as electronic devices. Furthermore, the embodiments described above may be implemented as a program executed by a device such as an electronic device, or as a storage medium or recording medium on which a program is recorded. [Explanation of symbols]

[0088] 1 Physical layer 2. Transmitting antenna (transmitting antenna array) 3. Receiving antenna (receiving antenna array) 4 RF section 5. Relay section 6 DSP(Digital Signal Processor) 7 FPGA(Field Programmable Gate Array) 10 Signal Processing Layer 12 Antenna setting section 14. FFT (Fast Fourier Transform) Processing Unit 16. CFAR (Constant False Alarm Rate) Processing Unit 18 Direction of Arrival Estimator 20 Application Layer 22 Front End 222 NDT (Normal Distribution Transform) Processing Unit 224 Doppler Processing Unit 24 backend 242 Odometry Processing Unit 244 Loop Closing Processing Unit 246 Pose Graph Optimization Processing Unit

Claims

1. An electronic device that estimates the position of a sensor equipped with an antenna, Let N be an integer greater than or equal to 1. Based on the position X(N-1) of the object and the received intensity P(N-1) based on the reflected wave, obtained from the transmitted wave transmitted from the antenna and the reflected wave of the transmitted wave reflected by the object in the N-1th frame, which is a predetermined time interval, The average A(N-1) of the product of the position X(N-1) of the object within a predetermined region of the voxel and the received intensity P(N-1), The variance V(N-1) of the product of the position X(N-1) of the object within the voxel and the received intensity P(N-1) is calculated. The above average A(N-1) and, The aforementioned variance V(N-1) and, The position X(N) of the object is obtained from the transmitted wave sent from the antenna and the reflected wave that the transmitted wave was reflected by the object in the Nth frame following the (N-1)th frame, A function V(N-1) representing the density of the point cloud within the voxel, Based on this, the evaluation function ρ(N) is calculated, An electronic device that estimates the position of the sensor based on the evaluation function ρ(N).

2. Let n be the number of points contained within the voxel. The point cloud contained within the voxel is x ｋ as, The received intensity of the point cloud contained within the voxel is p ｋ as, The ratio of the detection range of the antenna element to the voxel is V ｊ as, Let the currently scanned point cloud be given by the following equation (1): Let the above average A(N-1) be q in the following equation (2), Let the aforementioned variance V(N-1) be C in the following equation (3) (where T represents the transpose of the matrix), The electronic device according to claim 1, wherein the evaluation function ρ(N) is given by the following equation (4).

3. When there are n point clusters within the aforementioned voxel, ρ ｊ The electronic device according to claim 1, which estimates the position of the sensor using the maximum or minimum value of the value obtained by multiplying (N) by n points.

4. The electronic device according to claim 1, wherein the function V is a function that indicates the ratio of the number of detected objects in the voxel according to distance.

5. The electronic device according to claim 1, wherein the function V is a function that represents the ratio of the area or volume of the detection range of the antenna to the area or volume of the voxel.

6. The electronic device according to claim 1, wherein the function V is a function of the angle that can separate two objects.

7. The position X(N-1) of the object within the voxel, which is the predetermined region, The electronic device according to claim 1, which is calculated based on the result of performing a CFAR (Constant False Alarm Rate) process on the result of performing an FFT (Fast Fourier Transform) process on values ​​based on the transmitted wave and the reflected wave, and the result of estimating the direction of arrival for the reflected wave.

8. The position X(N-1) of the object within the voxel, which is the predetermined region, The electronic device according to claim 1, which is calculated based on the result of performing a CFAR (Constant False Alarm Rate) process on the result of performing an FFT (Fast Fourier Transform) process on the value based on the transmitted wave and the reflected wave.

9. The position X(N-1) of the object within the voxel, which is the predetermined region, The electronic device according to claim 1, calculated based on the result of performing an FFT (Fast Fourier Transform) operation on values ​​based on the transmitted wave and the reflected wave.

10. The electronic device according to claim 9, wherein a normal distribution is applied to the result of the FFT processing using the Gaussian Mixture Model (GMM) method to calculate the value after applying the normal distribution.

11. A control method for electronic equipment that estimates the position of a sensor equipped with an antenna, Let N be an integer greater than or equal to 1. Based on the position X(N-1) of the object and the received intensity P(N-1) based on the reflected wave, obtained from the transmitted wave transmitted from the antenna and the reflected wave of the transmitted wave reflected by the object in the N-1th frame, which is a predetermined time interval, The average A(N-1) of the product of the position X(N-1) of the object within a predetermined region of the voxel and the received intensity P(N-1), A step of calculating the variance V(N-1) of the product of the position X(N-1) of the object within the voxel and the received intensity P(N-1), The above average A(N-1) and, The aforementioned variance V(N-1) and, The position X(N) of the object is obtained from the transmitted wave sent from the antenna and the reflected wave that the transmitted wave was reflected by the object in the Nth frame following the (N-1)th frame, A function V(N-1) representing the density of the point cloud within the voxel, The steps include: calculating the evaluation function ρ(N) based on, The steps include: estimating the position of the sensor based on the evaluation function ρ(N); A control method including

12. In electronic devices that estimate the position of a sensor equipped with an antenna, Let N be an integer greater than or equal to 1. Based on the position X(N-1) of the object and the received intensity P(N-1) based on the reflected wave, obtained from the transmitted wave transmitted from the antenna and the reflected wave of the transmitted wave reflected by the object in the N-1th frame, which is a predetermined time interval, The average A(N-1) of the product of the position X(N-1) of the object within a predetermined region of the voxel and the received intensity P(N-1), A step of calculating the variance V(N-1) of the product of the position X(N-1) of the object within the voxel and the received intensity P(N-1), The above average A(N-1) and, The aforementioned variance V(N-1) and, The position X(N) of the object is obtained from the transmitted wave sent from the antenna and the reflected wave that the transmitted wave was reflected by the object in the Nth frame following the (N-1)th frame, A function V(N-1) representing the density of the point cloud within the voxel, The steps include: calculating the evaluation function ρ(N) based on, The steps include: estimating the position of the sensor based on the evaluation function ρ(N); A program that executes something.

Images