Information processing devices, information processing methods and program products
By generating calibration parameters and calculating the phase difference vector and the reference phase difference vector, the problem of insufficient sensor calibration accuracy was solved, achieving higher precision sensor calibration and improving the accuracy of the vehicle control system.
Patent Information
- Application Number
- CN202180075779.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-11-17
- Filing Date
- 2021-10-19
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-10-19
AI Technical Summary
Existing sensor calibration technologies lack sufficient calibration accuracy, making it difficult to meet the demands for high precision.
The calibration parameters are generated by the generation unit based on the phase difference vector and the reference phase difference vector. The phase difference vector is calculated using the sensing results of multiple receiving and transmitting antennas, thereby improving the calibration accuracy.
This improved the accuracy of sensor calibration, ensured accurate measurement of target positions, and enhanced the precision of the vehicle control system.
Smart Images

Figure CN116438470B_ABST
Abstract
Description
Technical Field
[0001] This technology relates to information processing devices, information processing methods, and procedures that can be applied to calibration. Background Technology
[0002] The calibration device described in Patent Document 1 is mounted on a vehicle and detects the relative position of a plurality of targets arranged at reference intervals. Changes in the relative positions of the targets are detected based on detection data detected during the vehicle's movement, using the reference intervals. This appropriately calibrates the sensors used to detect the relative positions of the targets relative to the vehicle (paragraphs
[0028] and
[0091] of Patent Document 1). Figure 1 wait).
[0003] Patent documents
[0004] Patent Document 1: Japanese Patent Application Publication No. 2020-26955 Summary of the Invention
[0005] Technical issues
[0006] In such sensor calibration, it is desirable to provide techniques that can improve calibration accuracy.
[0007] In view of the above, the purpose of this technology is to provide an information processing device, information processing method and program that can improve calibration accuracy.
[0008] Solution to the problem
[0009] To achieve the above objectives, the information processing apparatus according to embodiments of the present technology includes a generation unit.
[0010] The generation unit generates calibration parameters based on the phase difference vector and a reference phase difference vector used as a reference. The phase difference vector is calculated based on the sensing results, which are measured by a distance measurement sensor. The distance measurement sensor includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include a reference receiving antenna and one or more other receiving antennas. The phase difference vector is the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas.
[0011] In this information processing device, calibration parameters are generated based on the phase difference vector between the reference receiving antenna and each of one or more other receiving antennas, and a reference phase difference vector serving as a reference. This phase difference vector is calculated based on sensing results measured by a distance measurement sensor. This configuration can improve calibration accuracy.
[0012] The sensing results may include measurement data that serves as a target for calibration.
[0013] The information processing apparatus may further include an estimation unit that estimates the direction of arrival of the transmitted wave, which is transmitted from a transmitting antenna, reflected on a target, and received by at least one of a plurality of receiving antennas.
[0014] The generation unit can generate calibration parameters by projecting the phase difference vector onto the reference phase difference vector.
[0015] The information processing device may also include a computing unit that calculates the relative position between the target and each of the plurality of receiving antennas.
[0016] The computing unit can transform the relative position of each of the multiple receiving antennas onto the coordinate axes of the distance measurement sensor.
[0017] The information processing device may also include an execution unit that performs calibration based on calibration parameters.
[0018] Distance measurement sensors can be mounted on mobile devices.
[0019] The generation unit can generate calibration parameters based on the phase difference vector and the reference phase difference vector. The phase difference vector is calculated based on multiple sensing results of multiple targets measured by the distance measurement sensor.
[0020] The measurement data can be multiple measurements acquired when the distance measurement sensor and the target have different positional relationships.
[0021] The calibration parameters may include at least parameters related to the relative position and relative angle of the distance measurement sensor relative to the target being measured.
[0022] The calibration parameters can include at least six-axis parameters related to the relative position of the distance measurement sensor in three mutually orthogonal axial directions and the relative angles of the three axes.
[0023] Distance measurement sensors can transmit extremely high frequency (EHF) radio waves from a transmitting antenna.
[0024] The target can be stationary.
[0025] The relative position of the target with respect to the reference point used for calibration may be unknown.
[0026] The target can be at a distance from the distance measurement sensor, where aliasing occurs due to phase difference characteristics.
[0027] The information processing method according to embodiments of the present technology is an information processing method executed by a computer system, and includes:
[0028] By computer system:
[0029] Calibration parameters are generated based on the phase difference vector and a reference phase difference vector used as a reference. The phase difference vector is calculated based on sensing results measured by a distance measurement sensor, which includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include a reference receiving antenna and one or more other receiving antennas. The phase difference vector is the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas.
[0030] The procedure according to embodiments of the present technology causes a computer system to perform the following steps.
[0031] The steps for generating calibration parameters are based on a phase difference vector and a reference phase difference vector used as a reference. The phase difference vector is calculated based on sensing results measured by a distance measurement sensor, which includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include a reference receiving antenna and one or more other receiving antennas. The phase difference vector is the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas. Attached Figure Description
[0032] Figure 1 This is a block diagram illustrating an example configuration of a vehicle control system.
[0033] Figure 2 It is shown Figure 1 An example of the sensing area of an external identification sensor such as a camera, radar, LiDAR, and millimeter-wave radar.
[0034] Figure 3 This is a schematic diagram used to describe the overview of the calibration system.
[0035] Figure 4 This is a schematic diagram illustrating an example of the functional configuration of an information processing device.
[0036] Figure 5 This is a flowchart illustrating an example of generating calibration parameters.
[0037] Figure 6 This is a schematic diagram illustrating a calibration example.
[0038] Figure 7 This is a block diagram illustrating an example of the hardware configuration of an information processing device. Detailed Implementation
[0039] In the following description, embodiments according to the present technology will be described with reference to the accompanying drawings.
[0040] [Vehicle control system configuration example]
[0041] Figure 1This is a block diagram illustrating a configuration example of a vehicle control system 11 according to the present technology.
[0042] The vehicle control system 11 includes a vehicle control electronic control unit (ECU) 21, a communication unit 22, a map information storage unit 23, a global navigation satellite system (GNSS) receiving unit 24, an external identification sensor 25, an in-vehicle sensor 26, a vehicle sensor 27, a recording unit 28, a driver assistance and automatic driving control unit 29, a driver monitoring system (DMS) 30, a human-machine interface (HMI) 31, and a vehicle control unit 32.
[0043] The vehicle control ECU 21, communication unit 22, map information storage unit 23, GNSS receiving unit 24, external identification sensor 25, in-vehicle sensor 26, vehicle sensor 27, recording unit 28, driver assistance and automatic driving control unit 29, driver monitoring system (DMS) 30, human-machine interface (HMI) 31 and vehicle control unit 32 are connected to communicate with each other via communication network 41.
[0044] For example, communication network 41 consists of in-vehicle communication networks and buses compatible with digital two-way communication standards, such as Controller Area Network (CAN), Local Interconnect Network (LIN), Local Area Network (LAN), FlexRay (registered trademark), and Ethernet (registered trademark). Communication network 41 can be used depending on the type of data to be communicated. For example, CAN is used for data related to vehicle control, while Ethernet is used for large-capacity data.
[0045] It should be noted that the various units of the vehicle control system 11 can be directly connected via wireless communication (e.g., near field communication (NFC) and Bluetooth (registered trademark)) that assumes relatively short-range communication, without using the communication network 41.
[0046] It should be noted that, in the following text, when the various units of the vehicle control system 11 communicate via the communication network 41, the description of the communication network 41 will be omitted. For example, when the vehicle control ECU 21 and the communication unit 22 communicate via the communication network 41, it will be simply stated as: "The processor 21 and the communication unit 22 communicate."
[0047] The vehicle control ECU 21 consists of various processors, such as a central processing unit (CPU) and a microprocessor unit (MPU). The vehicle control ECU 21 controls all or some of the functions of the vehicle control system 11.
[0048] The communication unit 22 communicates with various devices inside and outside the vehicle, other vehicles, servers, base stations, etc., and sends and receives various types of data. At this time, the communication unit 22 can communicate by using multiple communication schemes.
[0049] The communication unit 22 can perform communication with the outside of the vehicle in an illustrative manner. According to wireless communication methods such as 5G, LTE, and Dedicated Short Range Communication (DSRC), the communication unit 22 communicates with a server (hereinafter referred to as an external server) in an external network via a base station or access point. The external network in which the communication unit 22 performs communication is, for example, the Internet, a cloud network, or a company-specific network. The communication scheme by which the communication unit 22 performs communication in the external network is not particularly limited, as long as it can perform digital two-way communication at a predetermined communication speed or higher and at a predetermined distance or greater.
[0050] Furthermore, for example, the communication unit 22 can communicate with a terminal located near its own vehicle using peer-to-peer (P2P) technology. The terminal located near the vehicle could be, for example, a terminal installed on a mobile object moving at a relatively low speed (such as a pedestrian or bicycle), a terminal installed in a fixed location in a shop, or a machine-type communication (MTC) terminal. Additionally, the communication unit 22 can also perform V2X communication.
[0051] V2X communication refers to communication between a vehicle and other objects, such as vehicle-to-vehicle communication with other vehicles, vehicle-to-infrastructure communication with roadside units, vehicle-to-home communication with houses, and vehicle-to-pedestrian communication with pedestrian-owned terminals.
[0052] For example, the communication unit 22 can receive programs (via air) from the outside for updating the software used to control the operation of the vehicle control system 11. The communication unit 22 can also receive map information, traffic information, and information about the surroundings of the vehicle 1 from the outside. Furthermore, for example, the communication unit 22 can send information about the vehicle 1 and information about the surroundings of the vehicle 1 to the outside.
[0053] Examples of information about vehicle 1 that the communication unit 22 sends to the outside include data indicating the status of vehicle 1, identification results from the identification unit 73, etc. Furthermore, for example, the communication unit 22 performs communications compatible with vehicle emergency alarm systems such as emergency call (eCall)
[0054] The communication unit 22 can perform communications with the interior of the vehicle, as illustrated below. For example, the communication unit 22 can communicate with each in-vehicle device using wireless communication. The communication unit 22 can perform wireless communication with in-vehicle devices via communication schemes (such as wireless LAN, Bluetooth, NFC, or wireless USB (WUSB)) that enable bidirectional digital communication at a predetermined or higher communication speed.
[0055] This technology is not limited to this, and the communication unit 22 can also communicate with each in-vehicle device via wired communication. For example, the communication unit 22 can communicate with each in-vehicle device via wired communication through a cable connected to a connection terminal (not shown). The communication unit 22 can communicate with each in-vehicle device via a communication scheme capable of digital bidirectional communication at a predetermined communication speed or higher via wired communication (such as Universal Serial Bus (USB), High Definition Multimedia Interface (HDMI) (registered trademark), and Mobile High Definition Link (MHL)).
[0056] Here, for example, in-vehicle devices refer to devices inside the vehicle that are not connected to communication network 41. Examples of in-vehicle devices include mobile and wearable devices owned by passengers (such as drivers), information devices carried and temporarily installed in the vehicle, etc.
[0057] For example, communication unit 22 receives electromagnetic waves transmitted by a vehicle information and communication system (VICS) (registered trademark), such as radio beacons, optical beacons, and FM multicast.
[0058] Map information storage unit 23 stores one or both of maps acquired from external sources and maps generated by vehicle 1. For example, map information storage unit 23 stores a three-dimensional high-precision map, a global map with lower precision than a high-precision map but covering a wider area than a high-precision map, etc.
[0059] High-precision maps include, for example, dynamic maps, point cloud maps, and vector maps. Dynamic maps, for example, include four layers of information: dynamic information, quasi-dynamic information, quasi-static information, and static information, and are provided to the vehicle from an external server.
[0060] A point cloud map is a map configured from point clouds (point cloud data). Here, a vector map refers to a map used by advanced driver assistance systems (ADAS), in which traffic information such as lane and signal locations is associated with the point cloud map.
[0061] For example, point cloud maps and vector maps can be provided from external servers, etc. Additionally, point cloud maps and vector maps can be generated at vehicle 1 based on sensing results from radar 52, LiDAR 53, etc., as maps for matching with the local map described later, and stored in map information storage unit 23. Furthermore, in the case of providing high-precision maps from external servers, for example, to reduce communication overhead, map data of several hundred square meters related to the planned route that vehicle 1 will travel can be obtained from the external server, etc.
[0062] The vehicle location information acquisition unit 24 receives GNSS signals from GNSS satellites and acquires the location information of vehicle 1. The received GNSS signals are provided to the driving assistance and automatic driving control unit 29. It should be noted that the vehicle location information acquisition unit 24 is not limited to the method of using GNSS signals, and the vehicle location information can be acquired, for example, by using beacons.
[0063] The external identification sensor 25 includes various sensors for identifying the external conditions of the vehicle 1, and provides sensor data from each sensor to various units of the vehicle control system 11. The external identification sensor 25 may include any type and any number of sensors.
[0064] For example, the external identification sensor 25 includes a camera 51, a radar 52, a laser imaging detection and ranging (LiDAR) 53, and a millimeter-wave radar 54. This technology is not limited to this, and the external identification sensor 25 can be configured to include one or more types of sensors selected from the camera 51, radar 52, LiDAR 53, and millimeter-wave radar 54. There is no particular limitation on the number of cameras 51, radars 52, LiDARs 53, and millimeter-wave radars 54, as long as they can actually be installed in the vehicle 1. Furthermore, the type of sensor in the external identification sensor 25 is not limited to this example, and the external identification sensor 25 can include other types of sensors such as ultrasonic sensors. Examples of the sensing areas of the various sensors in the external identification sensor 25 will be described later.
[0065] It should be noted that the imaging method of camera 51 is not particularly limited, as long as it can measure distance. For example, various cameras using different imaging methods, such as time-of-flight (ToF) cameras, stereo cameras, SLR cameras, and infrared cameras, can be used as camera 51 as needed. This technology is not limited to this, and camera 51 can be simply used to acquire captured images without being associated with distance measurement.
[0066] Furthermore, for example, the external identification sensor 25 may include an environmental sensor for detecting the environment surrounding the vehicle 1. The environmental sensor is a sensor used to detect environmental factors such as climate, weather, and brightness, and may include, for example, various sensors such as rain sensors, fog sensors, sunlight sensors, snow sensors, and illuminance sensors.
[0067] In addition, for example, the external identification sensor 25 includes a microphone for detecting, for example, sounds around the vehicle 1 and the location of sound sources.
[0068] The in-vehicle sensors 26 include various sensors for detecting in-vehicle information and provide sensor data from these sensors to various units of the vehicle control system 11. There are no particular limitations on the type and number of the various sensors in the in-vehicle sensors 26, as long as they can actually be installed in the vehicle 1.
[0069] For example, the in-vehicle sensor 26 may include one or more sensors, such as a camera, radar, seat sensor, steering wheel sensor, microphone, and biometric sensor. For instance, cameras using various imaging methods (such as ToF cameras, stereo cameras, SLR cameras, and infrared cameras) can be used as cameras for the in-vehicle sensor 26. This technology is not limited to this, and the camera of the in-vehicle sensor 26 can be simply used to acquire captured images without being associated with distance measurement. The biometric sensor of the in-vehicle sensor 26 may be disposed, for example, in the seat, steering wheel, etc., and detect various types of biometric information of passengers such as the driver.
[0070] Vehicle sensor 27 includes various sensors for detecting the state of vehicle 1, and provides sensor data from each sensor to various units of vehicle control system 11. There are no particular limitations on the type and number of sensors in vehicle sensor 27, as long as they can actually be installed in vehicle 1.
[0071] For example, vehicle sensor 27 includes a speed sensor, an acceleration sensor, an angular velocity sensor (gyroscope sensor), and an inertial measurement unit (IMU) integrating them. For example, vehicle sensor 27 includes a steering angle sensor for detecting the steering angle of the steering wheel, a yaw rate sensor, an accelerator sensor for detecting the amount of accelerator pedal operation, and a brake sensor for detecting the amount of brake pedal operation. For example, vehicle sensor 27 includes a rotation sensor for detecting the rotational speed of the engine and electric motor, a tire pressure sensor for detecting wheel air pressure, a slip rate sensor for detecting wheel slip ratio, and a wheel speed sensor for detecting wheel rotational speed. For example, vehicle sensor 27 includes a battery sensor for detecting the remaining battery level and temperature, and an impact sensor for detecting impacts from external sources.
[0072] Recording unit 28 includes at least one of non-volatile storage medium and volatile storage medium, and stores data and programs. For example, recording unit 28 is used as electrically erasable programmable read-only memory (EEPROM) and random access memory (RAM). Magnetic storage devices such as hard disk drives (HDDs), semiconductor storage devices, optical storage devices, and magneto-optical storage devices can be used as storage media. Recording unit 28 records various types of programs and data used by various units of vehicle control system 11. For example, recording unit 28 includes an event data recorder (EDR) and an automated driving data storage system (DSSAD), and records information about vehicle 1 before and after events such as accidents and biometric information acquired by in-vehicle sensors 26.
[0073] The driving assistance and automatic driving control unit 29 controls the driving assistance and automatic driving of the vehicle 1. For example, the driving assistance and automatic driving control unit 29 includes an analysis unit 61, an action planning unit 62, and an operation control unit 63.
[0074] The analysis unit 61 performs analysis and processing of the vehicle 1 and its surrounding conditions. The analysis unit 61 includes a self-position estimation unit 71, a sensor fusion unit 72, and an identification unit 73.
[0075] The self-position estimation unit 71 estimates the self-position of vehicle 1 based on sensor data from external identification sensor 25 and a high-precision map stored in map information storage unit 23. For example, the self-position estimation unit 71 generates a local map based on sensor data from external identification sensor 25, performs matching between the local map and the high-precision map, and estimates the self-position of vehicle 1. For example, the position of vehicle 1 is based on the center of the vehicle axle of a pair of rear wheels.
[0076] For example, a local map is a high-precision 3D map or an occupancy grid map generated using techniques such as Simultaneous Localization and Mapping (SLAM). For example, a high-precision 3D map is a point cloud map as described above. An occupancy grid map is a map that shows the occupancy status of an object on a grid-by-grid basis, where the 3D or 2D space around vehicle 1 is divided into grids of a predetermined size. For example, the occupancy status of an object is indicated by the object's presence / absence or probability of presence. For example, the local map is also used for the detection and recognition processing of the external condition of vehicle 1 performed by the recognition unit 73.
[0077] It should be noted that the self-position estimation unit 71 can estimate the self-position of vehicle 1 based on GNSS signals and sensor data from vehicle sensor 27.
[0078] The sensor fusion unit 72 performs sensor fusion processing to obtain new information by combining data from multiple different types of sensors (e.g., image data from camera 51 and sensor data from radar 52). Methods for combining different types of sensor data include integration, fusion, and joint processing.
[0079] The identification unit 73 performs detection processing of the external condition of the vehicle 1 and identification processing of the external condition of the vehicle 1.
[0080] For example, the identification unit 73 performs detection and identification processing on the external condition of the vehicle 1 based on information from the external identification sensor 25, information from its own position estimation unit 71, and information from the sensor fusion unit 72.
[0081] Specifically, for example, the recognition unit 73 performs detection and recognition processing on objects around the vehicle 1. For example, object detection processing involves detecting the presence / absence, size, shape, position, movement, etc., of an object. For example, object recognition processing involves recognizing the attributes of an object (such as type) or identifying a specific object. It should be noted that detection and recognition processing are not necessarily clearly separated and may overlap.
[0082] For example, the identification unit 73 detects objects around the vehicle 1 by clustering point clouds based on sensor data such as LiDAR 53, radar 52, etc., classifying them by each point cloud. Therefore, it detects the presence / absence, size, shape, and location of objects around the vehicle 1.
[0083] For example, the recognition unit 73 detects the movement of objects around vehicle 1 by tracking the movement of each point cloud classified by clustering. Therefore, the speed and direction of travel (movement vector) of objects around vehicle 1 are detected.
[0084] For example, the recognition unit 73 detects or recognizes vehicles, people, bicycles, obstacles, structures, roads, traffic lights, traffic signs, road signs, etc., based on image data provided from the camera 51. Furthermore, the types of objects surrounding the vehicle 1 can be identified through recognition processing such as semantic segmentation.
[0085] For example, the identification unit 73 can perform identification processing on traffic control around vehicle 1 based on the map stored in the map information storage unit 23, the estimation result of its own position by the self-position estimation unit 71, and the identification result of objects around vehicle 1 by the identification unit 73. Through this processing, the identification unit 73 can identify the location and status of signals, the content of traffic signs and road signs, the content of traffic control, and the lanes that are allowed to travel, etc.
[0086] For example, the identification unit 73 can perform identification processing of the surrounding environment of the vehicle 1. Climate, temperature, humidity, brightness, road surface conditions, etc. are assumed to be the surrounding environment to be identified by the identification unit 73.
[0087] Action planning unit 62 generates an action plan for vehicle 1. For example, action planning unit 62 performs path planning and path following processing to generate an action plan.
[0088] It should be noted that path planning (global path planning) is the process of planning a coarse path from the starting point to the destination. This path planning also includes trajectory generation (local path planning) processing, which is called trajectory planning, and enables safe and smooth travel near vehicle 1 on the path planned by path planning, taking into account the motion characteristics of vehicle 1. Path planning can be classified as long-term path planning, while trajectory generation can be classified as short-term path planning or local path planning. Safety-first path representation is a concept similar to trajectory generation, short-term path planning, or local path planning.
[0089] Path following is a process of planning operations to ensure safe and correct travel along a path planned by path planning within a planned time. For example, the action planning unit 62 can calculate the target speed and target angular velocity of vehicle 1 based on the results of this path following process.
[0090] The operation control unit 63 controls the operation of the vehicle 1 in order to implement the action plan generated by the action planning unit 62.
[0091] For example, the operation control unit 63 controls the steering control unit 81, braking control unit 82, and drive control unit 83 included in the vehicle control unit 32, which will be described later, and performs acceleration / deceleration control and directional control so that the vehicle 1 travels along a trajectory calculated through trajectory planning. For example, the operation control unit 63 performs cooperative control for implementing ADAS functions such as collision avoidance or impact mitigation, cruise control, constant speed driving, collision warning for the vehicle itself, and lane departure warning for the vehicle itself. For example, the operation control unit 63 performs cooperative control for autonomous driving, etc., which autonomously performs driving without driver intervention.
[0092] The DMS 30 performs driver authentication processing, driver condition recognition processing, etc., based on sensor data from the in-vehicle sensor 26 and input data input to the HMI 31, which will be described later. In this case, for example, health status, alertness, attention level, fatigue level, gaze direction, alcohol level, driving operation, posture, etc., are assumed to be driver conditions to be recognized by the DMS 30.
[0093] It should be noted that DMS 30 can perform authentication processing for passengers other than the driver and identification processing for passenger conditions. Furthermore, for example, DMS 30 can perform in-vehicle condition identification processing based on sensor data from in-vehicle sensors 26. For example, temperature, humidity, brightness, and odor are assumed to be in-vehicle conditions to be identified.
[0094] HMI 31 inputs various types of data and commands, and presents various types of data to the driver and others.
[0095] The data input by HMI 31 will be illustrated schematically. HMI 31 includes input devices for human input of data. Based on the data, instructions, etc., input through the input devices, HMI 31 generates input signals and provides these input signals to the various units of the vehicle control system 11. For example, as input devices, HMI 31 includes operating elements such as touch panels, buttons, switches, and joysticks. This technology is not limited to this, and HMI 31 may also include input devices capable of inputting information through methods other than manual operation (e.g., voice, gestures, etc.). Furthermore, as input devices, for example, HMI 31 can use external connection devices, such as remote control devices using infrared or radio waves, and mobile or wearable devices suitable for the operation of the vehicle control system 11.
[0096] The data presentation of HMI 31 will be illustrated schematically. HMI 31 generates visual, auditory, and tactile information related to the passenger or the exterior of the vehicle. Furthermore, HMI 31 performs output control to control the output, content, timing, and method of each of these types of information. As visual information, HMI 31 generates and outputs information displayed as images or light, such as operating screens, vehicle 1 status displays, warning displays, and monitoring images showing the surroundings of vehicle 1. As auditory information, HMI 31 generates and outputs information displayed as sound, such as audio guidance, warning sounds, and warning messages. Furthermore, as tactile information, HMI 31 generates and outputs information that provides the passenger with a tactile sensation, such as force, vibration, and motion.
[0097] For example, as the output device for visual information output by HMI 31, a display device that presents visual information by displaying images on its own or a projector device that presents visual information by projecting images can be used. It should be noted that, in addition to display devices with conventional displays, the display device can be a device that displays visual information within the passenger's field of vision, such as a head-up display, a perspective display, and a wearable device with augmented reality (AR) functionality. Furthermore, as the output device for visual information output, HMI 31 can use display devices such as navigation devices, instrument panels, camera monitoring systems (CMS), electronic mirrors, lights, etc., installed in vehicle 1.
[0098] For example, as an output device for HMI 31 to output auditory information, an audio speaker, headphones, or in-ear headphones can be used.
[0099] For example, as an output device for HMI 31 that outputs tactile information, tactile elements using tactile technology can be applied. For example, tactile elements can be placed in parts of vehicle 1 that passengers contact, such as the steering wheel and seats.
[0100] The vehicle control unit 32 controls various units of the vehicle 1. The vehicle control unit 32 includes a steering control unit 81, a brake control unit 82, a drive control unit 83, a body system control unit 84, a lighting control unit 85, and a horn control unit 86.
[0101] For example, the steering control unit 81 detects and controls the state of the steering system of vehicle 1. The steering system includes a steering mechanism, electric power steering, etc., including, for example, a steering wheel. The steering control unit 81 includes a control unit (e.g., ECU) that controls, for example, the steering system, and actuators that drive the steering system.
[0102] For example, the brake control unit 82 detects and controls the state of the braking system of vehicle 1. The braking system includes, for example, a braking mechanism, an anti-lock braking system (ABS), a regenerative braking mechanism, and a brake pedal. The brake control unit 82 includes a control unit, such as an ECU, that controls the braking system.
[0103] For example, the drive control unit 83 detects and controls the state of the drive system of the vehicle 1. The drive system includes, for example, an accelerator pedal, a drive force generating device for generating drive force for an internal combustion engine, drive motor, etc., and a drive force transmission mechanism for transmitting the drive force to the wheels. The drive control unit 83 includes a control unit, such as an ECU, that controls, for example, the braking system.
[0104] For example, the body system control unit 84 detects and controls the state of the body systems of vehicle 1. These body systems may include, for example, a keyless entry system, a smart key system, power windows, power seats, air conditioning, airbags, seat belts, a gear shift mechanism, etc. The body system control unit 84 includes control units that control, for example, the body systems, such as an ECU.
[0105] For example, the lighting control unit 85 detects and controls the status of various lights in the vehicle 1. For example, headlights, taillights, fog lights, turn signals, brake lights, projectors, bumper displays, etc., are assumed to be lights to be controlled. The lighting control unit 85 includes control units that control the lights, such as an ECU.
[0106] For example, the horn control unit 86 detects and controls the state of the vehicle horn of vehicle 1. The horn control unit 86 includes a control unit, such as an ECU, that controls, for example, the vehicle horn.
[0107] Figure 2 It is shown Figure 1 A diagram illustrating the exemplary sensing areas of the external identification sensor 25, including camera 51, radar 52, LiDAR 53, and millimeter-wave radar 54.
[0108] It should be noted that, Figure 2 The diagram schematically shows the state of vehicle 1 as viewed from above, with the left side being the front (front) side of vehicle 1 and the right side being the rear (rear) side of vehicle 1.
[0109] Sensing regions 101F and 101B illustrate exemplary sensing regions of the millimeter-wave radar 54. Sensing region 101F utilizes multiple millimeter-wave radars 54 to cover the periphery of the front end of the vehicle 1. Sensing region 101B utilizes multiple millimeter-wave radars 54 to cover the periphery of the rear end of the vehicle 1.
[0110] For example, the sensing results of sensing areas 101F and 101B are used for parking assistance of vehicle 1.
[0111] Sensing areas 102F to 102B illustrate examples of the sensing areas of radar 52 for short or medium range. Sensing area 102F covers an area further in front of vehicle 1 than sensing area 101F. Sensing area 102B covers an area further in front of vehicle 1 than sensing area 101B. Sensing area 102L covers the rear perimeter of the left side of vehicle 1. Sensing area 102R covers the rear perimeter of the right side of vehicle 1.
[0112] For example, the sensing results of sensing area 102F are used to detect vehicles, pedestrians, etc. located in front of vehicle 1. For example, the sensing results of sensing area 102B are used for the function of preventing rear-end collisions of vehicle 1. For example, the detection results of detection areas 102L and 102R are used to detect objects in the blind spot on the side of vehicle 1.
[0113] Sensing areas 103F to 103B illustrate exemplary sensing areas of camera 51. Sensing area 103F covers an area further in front of vehicle 1 than sensing area 102F. Sensing area 103B covers an area further in front of vehicle 1 than sensing area 101B. Sensing area 103L covers the perimeter of the left side of vehicle 1. Sensing area 103R covers the perimeter of the right side of vehicle 1.
[0114] For example, the sensing results of sensing area 103F can be used for traffic light and traffic sign recognition, lane departure prevention assist systems, or automatic headlight control systems. For example, the sensing results of sensing area 103B can be used for parking assistance and surround view systems. For example, the sensing results of sensing areas 103L and 103R can be used for surround view systems.
[0115] Sensing region 104 illustrates an exemplary sensing region of LiDAR 53. Sensing region 104 covers an area further in front of vehicle 1 than sensing region 103F. On the other hand, sensing region 104 has a narrower area in the left-right direction than sensing region 103F.
[0116] For example, the sensing results of sensing area 104 are used to detect objects such as surrounding vehicles.
[0117] Sensing area 105 illustrates an exemplary sensing area for radar 52 used for long-range applications.
[0118] The sensing area 105 covers a greater area in front of the vehicle 1 than the sensing area 104. On the other hand, the sensing area 105 has a narrower area in the left-right direction than the sensing area 104.
[0119] For example, the sensing results of sensing area 105 are used for adaptive cruise control (ACC), emergency braking, collision avoidance, etc.
[0120] It should be noted that the sensing areas of the various sensors of the external identification sensor 25, including camera 51, radar 52, LiDAR 53, and millimeter-wave radar 54, can be used in addition to... Figure 2In addition to the above configuration, there are various other configurations. Specifically, the millimeter-wave radar 54 can sense the side of vehicle 1, or the LiDAR 53 can sense the rear of vehicle 1. Furthermore, the mounting location of each sensor is not limited to each example above. Moreover, a single sensor or multiple sensors can be provided for each type of sensor.
[0121] [Configuration example of a calibration system]
[0122] Figure 3 This is a schematic diagram used to describe the outline of the calibration system 110.
[0123] like Figure 3 As shown in Figure A, the calibration system 110 includes a vehicle 1 and an information processing device 120. The vehicle 1 and the information processing device 120 are connected to enable them to communicate with each other via wired or wireless means. The form of connection between the devices is not limited, and for example, wireless LAN communication such as Wi-Fi or near-field communication such as Bluetooth (registered trademark) can be used.
[0124] In this embodiment, the millimeter-wave radar 54 of the external identification sensor 25 is used as a distance measurement sensor installed on the vehicle 1.
[0125] The millimeter-wave radar 54 is capable of transmitting extremely high frequency (EHF) radio waves toward a target and measuring the distance to the target based on the time taken to receive the radio waves reflected from and returned by the target. In this embodiment, two millimeter-wave radars 54 are mounted on the front (front) side of the vehicle 1 and measure the distance to the target 115, which serves as a calibration reference. Hereinafter, one of the millimeter-wave radars 54 mounted on the vehicle 1 will be referred to as millimeter-wave radar 54R, and the other will be referred to as millimeter-wave radar 54L.
[0126] It should be noted that in this embodiment, the parameters to be calibrated are six-axis parameters that combine the relative positions of the external identification sensor 25 in three mutually orthogonal axial directions (XYZ directions) with the relative angles of the three axes (e.g., pitch, yaw, and roll). Furthermore, in this embodiment, the millimeter-wave radar 54L is set as the calibration target.
[0127] Furthermore, in this embodiment, target 115 is in a stationary state, that is, its position information is known. The stationary state is a state in which the position (coordinates) remains unchanged during calibration.
[0128] Position information refers to the relative position between target 115 and a reference point used for calibration. For example, target 115 and the reference point are fixed using a clamp, or the relative position information is measured by a range measuring device other than the range measuring sensor described above. For example, the reference point used for calibration is the position of a radar among multiple radars that serves as a reference.
[0129] It should be noted that the shape of target 115 is not limited and any shape can be used, as long as the sensor can acquire distance measurement data.
[0130] Figure 3 B is a schematic diagram showing the configuration of the millimeter-wave radar 54.
[0131] like Figure 3 As shown in Figure B, the millimeter-wave radar 54R includes a transmitting antenna Tx1 and four receiving antennas Rx11, Rx12, Rx13, and Rx14. Furthermore, the millimeter-wave radar 54L includes a transmitting antenna Tx2 and four receiving antennas Rx21, Rx22, Rx23, and Rx24.
[0132] In this embodiment, the transmitting antenna and four receiving antennas are arranged in a straight line.
[0133] For example, millimeter waves transmitted from transmitting antenna Tx1 are reflected on target 115 and received by four receiving antennas Rx11, Rx12, Rx13, and Rx14. The sensing results are measured in this manner. In this embodiment, the sensing results measured by millimeter-wave radar 54 are provided to information processing device 120.
[0134] The information processing device 120 generates calibration parameters based on the phase difference vector between the reference receiving antenna and each other receiving antenna except the reference receiving antenna, and the reference phase difference vector.
[0135] In this embodiment, receiving antenna Rx11 is designated as the reference receiving antenna among receiving antennas Rx11, Rx12, Rx13, and Rx14. Therefore, receiving antennas Rx11, Rx12, Rx13, and Rx14 correspond to one or more other receiving antennas.
[0136] The phase difference vector is a vector indicating the phase difference between the reflected waves received by each receiving antenna and reflected at target 115. That is, the phase difference vector is a vector representing the change in phase difference between the reflected wave received by receiving antenna Rx11, which serves as the reference receiving antenna, and the reflected waves received by each of receiving antennas Rx12, Rx13, and Rx14.
[0137] The reference phase difference vector is a vector indicating the change in the phase difference of the reflected wave received by each receiving antenna under ideal conditions. Ideal conditions include a state in which there is no external interference such as rain and wind around the vehicle 1 and a state in which the position of the millimeter-wave radar 54 is not shifted due to faults, etc.
[0138] Figure 4 This is a schematic diagram illustrating an example of the functional configuration of the information processing device 120.
[0139] Information processing device 120 has the hardware required for a computer configuration, which includes, for example, processors such as CPU, GPU and DSP, memories such as ROM and RAM, and storage devices such as HDD (see...). Figure 7 For example, the CPU loads a program according to the present technology, which is pre-stored in ROM or the like, into RAM and executes the program. In this way, the information processing method according to the present technology is executed.
[0140] For example, any computer, such as a PC, can be used to implement the information processing device 120. Of course, hardware such as field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs) can also be used.
[0141] In this embodiment, the parameter generation unit, which is a function block, is configured by the CPU executing a predetermined program. Of course, the function block can be implemented using dedicated hardware such as an integrated circuit (IC).
[0142] For example, the program can be installed in the information processing device 120 via various recording media. Alternatively, the program can be installed via the Internet or the like.
[0143] The type of recording medium used for recording the program is not limited, and any computer-readable recording medium can be used. For example, any computer-readable, non-transient storage medium can be used.
[0144] like Figure 4 As shown, the information processing device 120 includes an information acquisition unit 121, a target detection unit 122, a distance calculation unit 123, a vector calculation unit 124, a relative position calculation unit 125, a relative angle estimation unit 126, a parameter generation unit 127, and a calibration execution unit 128.
[0145] The information acquisition unit 121 acquires various types of information. In this embodiment, the information acquisition unit 121 acquires the surrounding information of the vehicle 1 measured by the external identification sensor 25 as a sensing result.
[0146] Furthermore, in this embodiment, the information acquired by the information acquisition unit 121 is provided to the target detection unit 122 and the distance calculation unit 123.
[0147] The target detection unit 122 detects a target 115 as a calibration reference. In this embodiment, the target detection unit 122 detects objects suitable for the calibration reference from surrounding objects measured by the external identification sensor 25 as targets. For example, a triangular marker with vertices having a pre-known relative distance is detected as a target.
[0148] Furthermore, in this embodiment, information about the target detected by the target detection unit 124 is provided to the distance calculation unit 123.
[0149] The distance calculation unit 123 calculates the distance between the target and each millimeter-wave radar 54. In this embodiment, the distance calculation unit 123 calculates the distance to the target 115 based on the target 115 detected by the target detection unit 122 and the sensing results acquired by the information acquisition unit 121. For example, the distance calculation unit 123 calculates the distance to the target 115 based on the time taken for the receiving antenna to receive the millimeter waves transmitted from the transmitting antenna.
[0150] Furthermore, in this embodiment, the information calculated by the distance calculation unit 123 is provided to the vector calculation unit 124.
[0151] The vector calculation unit 124 calculates the phase difference vector. In this embodiment, the vector calculation unit 124 calculates the phase difference vector between the reference receiving antenna and each receiving antenna.
[0152] Furthermore, the vector calculation unit 124 calculates the reference phase difference vector between the reference receiving antenna and each receiving antenna under ideal conditions. It should be noted that the reference phase difference vector can be prepared in advance based on the positional relationship, type, and performance of the millimeter-wave radar 54 mounted on the vehicle 1. For example, the reference phase difference vector corresponding to the millimeter-wave radar 54 mounted on the vehicle 1 can be obtained based on the phase difference acquired by the information acquisition unit 121.
[0153] In addition, the vector calculation unit 124 projects the calculated phase difference vector onto the reference phase difference vector.
[0154] The relative position calculation unit 125 calculates the relative position between the target 115 and each receiving antenna of the millimeter-wave radar 54. In this embodiment, the relative position calculation unit 125 calculates the relative position between the receiving antenna and the target 115 based on the calculation results calculated by the vector calculation unit 124.
[0155] Furthermore, the relative position calculation unit 125 transforms the calculated positional relationships between the various receiving antennas onto the coordinate axes of the millimeter-wave radar 54. It should be noted that the positional relationships include the relative position and relative angle of each receiving antenna.
[0156] Furthermore, in this embodiment, the relative position between the target 115 and each receiving antenna calculated by the relative position calculation unit 125 is provided to the parameter generation unit 127.
[0157] The relative angle estimation unit 126 estimates the direction of arrival of the millimeter wave transmitted from the transmitting antenna, reflected off the target 115, and received by the receiving antenna. In this embodiment, the relative angle estimation unit 126 estimates the relative angle between the receiving antenna and the target 115 based on the calculation results calculated by the vector calculation unit 124.
[0158] Furthermore, in this embodiment, the relative angle between the target 115 and each receiving antenna estimated by the relative angle estimation unit 126 is provided to the parameter generation unit 127.
[0159] The parameter generation unit 127 generates calibration parameters required for the calibration of the millimeter-wave radar 54, which serves as a distance measurement sensor. In this embodiment, the parameter generation unit 127 generates calibration parameters based on the phase difference vector between the reference receiving antenna and each other receiving antenna besides the reference receiving antenna, and a reference phase difference vector, which is calculated based on the sensing results measured by the millimeter-wave radar 54.
[0160] The calibration execution unit 128 performs calibration based on calibration parameters. In this embodiment, the calibration execution unit 128 calibrates the millimeter-wave radars 54R and 54L based on the calibration parameters generated by the parameter generation unit 127.
[0161] It should be noted that in this embodiment, parameter generation unit 127 corresponds to generation unit, which generates calibration parameters based on phase difference vector and reference phase difference vector as a reference. The phase difference vector is calculated based on sensing results measured by a distance measurement sensor, which includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include a reference receiving antenna and one or more other receiving antennas. The phase difference vector is the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas.
[0162] It should be noted that in this embodiment, the relative angle estimation unit 126 corresponds to the estimation unit, which estimates the direction of arrival of the transmitted wave, which is transmitted from the transmitting antenna, reflected on the target, and received by at least one of a plurality of receiving antennas.
[0163] It should be noted that in this embodiment, the relative position calculation unit 125 corresponds to the calculation unit, which calculates the relative position between the target and each of the plurality of receiving antennas.
[0164] It should be noted that in this embodiment, the calibration execution unit 128 corresponds to the execution unit, which performs calibration based on calibration parameters.
[0165] It should be noted that in this embodiment, vehicle 1 corresponds to a mobile device, and the distance measurement sensor is mounted on the mobile device.
[0166] Figure 5 This is a flowchart illustrating an example of generating calibration parameters.
[0167] The information acquisition unit 121 acquires a rough estimate of the relative position and relative angle between the transmitting antenna and each receiving antenna (step 101). For example, when designing or manufacturing vehicle 1, the information acquisition unit 121 acquires the relative position and relative angle between the transmitting antenna and each receiving antenna.
[0168] In addition, the information acquisition unit 121 acquires the sensing results of the target 115 measured by the millimeter-wave radar 54 (step 102).
[0169] The distance calculation unit 123 calculates the distance to the target 115 based on the sensing results (step 103).
[0170] Vector calculation unit 124 calculates the phase difference vector between the reference receiving antenna and each receiving antenna (step 104). It should be noted that the phase difference vector may not be uniquely determined based on the relative angle between the target 115 and the millimeter-wave radar 54. In this case, vector calculation unit 124 uniquely determines the phase difference vector by using a coarse estimate of the relative position between the transmitting antenna and each receiving antenna obtained in step 101 as a condition for the relative angle relative to the target 115. For example, in the cases of phase differences of 10 degrees and 370 degrees, the 370-degree phase difference vector is excluded from the coarse estimate of the relative position between the transmitting antenna and each receiving antenna, and the 10-degree phase difference vector is calculated.
[0171] In addition, the vector calculation unit 124 projects the phase difference vector onto the reference phase difference vector (step 105).
[0172] The relative position calculation unit 125 calculates the relative position between the target 115 and each receiving antenna, and the relative angle estimation unit 126 estimates the relative angle (step 106).
[0173] The relative position calculation unit 125 transforms the calculated positional relationship between each receiving antenna onto the coordinate axis of the millimeter-wave radar 54 (step 107).
[0174] The parameter generation unit 127 generates calibration parameters based on the calculated relative position and relative angle of each receiving antenna (step 108). The calibration execution unit 128 calibrates the millimeter-wave radar 54 based on the generated calibration parameters.
[0175] The above steps complete the calibration of the millimeter-wave radar 54L. Additionally, steps 101 to 108 complete the calibration of the millimeter-wave radar 54R.
[0176] For example, if three or more millimeter-wave radars 54 are provided, steps 101 to 108 are performed until the calibration of each millimeter-wave radar 54 is completed. Furthermore, in this case, arithmetic processing such as an arithmetic mean can be performed on the individual generated calibration parameters.
[0177] [Calibration Example]
[0178] Figure 6 This is a schematic diagram illustrating a calibration example.
[0179] exist Figure 6 In this process, the position of the target road cone was changed four times, and calibration was performed each time the road cone's position was changed. That is, assuming that the millimeter-wave radar 54 first calibrated the road cone 131 placed at the left end (solid line), and the millimeter-wave radar 54 second calibrated the road cone 132 placed at the adjacent dashed line position. In addition, road cone 133 (and road cone 134) were placed at the adjacent dashed line positions. The calibration was performed a total of four times.
[0180] In this embodiment, the transmitting and receiving antennas constituting the millimeter-wave radars 54R and 54L are at a sufficient distance relative to the detected wavelength in order to generate a phase difference between them.
[0181] Furthermore, in this embodiment, it is assumed that calibration is performed during shipment from the factory or during maintenance. That is, calibration is performed when vehicle 1 is not moving, i.e., when vehicle 1 is stationary.
[0182] The millimeter-wave radar 54 transmits millimeter waves toward the road cone 131 and receives the reflected waves reflected off the road cone 131.
[0183] Based on the sensing results of the received road cones 131, the vector calculation unit 124 calculates the phase difference vector between the reference receiving antenna and each receiving antenna.
[0184] Furthermore, the vector calculation unit 124 calculates the reference phase difference vector. For example, the phase difference vector when the positional relationship between the various receiving antennas arranged in the vehicle 1 is as designed is calculated as the reference phase difference vector.
[0185] In addition, the vector calculation unit 124 projects the calculated phase difference vector onto the reference phase difference vector.
[0186] The relative position calculation unit 125 calculates the relative position between the road cone 131 and each receiving antenna, and the relative angle estimation unit 126 estimates the relative angle.
[0187] In addition, the relative position calculation unit 125 transforms the calculated relative position and relative angle of each receiving antenna onto the coordinate axis of the millimeter-wave radar 54.
[0188] The parameter generation unit 127 generates calibration parameters based on the calculated relative position and relative angle of each receiving antenna.
[0189] The calibration execution unit 128 calibrates the millimeter-wave radar 54 based on the generated calibration parameters.
[0190] This process completes the first calibration. Performing a second, third, and fourth calibration similarly can improve calibration accuracy. That is, it ensures the accurate acquisition of sensing results of the target object located outside the vehicle 1 and measured by the external identification sensor 25.
[0191] In the preceding text, the information processing apparatus 120 according to this embodiment generates calibration parameters based on the phase difference vector between the reference receiving antenna Rx11 and each receiving antenna, and a reference phase difference vector. This phase difference vector is calculated based on sensing results measured by a millimeter-wave radar 54, which includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include the reference receiving antenna Rx11 and receiving antennas Rx11, Rx12, Rx13, and Rx14. Therefore, calibration accuracy can be improved.
[0192] Traditionally, during calibration, a point target with a known location is required, and a sufficient distance is needed between the radar and the target to avoid aliasing due to phase difference characteristics.
[0193] Furthermore, techniques are being considered to combine information from multiple vehicle-mounted millimeter-wave radars to achieve higher accuracy than a single millimeter-wave radar. However, the accuracy of the combined information is low if the distances and orientations between the sensors in the combined sensor information are inaccurate. As a method to correct information about the mutual positions between sensors, estimated angles relative to a point target are used. However, the distance between the point target and the sensor is required. Moreover, the accuracy of the mutual positions between the sensors depends on the accuracy of the angle estimation for each sensor.
[0194] Therefore, this technique generates calibration parameters based on the phase difference vector between the reference receiving antenna and each receiving antenna, and a reference phase difference vector calculated based on sensing results measured by the range measurement sensor. Furthermore, multiple target positions can be set to generate multiple phase differences between the target and each millimeter-wave radar, and then the target position is uniquely determined. Thus, even when the radar and target have aliasing due to phase difference characteristics, a target with known position information can be used as a calibration target, and calibration can be performed between millimeter-wave radars.
[0195] <Other Embodiments>
[0196] This technology is not limited to the above embodiments, and various other embodiments can be implemented.
[0197] In the above embodiment, the millimeter-wave radar 54 is the calibration target. This technology is not limited to this, and various sensors can be calibration targets. For example, an ultrasonic sensor capable of determining the direction of arrival of a reflected wave using multiple sensors can be a calibration target.
[0198] In the above embodiments, the information processing device performs calibration. This technology is not limited to this, and the information processing device can be installed on vehicle 1.
[0199] In the above embodiments, the vehicle-mounted sensor is the calibration target. This technology is not limited to this, and it can also be a sensor installed in a wall such as a ceiling, or a sensor installed on an autonomous mobile robot, etc.
[0200] In the above embodiment, the transmitting and receiving antennas constituting the millimeter-wave radar 54 are arranged in a straight line. This technology is not limited to this, and the transmitting and receiving antennas can be arranged arbitrarily. For example, the receiving antennas can be arranged at uniform intervals around the transmitting antenna as the center on a plane.
[0201] In the above embodiments, calibration is performed multiple times while the target position is changed. This technology is not limited to this, and calibration can be performed in a single operation with high accuracy. For example, the information processing device may include a determining unit that determines the calibration accuracy and can determine whether the calibration accuracy is above a threshold each time calibration is performed.
[0202] Figure 7 This is a block diagram illustrating an example of the hardware configuration of the information processing device 120.
[0203] The information processing device 120 includes a CPU 141, a ROM 142, a RAM 143, an input / output interface 145, and a bus 144 for connecting them to each other. A display unit 146, an input unit 147, a storage unit 148, a communication unit 149, a drive unit 150, etc., are connected to the input / output interface 145.
[0204] For example, display unit 146 is a display device using liquid crystal, electroluminescent (EL), etc. For example, input unit 147 is a keyboard, pointing device, touch panel, or other operating device. If input unit 147 includes a touch panel, the touch panel can be integrated with display unit 146.
[0205] Storage unit 148 is a non-volatile storage device. For example, storage unit 148 is an HDD, flash memory, or other solid-state storage. For example, drive unit 150 is a device capable of driving a removable recording medium 151 such as an optical recording medium or a magnetic recording tape.
[0206] Communication unit 149 is a modem, router, or other communication device that can be connected to a LAN, WAN, etc., for communicating with other devices. Communication unit 149 can perform wired communication or wireless communication. Communication unit 149 is often used separately from information processing device 20.
[0207] The information processing device 20, equipped with the aforementioned hardware configuration, achieves information processing through the cooperation of software stored in storage unit 148, ROM 142, etc. Specifically, loading and executing programs stored in ROM 142 and other memory for configuring software into RAM 143 implements the information processing method according to this technology.
[0208] For example, the program is installed in the information processing device 20 via recording medium 151. Alternatively, the program can be installed in the information processing device 20 via a global network or the like. Furthermore, any computer-readable non-transitory storage medium can be used.
[0209] A computer installed on a communication terminal, in cooperation with other computers that can communicate with it via a network or the like, can execute information processing apparatus, information processing methods and programs according to the present technology, and construct an information processing apparatus according to the present invention.
[0210] In other words, the information processing apparatus, information processing method, and program according to this technology can be executed not only in a computer system configured with a single computer, but also in a computer system in which multiple computers operate in cooperation with each other. It should be noted that in this disclosure, a system means a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are housed in the same housing. Therefore, multiple devices housed in a single housing and connected via a network, and devices having multiple modules housed in a single housing, are both systems.
[0211] The execution of information processing apparatus, information processing methods, and programs according to this technology by a computer system includes, for example, cases where a single computer performs relative position calculations, phase difference vector calculations, calibration parameter generation, etc., as well as cases where different computers perform various processes. Furthermore, the execution of various processes by a predetermined computer includes having other computers perform some or all of the various processes and obtain the results.
[0212] In other words, the information processing apparatus, information processing method and program according to this technology can also be applied to cloud computing configurations, in which multiple devices share a network and jointly process a single function.
[0213] The various configurations, such as the vector calculation unit, relative position calculation unit, and parameter generation unit, as well as the control flow of the communication system, described with reference to the accompanying drawings are merely embodiments and can be modified arbitrarily without departing from the spirit of this technology. Therefore, any other configurations, algorithms, etc., used to implement this technology can be employed.
[0214] It should be noted that the various effects described in this disclosure are merely illustrative and not limiting, and other effects may be provided. The description of multiple effects above does not imply that these effects are always provided simultaneously. This means that at least one of the above effects is provided depending on conditions, etc. Of course, effects not described in this disclosure may be provided.
[0215] At least two features according to the above embodiments can be combined. That is, the various features described in the various embodiments can be arbitrarily combined among the various embodiments.
[0216] It should be noted that this technology can also be configured as follows.
[0217] (1) An information processing device, comprising:
[0218] The generation unit generates calibration parameters based on a phase difference vector and a reference phase difference vector used as a reference. The phase difference vector is calculated based on sensing results measured by a distance measurement sensor, which includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include a reference receiving antenna and one or more other receiving antennas. The phase difference vector is the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas.
[0219] (2) The information processing apparatus according to (1), wherein,
[0220] The sensing results include measurement data that serves as the target for calibration.
[0221] (3) The information processing apparatus according to (1) or (2) further includes:
[0222] An estimation unit estimates the direction of arrival of a transmitted wave, which is transmitted from the transmitting antenna, reflected on the target, and received by at least one of the plurality of receiving antennas.
[0223] (4) The information processing apparatus according to any one of (1) to (3), wherein,
[0224] The generation unit generates the calibration parameters by projecting the phase difference vector onto the reference phase difference vector.
[0225] (5) The information processing apparatus according to any one of (1) to (4) further comprises:
[0226] A calculation unit calculates the relative position between the target and each of the plurality of receiving antennas.
[0227] (6) The information processing apparatus according to (5), wherein,
[0228] The calculation unit transforms the relative position of each of the plurality of receiving antennas onto the coordinate axis of the distance measurement sensor.
[0229] (7) The information processing apparatus according to any one of (1) to (6) further comprises:
[0230] An execution unit performs calibration based on the calibration parameters.
[0231] (8) The information processing apparatus according to any one of (1) to (7), wherein,
[0232] The distance measurement sensor is mounted on the mobile device.
[0233] (9) The information processing apparatus according to any one of (1) to (8), wherein,
[0234] The generation unit generates the calibration parameters based on the phase difference vector and the reference phase difference vector, wherein the phase difference vector is calculated based on multiple sensing results of multiple targets measured by the distance measurement sensor.
[0235] (10) The information processing apparatus according to (2), wherein,
[0236] The measurement data are multiple measurement data acquired when the positional relationship between the distance measurement sensor and the target is different.
[0237] (11) The information processing apparatus according to any one of (1) to (10), wherein,
[0238] The calibration parameters include at least parameters related to the relative position and relative angle of the distance measurement sensor relative to the target to be measured.
[0239] (12) The information processing apparatus according to (11), wherein,
[0240] The calibration parameters include at least six axis parameters relating to the relative positions of the three mutually orthogonal axial directions of the distance measuring sensor and the relative angles of the three axes.
[0241] (13) The information processing apparatus according to any one of (1) to (12), wherein,
[0242] The distance measurement sensor transmits extremely high frequency (EHF) radio waves from the transmitting antenna.
[0243] (14) The information processing apparatus according to any one of (1) to (13), wherein,
[0244] The target is in a stationary state.
[0245] (15) The information processing apparatus according to any one of (1) to (14), wherein,
[0246] The relative position of the target with respect to the reference point used for calibration is unknown.
[0247] (16) The information processing apparatus according to any one of (1) to (15), wherein,
[0248] The target is at a distance from the distance measurement sensor, and aliasing occurs at this distance due to phase difference characteristics.
[0249] (17) An information processing method, comprising:
[0250] By computer system:
[0251] Calibration parameters are generated based on a phase difference vector and a reference phase difference vector used as a reference. The phase difference vector is calculated based on sensing results measured by a distance measurement sensor, which includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include a reference receiving antenna and one or more other receiving antennas. The phase difference vector is the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas.
[0252] (18) A program that causes a computer system to perform:
[0253] The step of generating calibration parameters based on a phase difference vector and a reference phase difference vector as a reference, wherein the phase difference vector is calculated based on sensing results measured by a distance measurement sensor, the distance measurement sensor including a plurality of receiving antennas and one or more transmitting antennas, the plurality of receiving antennas including a reference receiving antenna and one or more other receiving antennas, and the phase difference vector being the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas.
[0254] Reference tag list
[0255] 1 vehicle
[0256] 54 mmWave Radar
[0257] 110 Calibration System
[0258] 115 Target
[0259] 120 Information Processing Device
[0260] 124 Vector Calculation Units
[0261] 125 Relative Position Calculation Unit
[0262] 126 Relative Angle Estimation Units
[0263] 127 Parameter Generation Unit
[0264] 128 Calibration Execution Unit
Claims
1. An information processing apparatus, comprising: A generation unit generates calibration parameters based on a phase difference vector and a reference phase difference vector used as a reference. The phase difference vector is calculated based on sensing results measured by a distance measurement sensor. The distance measurement sensor includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include a reference receiving antenna and one or more other receiving antennas. The phase difference vector is the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas. The calibration parameters include at least parameters related to the relative position and relative angle of the distance measurement sensor relative to the target being measured; and The phase difference vector is uniquely determined by using a rough estimate of the relative position between the one or more transmitting antennas and each of the plurality of receiving antennas as a condition for the relative angle of the distance measuring sensor relative to the target to be measured.
2. The information processing apparatus according to claim 1, wherein, The sensing results include measurement data that serves as the target for calibration.
3. The information processing apparatus according to claim 1, further comprising: An estimation unit estimates the direction of arrival of a transmitted wave, which is transmitted from the transmitting antenna, reflected on the target, and received by at least one of the plurality of receiving antennas.
4. The information processing apparatus according to claim 1, wherein, The generation unit generates the calibration parameters by projecting the phase difference vector onto the reference phase difference vector.
5. The information processing apparatus according to claim 1, further comprising: A calculation unit calculates the relative position between the target and each of the plurality of receiving antennas.
6. The information processing apparatus according to claim 5, wherein, The calculation unit transforms the relative position of each of the plurality of receiving antennas onto the coordinate axis of the distance measurement sensor.
7. The information processing apparatus according to claim 1, further comprising: An execution unit performs calibration based on the calibration parameters.
8. The information processing apparatus according to claim 1, wherein, The distance measurement sensor is mounted on the mobile device.
9. The information processing apparatus according to claim 1, wherein, The generation unit generates the calibration parameters based on the phase difference vector and the reference phase difference vector, wherein the phase difference vector is calculated based on multiple sensing results of multiple targets measured by the distance measurement sensor.
10. The information processing apparatus according to claim 2, wherein, The measurement data are multiple measurement data acquired when the positional relationship between the distance measurement sensor and the target is different.
11. The information processing apparatus according to claim 1, wherein, The calibration parameters include at least six axis parameters relating to the relative positions of the three mutually orthogonal axial directions of the distance measuring sensor and the relative angles of the three axes.
12. The information processing apparatus according to claim 1, wherein, The distance measurement sensor transmits extremely high frequency (EHF) radio waves from the transmitting antenna.
13. The information processing apparatus according to claim 1, wherein, The target is in a stationary state.
14. The information processing apparatus according to claim 1, wherein, The relative position of the target with respect to the reference point used for calibration is unknown.
15. The information processing apparatus according to claim 1, wherein, The target is at a distance from the distance measurement sensor, and aliasing occurs at this distance due to phase difference characteristics.
16. An information processing method, comprising: By computer system: Calibration parameters are generated based on a phase difference vector and a reference phase difference vector, which is calculated based on sensing results measured by a distance measurement sensor. The distance measurement sensor includes multiple receiving antennas and one or more transmitting antennas. The multiple receiving antennas include a reference receiving antenna and one or more other receiving antennas. The phase difference vector is the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas. The calibration parameters include at least parameters related to the relative position and relative angle of the distance measurement sensor relative to the target being measured; and The phase difference vector is uniquely determined by using a rough estimate of the relative position between the one or more transmitting antennas and each of the plurality of receiving antennas as a condition for the relative angle of the distance measuring sensor relative to the target to be measured.
17. A program product that causes a computer system to perform: The step of generating calibration parameters based on a phase difference vector and a reference phase difference vector as a reference, wherein the phase difference vector is calculated based on sensing results measured by a distance measurement sensor, the distance measurement sensor including multiple receiving antennas and one or more transmitting antennas, the multiple receiving antennas including a reference receiving antenna and one or more other receiving antennas, and the phase difference vector being the phase difference vector between the reference receiving antenna and each of the one or more other receiving antennas. in, The calibration parameters include at least parameters related to the relative position and relative angle of the distance measurement sensor relative to the target to be measured; as well as The phase difference vector is uniquely determined by using a rough estimate of the relative position between the one or more transmitting antennas and each of the plurality of receiving antennas as a condition for the relative angle of the distance measuring sensor relative to the target to be measured.
Citation Information
Patent Citations
DBF radar device
JP2001166029A
