Position information acquisition system, position information acquisition method

Abstract

This disclosure relates to a location information acquisition system and a location information acquisition method. The location information acquisition system of this disclosure includes: a server that receives request data from a terminal and sends information corresponding to the content of the received request data to the terminal; and multiple markers representing codes indicating that the request data can be acquired through a predetermined identification method. A vehicle identifies the image markers to acquire the request data and sends the request data to the server. When the source of the received request data is a vehicle, the server, regardless of the content of the request data, sends the location information of a specific location marked with an image marker to the vehicle.

Description

Technical Field

[0001] This invention relates to a location information acquisition system and a location information acquisition method for obtaining location information that enables a vehicle to determine its position and orientation on a map. Background Technology

[0002] Japanese Patent Application Publication No. 2011-013075 discloses a vehicle position estimation system that can reliably detect location at low cost even in environments where GPS signals cannot be received. This vehicle position estimation system consists of image markers embedded with location information and a vehicle that determines its position by receiving GPS signals. The vehicle has an image recognition unit that acquires the location information embedded in the image markers from images captured by a camera. Even when GPS signals are unavailable, the vehicle estimates its position based on the location information acquired by the image recognition unit.

[0003] In autonomous vehicles, high-precision self-positioning is required to estimate the vehicle's location and orientation on a map. Self-positioning typically involves using the vehicle's initial location and orientation on the map as a baseline, and then estimating the vehicle's position and orientation based on the amount of movement. However, the accuracy of the initial location's position and orientation information directly affects the accuracy of the self-positioning estimation. Therefore, it is essential to determine the vehicle's location and orientation on the map with high precision.

[0004] The applicants of this disclosure consider a scenario where a vehicle, assuming it originates from a specific location such as a bus or taxi stop, autonomously navigates, acquiring information (hereinafter referred to as "location information") that allows the vehicle to determine its position and orientation on a map using image markers. That is, image markers are placed at specific locations such as bus stops, and the vehicle acquires location information at those locations. Then, based on the acquired location information, the vehicle determines its position and orientation on the map and begins its own position estimation and autonomous driving. In this case, from the viewpoint of convenience and cost, the image markers represent generally common codes, rather than special codes. Therefore, it is assumed that a typical user would obtain information from the image markers out of curiosity.

[0005] As disclosed in Japanese Patent Application Publication No. 2011-013075, if the information obtainable from the image marker is location information, the user will obtain information that is meaningless to them, which may cause the user trouble. Summary of the Invention

[0006] This disclosure was made in view of the above-mentioned problems, and its purpose is to provide a location information acquisition system and method that allows a vehicle to obtain location information from image markers without the general user acquiring meaningless information.

[0007] The first disclosed location information acquisition system is a system for acquiring location information that enables a vehicle to determine its position and orientation on a map. This location information acquisition system includes: a server that receives request data from a terminal and sends information corresponding to the content of the received request data to the terminal; multiple image markers representing codes that allow the request data to be acquired through a predetermined discrimination method; a camera equipped in the vehicle that captures images of the environment surrounding the vehicle; an information processing device equipped in the vehicle that performs processing to acquire the request data by discerning the image markers captured by the camera based on the predetermined discrimination method; and a communication device equipped in the vehicle that sends the request data to the server and receives information from the server. Here, the image markers are respectively set at specific locations. Furthermore, upon receiving request data from the vehicle, the server, regardless of the content of the request data, sends the location information of the specific locations where the image markers are set to the vehicle.

[0008] The second disclosed location information acquisition system, compared to the first disclosed location information acquisition system, also includes the following features.

[0009] The server is a web server, and the requested data is a URL.

[0010] The third disclosed location information acquisition system is a system for acquiring location information that enables a vehicle to determine its position and orientation on a map. This location information acquisition system includes: multiple image markers representing codes that allow data to be acquired through a prescribed discrimination method; a camera equipped on the vehicle for capturing images of the environment surrounding the vehicle; and an information processing device equipped on the vehicle. Here, the image markers are each positioned at a specific location. Furthermore, the information processing device stores a mapping table that maps location information at a specific location to data acquired from the image markers. The information processing device performs: processing for acquiring information from the camera; discrimination processing for acquiring data from the image markers captured by the camera based on a prescribed discrimination method; and transformation processing for acquiring location information corresponding to the data acquired through the discrimination processing based on the mapping table.

[0011] The fourth disclosed location information acquisition system, compared to the third disclosed location information acquisition system, also includes the following features.

[0012] The data obtained from the image tags is a URL.

[0013] The location information acquisition system disclosed in the fifth disclosure, compared with the location information acquisition system disclosed in the third or fourth disclosure, also includes the following features.

[0014] The correspondence table maps location information at a specific location to a combination of data obtained from image markers and the category of the code represented by the image markers. Furthermore, in the discrimination process, the information processing device also acquires information about the category of the code represented by the image markers captured by the camera. And, in the transformation process, the information processing device uses the correspondence table to acquire location information corresponding to the combination of data and code category acquired through the discrimination process.

[0015] The sixth disclosed method for acquiring location information is a method for acquiring location information that enables a vehicle to determine its position and orientation on a map. In this method, a processor in the vehicle executes at least one program: processing to acquire information from a camera capturing images of the vehicle's surroundings; processing to identify image markers captured by the camera using a predetermined discrimination method to acquire requested data; and processing to send the requested data to a server and receive information from the server. Furthermore, in the server, a processor executing at least one program executes: processing to determine whether the source of the received requested data is the vehicle; and, if the source of the received requested data is the vehicle, processing to send location information of a specific location marked with an image marker to the vehicle, regardless of the content of the requested data. Here, the server is a device that receives requested data from a terminal and sends information corresponding to the content of the requested data to the terminal. An image marker is a marker set in a specific location and representing a code that allows the requested data to be acquired using a predetermined discrimination method.

[0016] The seventh disclosed method for acquiring location information is a method for obtaining location information that enables a vehicle to determine its position and orientation on a map. In this method, a processor executing at least one program performs: processing to acquire information from a camera that captures images of the vehicle's surroundings; processing to acquire data by identifying image markers captured by the camera based on a predetermined discrimination method; and processing to acquire location information corresponding to the data acquired through the discrimination process based on a mapping table obtained by matching location information at a specific location with the data acquired from the image markers.

[0017] According to the location information acquisition system and method disclosed herein, a vehicle can acquire location information at a specific location by using image markers placed at that location. Furthermore, the code represented by the image marker can be configured to represent appropriate request data or data. In particular, the request data or data can be configured so that a user can obtain meaningful information from it. This prevents the user from obtaining meaningless information from the image marker. Attached Figure Description

[0018] Hereinafter, with reference to the accompanying drawings, the features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described, wherein the same reference numerals denote the same elements, wherein:

[0019] Figure 1 This is a conceptual diagram used to illustrate the outline of the location information acquisition system of the first embodiment.

[0020] Figure 2 This is a conceptual diagram illustrating an example of a location information acquisition system being applied to a vehicle autonomously driving in multiple specific locations.

[0021] Figure 3 This is a block diagram used to illustrate an example of the configuration of the vehicle according to the first embodiment.

[0022] Figure 4 This is a block diagram used to explain the processing performed by the information processing apparatus of the first embodiment.

[0023] Figure 5A This is a conceptual diagram illustrating an example of location determination processing performed by the location information obtained from the server and the location estimation processing unit.

[0024] Figure 5B This is a conceptual diagram illustrating an example of location determination processing performed by the location information obtained from the server and the location estimation processing unit.

[0025] Figure 6 This is a flowchart illustrating the processing within a vehicle in a location information acquisition method implemented by the location information acquisition system of the first embodiment.

[0026] Figure 7 This is a flowchart illustrating the processing in the server within the location information acquisition method implemented by the location information acquisition system of the first embodiment.

[0027] Figure 8 This is a conceptual diagram used to explain the outline of the processing performed by the information processing apparatus in a modified example of the first embodiment.

[0028] Figure 9 This is a block diagram for explaining the processing performed by the information processing apparatus in a modified example of the first embodiment.

[0029] Figure 10 This is a conceptual diagram used to illustrate the outline of the location information acquisition system of the second embodiment.

[0030] Figure 11A This is a conceptual diagram representing an example of a correspondence table for the second embodiment.

[0031] Figure 11B This is a conceptual diagram representing an example of a correspondence table for the second embodiment.

[0032] Figure 12 This is a block diagram used to explain the processing performed by the information processing apparatus of the second embodiment.

[0033] Figure 13 This is a flowchart illustrating the location information acquisition method implemented by the location information acquisition system of the second embodiment.

[0034] Figure 14 This is a flowchart illustrating the processing performed by the transformation processing unit in the location information acquisition system of Modified Example 1 of the second embodiment.

[0035] Figure 15 This is a conceptual diagram representing an example of a correspondence table for a variation of the second embodiment, Example 1.

[0036] Figure 16 This is a conceptual diagram showing an example of a correspondence table for a variation of the second embodiment, 2. Detailed Implementation

[0037] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Where numerical values ​​such as the number, quantity, amount, and range of each element are mentioned in the embodiments shown below, the concept of this disclosure is not limited to these mentioned values, except where specifically stated or clearly determined in principle. Furthermore, regarding the configurations described in the embodiments shown below, they are not necessarily essential to the concept of this disclosure, except where specifically stated or clearly determined in principle. It should be noted that in the various figures, the same or equivalent parts are labeled with the same reference numerals, and repetitive descriptions are appropriately simplified or omitted.

[0038] 1. First Implementation Method

[0039] 1-1. Overview

[0040] The location information acquisition system 10 of the first embodiment is applied to vehicles that autonomously drive from specific locations such as bus and taxi stations. Figure 1 This is a conceptual diagram used to explain the outline of the location information acquisition system 10 of the first embodiment. Figure 1 The vehicle 1 shown is an autonomous vehicle that departs from a specific location (SP) and operates autonomously. Typically, vehicle 1 is a bus or taxi used by a general user (USR) and operating autonomously. Figure 1In the image, SP, a specific location, is shown as a station where vehicle 1 can park and where user USR can get on and off vehicle 1.

[0041] The location information acquisition system 10 includes an image tag MK and a server 3. The image tag MK represents a code that can acquire data through a prescribed identification method. For example, it is a stacked or matrix-type QR code. However, it can also be other codes. Typically, the code represented by the image tag MK is a commonly used code that can acquire data through a user terminal 2 (e.g., a smartphone) held by the user USR.

[0042] Image markers MK are set at specific locations SP. For example, such as... Figure 1 As shown, a billboard BD is installed at a specific location SP.

[0043] Server 3 is a device (or a virtual device) configured on a communication network. It receives request data in a specified format from terminals connected to the network and sends information (response information) corresponding to the content of the request data to the terminals. Typically, server 3 is a web server configured on the Internet. Typically, the request data is a URL (Uniform Resource Locator).

[0044] In the location information acquisition system 10, the code represented by the image tag MK indicates a request for data from the server 3. That is, the terminal obtains the request data from the image tag MK and sends the obtained request data to the server 3, thereby enabling the terminal to receive information (response information) corresponding to the content of the request data from the server 3.

[0045] Therefore, the user (USR) can obtain information from the image tag MK via user terminal 2 as follows. Here, we will illustrate the case where server 3 is a web server and the requested data is a URL. The user (USR) obtains the URL from the image tag MK through the functions of user terminal 2 (e.g., an application installed on user terminal 2). The URL specifies the data stored on server 3. Typically, it specifies an HTML (Hypertext Markup Language) file, image file, etc., representing a specified webpage.

[0046] Next, user terminal 2, connected to the Internet, requests data from server 3 according to the URL. Server 3 then sends data to user terminal 2 corresponding to the content of the URL. User terminal 2 then receives the data from server 3 and notifies user USR of the data information. Typically, user terminal 2 displays the information via an appropriate web browser, using an HTML file or similar file received from server 3, thereby notifying user USR.

[0047] In this way, the user USR can obtain information from the image tag MK via user terminal 2. Here, by setting the data specified by the URL to appropriate data such as an HTML file displaying timetables and service information, the user USR can obtain meaningful information from the image tag MK.

[0048] On the other hand, the vehicle 1 in the first embodiment is equipped with a camera (CAM) that captures images of the surrounding environment, and the vehicle 1 acquires image data of the captured area (IMG). Next, the vehicle 1 is equipped with... Figure 1 The information processing device (not shown) allows vehicle 1 to obtain requested data from image tags MK included in image data based on a predetermined identification method. Furthermore, vehicle 1 is equipped with... Figure 1 The communication device (not shown) allows vehicle 1 to communicate with server 3 to send data requests to server 3 and receive information from server 3.

[0049] Here, in the first embodiment, when the server 3 receives request data from the vehicle 1, regardless of the content of the request data, it sends information (location information) to the vehicle 1 that enables the vehicle 1 to determine its position and orientation on the map at a specific location SP. That is, by sending the request data obtained from the image marker MK via the communication device to the server 3, the vehicle 1 can obtain the location information at the specific location SP from the server 3.

[0050] It should be noted that the location information acquisition system 10 can also be configured such that there are multiple specific locations SP, and image markers MK are set in each specific location SP. Figure 2 This is a conceptual diagram illustrating an example of a location information acquisition system 10 being applied to a vehicle 1 autonomously driving in multiple specific locations SP1, SP2, and SP3.

[0051] Figure 2 This illustrates a scenario where vehicle 1 is pre-programmed to autonomously travel to specific locations SP1, SP2, and SP3 in the order of SP1, SP2, and SP3. For example, vehicle 1 is a bus, and the specific locations SP1, SP2, and SP3 are bus stops.

[0052] Image markers MK are set at specific locations SP1, SP2, and SP3. For example... Figure 2 As shown, in each image marker MK located at specific locations SP1, SP2, and SP3, numbers are attached to the reference numerals to distinguish each image marker MK.

[0053] Vehicle 1 first retrieves request data from image marker MK1 at a specific location SP1 and sends the request data, thereby obtaining location information for location SP1 from server 3. Then, based on the location information, Vehicle 1 determines its position and orientation on the map, begins its own position estimation and autonomous driving, and moves towards a specific location SP2. Next, Vehicle 1 retrieves request data from image marker MK2 at the specific location SP2 and sends the request data, thereby obtaining location information for location SP2 from server 3. Then, based on the location information, Vehicle 1 determines its own position and orientation on the map, begins its own position estimation and autonomous driving, and moves towards a specific location SP3.

[0054] Subsequently, vehicle 1 repeatedly performs the same process at the specific location SP3 to determine its position and orientation on the map, and begins its own position estimation and autonomous driving. The location information acquisition system 10 does the same in the case of image markers MK set at each of more specific locations SP.

[0055] Thus, at each of the multiple specific locations SP, vehicle 1 obtains location information from the image marker MK to determine its position and orientation on the map, and begins its own position estimation and autonomous driving. Based on the updated and more accurate self-position estimation, it can drive autonomously until the next specific location SP.

[0056] Here, typically, the codes represented by image markers MK1, MK2, and MK3 represent different request data. That is, when server 3 receives request data from vehicle 1, it determines which of the image markers MK located at specific locations SP1, SP2, and SP3 the request data was obtained from, and sends the corresponding location information of that specific location SP to vehicle 1. Thus, server 3 can select and send the location information of each of the specific locations SP1, SP2, and SP3.

[0057] However, it is also possible that the codes represented by image markers MK1, MK2, and MK3 represent the same request data, and server 3 selects and sends location information based on information related to the communication of the request data. For example, if the communication device of vehicle 1 sends the request data via a base station, it is also possible that the location of the base station determines which image marker MK among the specific locations SP1, SP2, and SP3 the request data was obtained from, and sends the location information of the corresponding specific location SP to vehicle 1.

[0058] It should be noted that, as mentioned above, when a terminal other than vehicle 1 sends request data obtained from image markers MK1, MK2 and MK3 to server 3, server 3 will send information corresponding to the content of the request data to the terminal.

[0059] 1-2. Examples of vehicle composition

[0060] Figure 3 This is a block diagram illustrating an example of the configuration of vehicle 1 according to the first embodiment. Vehicle 1 includes a camera (CAM), an information processing unit 100, a sensor group 200, an HMI device 300, a communication device 400, and an actuator group 500. The information processing unit 100 is configured to exchange information with the camera (CAM), sensor group 200, HMI device 300, communication device 400, and actuator group 500. Typically, these are electrically connected via wiring harnesses.

[0061] The camera CAM captures images of the environment surrounding vehicle 1 and outputs image data. Here, the camera CAM can also be a camera that captures images of a specific area around vehicle 1. For example, the camera CAM can also be a camera that captures images of the environment in front of vehicle 1. The image data output by the camera CAM is transmitted to the information processing device 100.

[0062] Sensor group 200 is a group of sensors that detect and output information representing the driving environment of vehicle 1 (driving environment information). The driving environment information output by sensor group 200 is transmitted to information processing device 100. Typically, sensor group 200 includes sensors that detect information about the environment of vehicle 1, such as the driving state of vehicle 1 (vehicle speed, acceleration, yaw rate, etc.), and sensors that detect information about the surrounding environment of vehicle 1 (overtaking vehicles, lanes, obstacles, etc.).

[0063] Examples of sensors used to detect information about the environment of vehicle 1 include wheel speed sensors for detecting vehicle speed, acceleration sensors for detecting vehicle acceleration, and angular velocity sensors for detecting vehicle yaw rate. Examples of sensors used to detect the environment surrounding vehicle 1 include millimeter-wave radar, sensor cameras, and LiDAR (Light Detection and Ranging). Here, a camera (CAM) can also be used as a sensor to detect the environment surrounding vehicle 1. For example, a sensor camera can also function as a camera (CAM).

[0064] HMI device 300 is a device with HMI (Human Machine Interface) functionality. HMI device 300 provides various HMI information to information processing device 100 through operation by an operator of vehicle 1, and also notifies the operator of HMI information related to the processing performed by information processing device 100. HMI device 300 may be, for example, a switch, a touch panel display, an automotive instrument cluster, or a combination thereof.

[0065] The information processing device 100 performs various processes, such as controlling the vehicle 1, based on the acquired information and outputs the execution results. The execution results are transmitted to the actuator assembly 500, for example, as control signals. Alternatively, they are transmitted to the communication device 400 as communication information. The information processing device 100 may also be an external device to the vehicle 1. In this case, the information processing device 100 acquires information and outputs execution results through communication with the vehicle 1.

[0066] The information processing device 100 is a computer equipped with a memory 110 and a processor 120. Typically, the information processing device 100 is an ECU (Electronic Control Unit). The memory 110 stores a program PG that can be executed by the processor, as well as data DT including information acquired by the information processing device 100 and various information related to the program PG. Alternatively, the memory 110 may store timing data of the acquired information for a certain period as the data DT. The processor 120 reads the program PG from the memory 110 and performs processing according to the program PG based on the information in the data DT read from the memory 110.

[0067] The processing performed by the information processing device 100, and more specifically, the processing performed by the processor 120 according to the program PG, includes processing for identifying the image tag MK to obtain requested data, processing related to self-position estimation, and processing related to autonomous driving. Details of these processes will be described later. Here, the requested data obtained from the image tag MK through the processing performed by the information processing device 100 is transmitted to the communication device 400 as communication information.

[0068] It should be noted that the information processing device 100 can also be a system composed of multiple computers. In this case, the individual computers are configured to exchange information with each other to the extent that they can acquire the information required for the execution of the processing. Furthermore, the program PG can also be a combination of multiple programs.

[0069] The communication device 400 is a device for transmitting and receiving various types of information (communication information) by communicating with external devices of the vehicle 1. The communication device 400 is configured to be connected to at least a communication network NET comprising the server 3, and is capable of transmitting and receiving information with the server 3. For example, if the server 3 is located on the Internet, the communication device 400 is a device capable of connecting to the Internet to transmit and receive information. In this case, typically, the communication device 400 is a terminal that connects to the Internet via a base station to transmit and receive information wirelessly.

[0070] The communication information received by the communication device 400 is transmitted to the information processing device 100. The communication information transmitted to the information processing device 100 includes at least location information received from the server 3. Furthermore, request data obtained by the communication device 400 from the information processing device 100 is sent from the communication device 400 to the server 3.

[0071] It should be noted that the communication device 400 may also include other devices. For example, it may include devices for workshop communication, road-to-work communication, GPS (Global Positioning System) receivers, etc. In this case, the communication device 400 refers to a group of these devices.

[0072] The actuator group 500 is a group of actuators that operate according to control signals obtained from the information processing device 100. The actuators included in the actuator group 500 include, for example, actuators that drive an engine (internal combustion engine, electric motor, or a combination thereof), actuators that drive a braking mechanism of the vehicle 1, and actuators that drive a steering mechanism of the vehicle 1. The various actuators included in the actuator group 500 operate according to the control signals, thereby realizing various controls of the vehicle 1 performed by the information processing device 100.

[0073] As described above, vehicle 1 sends request data to server 3 and receives location information from server 3 via communication device 400. When server 3 receives request data from vehicle 1 via communication device 400, it sends location information to vehicle 1. Conversely, when it receives request data from a terminal other than vehicle 1 connected to the communication network NET (e.g., user terminal 2), it sends information corresponding to the content of the request data (response information). In other words, server 3 operates differently depending on whether the source of the received request data is vehicle 1, and the information to be sent varies accordingly.

[0074] 1-3. Processing performed by the information processing device

[0075] Figure 4This is a block diagram used to explain the processing performed by the information processing device 100. For example... Figure 4 As shown, the processing performed by the information processing device 100 consists of an image tag recognition processing unit (MRU), a self-position estimation processing unit (LCU), and an autonomous driving control processing unit (ADU). These can be implemented as part of a program (PG) or by a separate computer constituting the information processing device 100.

[0076] The image tag recognition processing unit (MRU) performs processing to identify the image tag MK captured by the camera CAM from the image data output from the camera CAM in order to obtain the requested data. The image tag recognition processing unit (MRU) performs processing based on a defined recognition method related to the image tag MK. For example, if the image tag MK represents a matrix-type QR code, the image tag recognition processing unit (MRU) performs image parsing of the image data to identify the portion of the image data containing the image tag MK. Then, through image processing of the image tag MK, it identifies the pattern of the QR code cells, thereby obtaining the requested data.

[0077] The information processing device 100 outputs the request data obtained through the processing performed by the image tag recognition processing unit MRU, and transmits the request data to the communication device 400. The communication device 400 sends the obtained request data to the server 3 and receives location information from the server 3. Then, the communication device 400 outputs the location information received from the server 3 and transmits the location information to the information processing device 100.

[0078] The self-position estimation processing unit (LCU) performs processing related to estimating the position and orientation of vehicle 1 on the map. Typically, based on driving environment information and map information, the position and orientation of vehicle 1 on the map are estimated at any time according to the amount of movement of vehicle 1 from the starting point of estimation and the relative position of vehicle 1 with respect to the surrounding environment. The self-position estimation result performed by the self-position estimation processing unit (LCU) (self-position estimation result) is transmitted to the autonomous driving control processing unit (ADU).

[0079] Here, there is no limitation on the degrees of freedom of the position and orientation of vehicle 1 on the map estimated by the LCU (Location Estimation Processing Unit). For example, the position of vehicle 1 on the map can be given by two-dimensional coordinate values ​​(X, Y) and the orientation of vehicle 1 can be given by the yaw angle θ, or the position and orientation of vehicle 1 on the map can be given by three degrees of freedom respectively.

[0080] Furthermore, the map information can be either information pre-stored in the memory 110 as data DT, or information acquired from an external source via the communication device 400. Alternatively, it can be information about an environmental map generated through processing performed by the information processing device 100.

[0081] The processing performed by the self-position estimation processing unit (LCU) includes determining the vehicle's position and orientation on the map based on the position information acquired by the information processing device 100 (hereinafter also referred to as "position determination processing"). Typically, the self-position estimation processing unit (LCU) uses the information on the vehicle's position and orientation on the map determined by the position determination processing as a base point to begin estimation. Examples of position information and position determination processing will be described later.

[0082] The Autonomous Driving Control Unit (ADU) performs processing related to the autonomous driving of vehicle 1 and generates control signals for autonomous driving. Typically, it sets a driving plan to the destination and generates a driving path based on the driving plan, driving environment information, map information, and its own position estimation results. Then, it generates control signals related to acceleration, braking, and steering to make vehicle 1 travel along the driving path.

[0083] It should be noted that the image marker recognition processing unit (MRU) can also be configured to perform processing when a specified operation of the HMI device 300 is performed. Furthermore, the self-position estimation processing unit (LCU) and the autonomous driving control processing unit (ADU) can also be configured to begin self-position estimation and autonomous driving when the information processing device 100 acquires position information. For example, the image marker recognition processing unit (MRU) may take HMI information as input and perform processing when a specified switch on the vehicle 1 is pressed. In this case, when the specified switch is pressed, the vehicle 1 begins self-position estimation and autonomous driving.

[0084] 1-4. Location Information and Location Determination Processing

[0085] The location information of vehicle 1 at a specific location SP obtained from server 3 is information that allows vehicle 1 to determine its position and orientation on the map at that specific location SP. Furthermore, Figure 4 The self-position estimation processing unit (LCU) shown performs position determination processing and determines the vehicle's position and orientation on the map based on the position information. The following is an example of the position information obtained by vehicle 1 from server 3 and the position determination processing performed by the self-position estimation processing unit (LCU).

[0086] Figure 5A , Figure 5B This is a conceptual diagram illustrating an example of the location information obtained by vehicle 1 from server 3 and the location determination processing performed by its own location estimation processing unit (LCU). Figure 5A , Figure 5B Two examples are shown to illustrate location information and location determination processing.

[0087] In the role of Figure 5A In the example shown, a parking frame FR is provided at a specific location SP for vehicle 1 to park. The parking frame FR is, for example, a parking position at a station. The location information obtained by vehicle 1 from server 3 is set as the position and orientation of vehicle 1 on the map when vehicle 1 is parking along the parking frame FR. For example, as... Figure 5A As shown, the two-dimensional coordinates and yaw angle (X, Y, θ) of vehicle 1 when it is parking along the parking frame FR will be used as the position information to be obtained.

[0088] Then, in the position determination process, the self-position estimation processing unit (LCU) can use the acquired position information as the determined position and orientation of the vehicle on the map. Alternatively, the self-position estimation processing unit (LCU) can correct the position information based on the relative position information of vehicle 1 and parking frame FR, and use the corrected position information as the determined position and orientation of the vehicle on the map. That is, vehicle 1 can determine its position and orientation on the map by acquiring position information while it is parked along parking frame FR.

[0089] In the role of Figure 5B In the example shown, the location information obtained by vehicle 1 from server 3 is set to its location on a map with image marker MK. For example, as Figure 5B As shown, when the image marker MK is set on the billboard BD, the two-dimensional coordinates (X, Y) of the billboard BD are used as the position information to be acquired. Furthermore, the sensor group 200 detects the relative position and relative angle of the vehicle 1 relative to the location where the image marker MK is set.

[0090] Then, in the location determination process, the self-position estimation processing unit (LCU) determines the vehicle's position and orientation on the map based on the acquired position information and the detected relative position and relative angle information. That is, vehicle 1 can determine its position and orientation on the map by sensing image markers MK at a specific location SP to acquire position information.

[0091] 1-5. Location Information Acquisition Methods

[0092] The location information acquisition method implemented by the location information acquisition system 10 of the first embodiment will be described below.

[0093] Figure 6 This is a flowchart illustrating the processing in vehicle 1 within the location information acquisition method implemented by the location information acquisition system 10 of the first embodiment. The process is performed when vehicle 1 is parked at a specific location SP and camera CAM is capturing an image of marker MK. Figure 6The process is shown. The determination to start the process can be repeated at a predetermined cycle, or it can be initiated based on the condition that the operator of vehicle 1 or others has performed the prescribed operation of the HMI device 300.

[0094] In step S100, the camera CAM captures images of the environment surrounding the vehicle 1, and the information processing device 100 acquires the image data from the camera CAM. After step S100, the processing proceeds to step S110.

[0095] In step S110, the image tag identification processing unit MRU identifies the image tag MK from the image data to obtain the requested data. After step S110, the process proceeds to step S120.

[0096] In step S120, request data is sent to server 3 via communication device 400. After step S120, the process proceeds to step S130.

[0097] In step S130, location information is obtained from server 3 via communication device 400. After step S130, the process ends.

[0098] exist Figure 6 After the processing shown is completed, typically, the self-position estimation processing unit (LCU) determines the vehicle's position and orientation on the map based on the acquired position information and begins self-position estimation. Furthermore, autonomous driving control processing unit (ADU) initiates autonomous driving.

[0099] Figure 7 This is a flowchart illustrating the processing in server 3 within the location information acquisition method implemented by the location information acquisition system 10 of the first embodiment. The process begins when server 3 receives the requested data from the terminal. Figure 7 The processing shown.

[0100] In step S200, server 3 determines the source of the received request data. This can be done, for example, assuming the communication network NET is the Internet, as follows.

[0101] A fixed IP address (Internet Protocol Address) is assigned to the communication device 400. The server 3 determines whether the source of the requested data is vehicle 1 based on the IP address or hostname of the sending source. Alternatively, the communication device 400 operates through a specific operating system (OS), and the server 3 determines whether the source of the requested data is vehicle 1 based on the OS name information of the sending source. Or, assuming the server 3 is a web server and the requested data is a URL, the communication device 400 requests data from the server 3 through a specific browser according to the URL, and the server 3 determines whether the source of the requested data is vehicle 1 based on the browser type information. However, other methods can also be used to determine whether the source of the requested data is vehicle 1.

[0102] After step S200, the process proceeds to step S210.

[0103] In step S210, server 3 determines whether the source of the obtained request data is a vehicle. If the source of the request data is a vehicle (step S210: Yes), the process proceeds to step S220. If the source of the request data is not a vehicle (step S210: No), the process proceeds to step S230.

[0104] In step S220, server 3 sends location information to vehicle 1. After step S220, the process ends.

[0105] In step S230, server 3 sends information (response information) to the terminal corresponding to the content of the requested data. After step S230, the processing ends.

[0106] 1-6. Effects

[0107] As explained above, according to the location information acquisition system 10 of the first embodiment, the vehicle 1 can acquire location information at a specific location SP by using an image marker MK set at that location SP. Furthermore, the code represented by the image marker MK can be configured to represent appropriate request data. In particular, information meaningful to the user USR (e.g., timetables, service information) can be used as request data received from the server 3. This prevents the user USR from obtaining meaningless information from the image marker MK.

[0108] 1-7. Variations

[0109] The location information acquisition system 10 of the first embodiment can also be modified as follows. Hereinafter, matters described above will be omitted as appropriate.

[0110] The information processing device 100 can also be configured to perform processing on image data acquired from the camera CAM, specifying the region for identifying image markers MK.

[0111] Figure 8 This is a conceptual diagram illustrating an outline of the processing performed by the information processing apparatus 100 in a modified example of the first embodiment. Figure 8 In this process, the information processing unit 100 acquires image data of the shooting area IMG (the area surrounded by a dashed line) from the camera CAM. The information processing unit 100 calculates a discrimination region IDA (the area surrounded by a single-dot-dash line) within the acquired image data of the shooting area IMG, which specifies the area for discrimination of the image marker MK. Then, the information processing unit 100 performs image marker MK discrimination on the image data of the discrimination region IDA.

[0112] The information processing device 100 calculates the identification area IDA based on driving environment information. For example, it calculates the height of the location where the image marker MK is set above the ground based on information detected by LiDAR, and uses the area within a range (e.g., 1.5m ± 50cm) defined from that height as the identification area IDA.

[0113] Figure 9 This is a block diagram used to explain the processing performed by the information processing apparatus 100 in a modified example of the first embodiment. Figure 3 Comparison, such as Figure 9 As shown, the processing performed by the information processing apparatus 100 in the modified example of the first embodiment is configured to further include a region designation processing unit (IDU).

[0114] The Region Designation Unit (IDU) performs the process of calculating the Region IDA based on image data, using driving environment information. The Region IDA calculated by the IDU is then transmitted to the Image Tag Recognition Unit (MRU). The MRU performs the process of identifying image tags (MK) for the image data corresponding to the Region IDA to obtain the requested data.

[0115] Thus, by calculating the identification area IDA, it is possible to reduce misidentification and improve reading speed in the identification of image markers MK performed by the information processing device 100. In addition, it is possible to improve the flexibility of the size and placement of image markers MK.

[0116] 2. Second Implementation Method

[0117] The second embodiment will now be described. However, details that are repeated in the first embodiment will be omitted as appropriate.

[0118] 2-1. Overview

[0119] The location information acquisition system of the second embodiment is similar to that of the first embodiment, and is applied to the autonomous driving of a vehicle 1 that departs from a specific location SP such as a bus or taxi station.

[0120] Figure 10 This is a conceptual diagram used to explain the outline of the location information acquisition system 20 of the second embodiment.

[0121] The location information acquisition system 20 of the second embodiment includes an image tag MK. The image tag MK represents a code that allows data to be acquired through a prescribed identification method. In the location information acquisition system 20, the data acquired from the image tag MK may also be appropriately provided. For example, the code represented by the image tag MK may represent a specific URL, and the webpage specified by the URL may represent timetable or service information. Thus, the user USR can acquire meaningful information from the image tag MK via the user terminal 2.

[0122] In the following description, it is assumed that the code represented by the image tag MK represents a URL.

[0123] The vehicle 1 in the second embodiment is equipped with a camera CAM that captures images of the surrounding environment, and the vehicle 1 acquires image data of the captured area IMG. Next, the vehicle 1 is equipped with an information processing device, and the vehicle 1 obtains a URL from the image tag MK included in the image data based on a predetermined recognition method.

[0124] On the other hand, the information processing device of vehicle 1 stores a mapping table TBL that maps the URL obtained from image marker MK to the location information of a specific location SP. Then, vehicle 1 uses the mapping table TBL to obtain the location information corresponding to the URL obtained from image marker MK as the location information of the specific location SP.

[0125] It should be noted that the location information acquisition system 20 can also be configured to have multiple specific locations SP, each with an image marker MK. For example, as in Figure 2 As explained, the location information acquisition system 20 can also be applied to situations where the vehicle 1 autonomously travels in multiple specific locations SP. In this case, the vehicle 1 acquires location information from image markers MK at each specific location SP.

[0126] Here, the code represented by the image marker MK set at each specific location SP is configured to represent a unique URL. Thus, vehicle 1 can select and obtain the location information at each specific location SP based on the correspondence table TBL.

[0127] However, it can also be configured so that the user USR can obtain the same information from each image tag MK via user terminal 2. For example, the codes represented by the image tags MK can represent different URLs through different URL parameters; on the other hand, each URL can also specify the same webpage.

[0128] 2-2. Example of vehicle composition

[0129] The configuration of vehicle 1 in the second embodiment can also be the same as... Figure 3 The configuration shown is equivalent. However, the communication device 400 may not need to send or receive information with the server 3. Furthermore, the URL and location information obtained from the image tag MK may not be included in the communication information related to the communication device 400. In addition, the server 3 may also be a general server specified by the URL obtained from the image tag MK. That is to say, the server 3 may not need to perform any actions corresponding to the source of the received URL.

[0130] Here, the memory 110 stores a correspondence table TBL as data DT. The correspondence table TBL can be pre-stored information or information obtained and stored from an external source via the communication device 400.

[0131] Figure 11A , Figure 11B This is a conceptual diagram representing an example of the correspondence table TBL of the second embodiment. Figure 11A , Figure 11B Two examples are shown to serve as examples of the corresponding table TBL.

[0132] The mapping table TBL is data that maps location information to URLs obtained from image markers MK. The location information corresponding to the image marker MK is information at the specific location SP where the image marker MK is set, enabling vehicle 1 to determine its position and orientation on the map. This information can be compared with... Figure 5A , Figure 5B The information provided is equivalent to that described in the text.

[0133] exist Figure 11A The diagram shows an example of a mapping table TBL in the location information acquisition system 20 where the URLs obtained from each image marker MK have different endings. In the mapping table TBL, the three-dimensional coordinates and yaw angles (X, Y, Z, θ) of vehicle 1 are mapped to each URL.

[0134] exist Figure 11BThe diagram shows an example of a mapping table TBL in the location information acquisition system 20 where the URL parameters of the URLs obtained from each image marker MK are different. Similar to case a, the mapping table TBL maps the three-dimensional coordinates and yaw angles (X, Y, Z, θ) of vehicle 1 to the respective URLs.

[0135] 2-3. Processing performed by the information processing device

[0136] Figure 12 This is a block diagram used to explain the processing performed by the information processing apparatus 100 of the second embodiment. For example... Figure 12 As shown, the processing performed by the information processing device 100 consists of an image tag recognition processing unit (MRU), a transformation processing unit (CVU), a self-position estimation processing unit (LCU), and an autonomous driving control processing unit (ADU). These can be implemented as part of a program (PG) or by a separate computer constituting the information processing device 100.

[0137] The self-position estimation processing unit (LCU) and the autonomous driving control processing unit (ADU) are in Figure 4 The self-position estimation processing unit (LCU) and autonomous driving control processing unit (ADU) described in the text are equivalent.

[0138] The Image Tag Recognition Processing Unit (MRU) performs the process of identifying image tags (MK) captured by the camera (CAM) from the image data output from the camera (CAM) to obtain the URL. The Image Tag Recognition Processing Unit (MRU) performs the processing based on a defined recognition method related to the image tags (MK). The URL obtained by the Image Tag Recognition Processing Unit (MRU) is then passed to the Transform Processing Unit (CVU).

[0139] It should be noted that the image tag recognition processing unit MRU can also be configured to perform processing when a specified operation of the HMI device 300 is performed.

[0140] The Transform Processing Unit (CVU) outputs location information corresponding to the URL obtained by the Image Tag Recognition Unit (MRU) based on the correspondence table (TBL). The location information output by the Transform Processing Unit (CVU) is then transmitted to its own Location Estimation Unit (LCU).

[0141] 2-4. Location Information Acquisition Methods

[0142] The location information acquisition method implemented by the location information acquisition system 20 of the second embodiment will be described below.

[0143] Figure 13This is a flowchart illustrating the location information acquisition method implemented by the location information acquisition system 20 of the second embodiment. It is executed when vehicle 1 is parked at a specific location SP and camera CAM is capturing an image of marker MK. Figure 13 The process is shown. The determination to start the process can be repeated at a predetermined cycle, or it can be initiated based on the condition that the operator of vehicle 1 or others has performed the prescribed operation of the HMI device 300.

[0144] In step S300, the camera CAM captures images of the environment surrounding the vehicle 1, and the information processing device 100 acquires the image data from the camera CAM. After step S300, the processing proceeds to step S310.

[0145] In step S310, the image tag MK is identified from the image data by the image tag identification processing unit MRU, and the URL is obtained. After step S310, the process proceeds to step S320.

[0146] In step S320, the transformation processing unit CVU obtains the location information corresponding to the obtained URL based on the correspondence table TBL. After step S320, the processing ends.

[0147] 2-5. Effects

[0148] As explained above, according to the location information acquisition system 20 of the second embodiment, the vehicle 1 can acquire location information at a specific location SP by using an image marker MK set at that location SP. Furthermore, the code represented by the image marker MK can be configured to represent appropriate data. In particular, the code represented by the image marker MK can be set to represent a URL, and the webpage specified by the URL can be set to information meaningful to the user USR (e.g., timetable, service information). This prevents the user USR from obtaining meaningless information from the image marker MK.

[0149] 2-6. Variations

[0150] The location information acquisition system 20 of the second embodiment can also be modified as follows. Hereinafter, matters described in the foregoing will be appropriately omitted.

[0151] 2-6-1. Variation Example 1

[0152] The transformation processing unit (CVU) can also be configured to perform processing to extract a specific portion from the URL obtained by the image tag discrimination processing unit (MRU), and output position information corresponding to the extracted portion. In this case, the correspondence table (TBL) becomes data obtained by mapping the position information to the extracted portion.

[0153] Figure 14 This refers to the processing performed by the transformation processing unit (CVU) in the position information acquisition system 20 of the second embodiment, variant 1. Figure 13 The flowchart for step S320 in the above is shown here. Figure 11A As shown, assuming the URL obtained from each image tag MK is in the form of http: / / XXX.IDj (j = 1, 2, ...), IDj is set as a specific part.

[0154] In step S321, the transformation processing unit (CVU) removes inappropriate URLs that are not objects. For example, if the obtained URL does not correspond to the form http: / / XXX.IDj, it is determined that location information will not be obtained. This prevents misjudgments caused by reading codes that only represent specific parts. After step S321, the process proceeds to step S322.

[0155] In step S322, the transformation processing unit (CVU) extracts specific portions. For example, when the obtained URL is http: / / XXX.IDj, the portion for IDj is extracted. After step S322, the process proceeds to step S323.

[0156] In step S323, the transformation processing unit CVU obtains the position information corresponding to the extracted specific part based on the correspondence table TBL. Figure 15 This is a conceptual diagram showing an example of the correspondence table TBL for a variation of the second embodiment 1. For example... Figure 15 As shown, the correspondence table TBL is data obtained by mapping location information to specific extracted parts (IDs). After step S323, the process ends.

[0157] By adopting a modified scheme as in Modified Example 1, the size of the data in the corresponding table TBL can be reduced.

[0158] 2-6-2. Variation Example 2

[0159] The image tag identification processing unit MRU can also be configured to acquire information about the category of the code represented by the image tag MK. The transformation processing unit CVU can also be configured to output location information corresponding to the combination of the URL and the category of the code acquired from the image tag MK.

[0160] Generally, the code represented by the image marker MK can be classified into multiple categories independent of the data. For example, in a matrix-type QR code, the orientation of the code is indicated by the finder pattern. The category of the code can be determined by its orientation. Alternatively, the category can be determined by the code version, the code mask pattern, differences in code size, error correction level, etc.

[0161] In Modification 2, the image tag discrimination processing unit MRU further acquires information about the category of the code represented by the image tag MK and transmits the acquired code category information to the transformation processing unit CVU. Then, the transformation processing unit CVU outputs location information corresponding to the combination of the URL and code category obtained from the image tag MK based on the correspondence table TBL. In this case, the correspondence table TBL becomes data obtained by mapping the location information to the combination of the URL and code category.

[0162] Figure 16 This is a conceptual diagram representing an example of the correspondence table TBL for a variation of the second embodiment. For example... Figure 16 As shown, the mapping table TBL is data that maps location information to combinations of URLs and code categories. That is, even with the same URL, different code categories will correspond to different location information. It should be noted that the mapping table TBL can also be data that maps location information to combinations of URLs and multiple code categories.

[0163] By adopting a modified scheme such as Variation Example 2, more location information that can be corresponding to a URL can be provided.

[0164] 2-6-3. Variation Example 3

[0165] The information processing device 100 can also be configured to perform processing on image data acquired from the camera CAM, specifying the region (identification region IDA) for identifying image markers MK.

[0166] By employing a modified scheme as in Modified Example 3, and by calculating the discrimination area IDA, the identification of image markers MK performed by the information processing device 100 can reduce misidentification and improve reading speed. Furthermore, it can improve the flexibility of the size and placement of image markers MK.

Claims

1. A location information acquisition system, for acquiring location information that enables a vehicle to determine its position and orientation on a map, the location information acquisition system being characterized by comprising: The server receives request data from terminals, including vehicles, and sends information corresponding to the content of the request data to the terminals. Multiple image tags represent codes that can be used to obtain the request data sent to the server to request information through a specified identification method; A camera, equipped on the vehicle, captures images of the environment surrounding the vehicle; An information processing device, equipped in the vehicle, performs processing to obtain the requested data by identifying the image marker captured by the camera based on the identification method; as well as A communication device, equipped in the vehicle, sends the request data to the server and receives the information from the server. The image markers are set in specific locations. Upon receiving the request data obtained from the image marker, the server determines whether the source of the received request data is the vehicle. Upon receiving the requested data from the vehicle, regardless of the content of the requested data, the server will send the location information of the specific location marked with the image to the vehicle. When the server receives the request data from a terminal other than the vehicle, it sends information different from the location information to the other terminal based on the content of the request data.

2. The location information acquisition system according to claim 1, characterized in that, The server mentioned is a web server. The requested data is a Uniform Resource Locator (URL).

3. A method for acquiring location information, wherein the method acquires location information that enables a vehicle to determine its position and orientation on a map, the method being characterized in that... A server is a device that receives request data from a terminal, including a vehicle, and sends information corresponding to the content of the request data to the terminal. An image tag is a tag set in a specific location that represents a code sent to the server to request information, which can be obtained through a prescribed identification method. In the vehicle, a processor that executes at least one program performs: Processing of information acquired from cameras that capture images of the vehicle's surroundings; The process of obtaining the requested data by identifying the image markers captured by the camera based on the identification method; as well as The process of sending the request data to the server and receiving the information from the server. In the server, a processor that executes at least one program performs the following: Determine whether the source of the received request data is the vehicle's processing. When the source of the received request data is the vehicle, regardless of the content of the request data, the location information of the specific location marked with the image will be sent to the vehicle for processing. as well as If the source of the received request data is a terminal other than the vehicle, information different from the location information is sent to the other terminal according to the content of the request data.

Images