Information processing device

Abstract

We will improve the technology related to correcting vehicle location information. [Solution] The information processing device 20 receives an image and location information of a first location from a moving vehicle 10, selects an image of a location within a predetermined distance from the first location from an image database 22A which stores multiple images embedded with location information of a higher accuracy than a predetermined accuracy, compares the image of the first location with the image of the selected location to calculate the similarity of the images, identifies the selected location as the second location if the similarity is above a threshold, and includes a control unit 23 that generates a correction function to correct the location information of the first location based on the difference between the location information of the first location and the location information of the second location.

Description

Technical Field

[0006] , , , ,

[0001] This disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] Conventionally, technologies related to correction of vehicle position information have been known. For example, Patent Document 1 discloses a technology for estimating the position and orientation of a vehicle from the deviation of position information by comparing an image captured by a camera with an image having position information stored in a database.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, many in-vehicle camera images are of high resolution, and there is a problem that the processing cost increases when pixel scanning processing or a learning model is used to confirm consistency in image comparison. Therefore, there was room for improvement in technologies related to correction of vehicle position information.

[0005] In view of such circumstances, an object of the present disclosure is to improve technologies related to correction of vehicle position information.

Means for Solving the Problems

[0006] An information processing device according to one embodiment of the present disclosure receives an image of a first location taken by a camera mounted on a moving vehicle and location information of the first location determined by a GPS sensor mounted on the vehicle. The device selects an image of a location within a predetermined distance from the first location from an image database that stores a plurality of images in which location information with a higher accuracy than the positioning accuracy of the GPS sensor is embedded. The device compares the image of the first location with the image of the selected location to calculate the similarity of the images. If the similarity is greater than or equal to a threshold, the device identifies the selected location as a second location. The device also includes a control unit that generates a correction function to correct the location information of the first location based on the difference between the location information of the first location and the location information of the second location.

[0007] An information processing device according to one embodiment of the present disclosure receives an image and location information of a first location from a moving vehicle, selects an image of a location within a predetermined distance from the first location from an image database storing multiple images in which location information with a higher accuracy than a predetermined accuracy is embedded, compares the image of the first location with the image of the selected location to calculate the similarity of the images, identifies the selected location as a second location if the similarity is greater than or equal to a threshold, and includes a control unit that generates a correction function to correct the location information of the first location based on the difference between the location information of the first location and the location information of the second location.

[0008] An information processing method according to one embodiment of the present disclosure involves an information processing device that receives an image and location information of a first location from a moving vehicle; selects an image of a location within a predetermined distance from the first location from an image database that stores a plurality of images in which location information of a higher accuracy than a predetermined accuracy is embedded; compares the image of the first location with the image of the selected location to calculate the similarity of the images; if the similarity is greater than or equal to a threshold, identifies the selected location as a second location; calculates the difference between the location information of the first location and the location information of the second location; and generates a correction function to correct the location information of the first location based on the calculated difference. [Effects of the Invention]

[0009] According to one embodiment of this disclosure, the technology for correcting vehicle location information is improved. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram showing a schematic configuration example of a system according to one embodiment of the present disclosure. [Figure 2] This flowchart shows an example of the operation of an information processing device. [Figure 3] This diagram explains the correction of vehicle location information. [Modes for carrying out the invention]

[0011] (Summary of the embodiment) Referring to Figure 1, an overview of System 1 according to an embodiment of this disclosure will be described. System 1 comprises a vehicle 10 and an information processing device 20. The vehicle 10 and the information processing device 20 are communicably connected to a network 2, which includes, for example, the Internet and a mobile communication network.

[0012] Vehicle 10 is, for example, an automobile, but is not limited to that and may be any vehicle. The automobile may be, but is not limited to, a gasoline car, a BEV (Battery Electric Vehicle), a HEV (Hybrid Electric Vehicle), a PHEV (Plug-in Hybrid Electric Vehicle), or an FCEV (Fuel Cell Electric Vehicle). The number of vehicles 10 provided in System 1 may be determined arbitrarily.

[0013] The information processing device 20 is, for example, a computer such as a server. The information processing device 20 can communicate with the vehicle 10 via the network 2.

[0014] First, an overview of this embodiment will be described, and details will be described later. The information processing device 20 receives an image and location information of a first location from a moving vehicle 10, selects an image of a location within a predetermined distance from the first location from an image database 22A that stores multiple images embedded with location information of a higher accuracy than a predetermined accuracy, compares the image of the first location with the image of the selected location to calculate the similarity of the images, identifies the selected location as the second location if the similarity is above a threshold, and generates a correction function to correct the location information of the first location based on the difference between the location information of the first location and the location information of the second location.

[0015] Thus, according to this embodiment, a correction function is generated to correct the positional information of the first location. Therefore, even if the accuracy of the positional information of the first location is low, it is possible to reduce the processing cost of comparing the image of the first location with the image of the second location.

[0016] Next, we will describe each component of System 1 in detail.

[0017] (Vehicle configuration) As shown in Figure 1, the vehicle 10 includes a communication unit 11, a measurement unit 12, a storage unit 13, and a control unit 14.

[0018] The communication unit 11 includes one or more communication interfaces connected to the network 2. These communication interfaces may, for example, support mobile communication standards such as 4G (4th Generation) or 5G (5th Generation), but are not limited to these. In this embodiment, the vehicle 10 communicates with the information processing device 20 via the communication unit 11 and the network 2.

[0019] The measurement unit 12 includes one or more devices for acquiring the position information of the vehicle 10. Specifically, the measurement unit 12 includes, for example, a GPS sensor 12B that receives radio waves from artificial satellites 3 (GPS satellites) and measures the position information of the location where the vehicle 10 travels. However, it is not limited to the GPS sensor, and sensors corresponding to any satellite positioning system may be provided. The measurement unit 12 further includes a camera 12A that captures an image outside the vehicle 10.

[0020] The storage unit 13 includes one or more memories. Each memory included in the storage unit 13 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 13 stores any information used for the operation of the vehicle 10. For example, the storage unit 13 may store a system program, an application program, and the like.

[0021] The control unit 14 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The processor is, for example, a general-purpose processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), or a dedicated processor specialized for specific processing, but is not limited thereto. The programmable circuit is, for example, an FPGA (Field-Programmable Gate Array), but is not limited thereto. The dedicated circuit is, for example, an ASIC (Application Specific Integrated Circuit), but is not limited thereto. The control unit 14 controls the operation of the entire vehicle 10.

[0022] (Configuration of the information processing device) As shown in FIG. 1, the information processing device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0023] The communication unit 21 includes one or more communication interfaces connected to the network 2. These communication interfaces may support, for example, mobile communication standards, wired LAN (Local Area Network) standards, or wireless LAN standards, but are not limited to these, and may support any communication standard. In this embodiment, the information processing device 20 communicates with the vehicle 10 via the communication unit 21 and the network 2.

[0024] The storage unit 22 includes one or more memories. Each memory included in the storage unit 22 may function as, for example, a main memory, an auxiliary memory, or a cache memory. The storage unit 22 stores any information used in the operation of the information processing device 20. For example, the storage unit 22 may store system programs, application programs, databases, and map information. The storage unit 22 includes an image database 22A.

[0025] The control unit 23 includes one or more processors, one or more programmable circuits, one or more dedicated circuits, or a combination thereof. The control unit 23 controls the operation of the entire information processing device 20.

[0026] (Operation flow of the information processing device) Referring to Figure 2, the operation of the information processing device 20 according to this embodiment will be described.

[0027] S101: The control unit 23 receives an image of the first point p1 taken by the camera 12A mounted on the vehicle 10, and position information p1(x1,y1) of the first point p1 determined by the GPS sensor 12B mounted on the vehicle 10, from the moving vehicle 10.

[0028] Camera 12A is a camera that takes video or still images, for example, installed in a dashcam, and takes "forward only," "forward and rear," or "all-around (360°)" shots. GPS sensor 12B is installed in a car navigation system or dashcam, etc., and determines the position information of vehicle 10. Position information is coordinate information of latitude and longitude (xn,yn). The positioning accuracy δ of GPS sensor 12B installed in vehicle 10 varies depending on the vehicle type, but is about 2m to 3m under good reception conditions.

[0029] S102: The control unit 23 selects an image from an image database 22A that stores multiple images, each containing positional information with a higher accuracy than a predetermined accuracy γ, of a location located within a predetermined distance λ from the first location p1.

[0030] The predetermined accuracy γ is set to 10 cm (10 centimeters) in this disclosure, but is not limited to this, and may be set to the positioning accuracy δ of the GPS sensor 12B. The images stored in the image database 22A are images taken by a special surveying vehicle, and are images in which positioning information with a positioning accuracy higher than the predetermined accuracy γ, for example, about 2 cm, is embedded. The predetermined distance λ is set to 5 m (5 meters) in this disclosure, but is not limited to this, and may be set arbitrarily. As described above, since the positioning accuracy δ of the GPS sensor 12B is about 2 m to 3 m, there is a high probability that the vehicle 10 is actually located within a radius of 2 m to 3 m from the first point p1. For this reason, the control unit 23 decided to select an image in which positioning information of a point within a distance of 5 m from the first point p1, with a margin of 2 m to 3 m, is embedded.

[0031] S103: The control unit 23 compares the image of the first location with the image of the selected location and calculates the image similarity S.

[0032] The similarity score S is the degree to which two images are similar. In this disclosure, the similarity score S is expressed on a scale of 0-100%, with 100% representing the similarity score S when identical images are input.

[0033] S104: The control unit 23 determines whether the similarity S is greater than or equal to the threshold α. If the similarity S is greater than or equal to the threshold α, the process proceeds to S105. If the similarity S is less than the threshold α, the process returns to S102.

[0034] The threshold α is, for example, 90%, but is not limited to this. If the similarity S is less than the threshold α, the process returns to S102 and selects an image that has not yet been determined from a location within a predetermined distance λ from the first location p1. The selection of such images is repeated until it is determined that the similarity S is equal to or greater than the threshold α. If there is no image within the predetermined distance λ from the first location p1 that is determined to have a similarity S equal to or greater than the threshold α, the process returns to S101.

[0035] S105: The control unit 23 identifies the selected location as the second location p2.

[0036] The second point p2 is a location with an error of approximately 2 cm between its latitude and longitude and the actual location of the vehicle 10. Therefore, the control unit 23 estimates that the second point p2 is the actual location of the vehicle 10.

[0037] Figure 3 illustrates the correction of the vehicle's location information p1(x1,y1). As shown in Figure 3, if the similarity S between the image of the first location p1 and the selected image is greater than or equal to the threshold α, the selected location is identified as the second location p2.

[0038] S106: The control unit 23 calculates the difference d between the position information p1(x1,y1) of the first point p1 and the position information p2(x2,y2) of the second point p2.

[0039] The difference in latitude d1 and the difference in longitude d2 can be calculated using the following equations (1) and (2). d1 = x² - x¹ (1) d2 = y2 - y1 (2)

[0040] S107: The control unit 23 generates a correction function f to correct the position information p1(x1,y1) of the first point p1 based on the calculated difference d.

[0041] As shown in Figure 3, once the second point p2 is identified, a correction function f (latitude correction function fx and longitude correction function fy) is generated. The corrected latitude x1' is obtained using the latitude correction function fx in equation (3) below. The corrected longitude y1' is obtained using the longitude correction function fy in equation (4) below. fx = x1 + d1 (3) fy = y1 + d2 (4)

[0042] The control unit 23 may divide the area in which the vehicle 10 travels into multiple areas, classify the vehicle 10 according to the same vehicle type, and generate a correction function f to correct the position information p1(x1,y1) for each group consisting of a combination of a divided area and a classified vehicle type. The control unit 23 may also generate a correction function f(t) at each time t. The reason for generating a correction function f for each area is that there may be differences in the signal strength from the artificial satellite 3 (GPS satellite) for each area. The reason for generating a correction function f for each vehicle type is that there may be differences in the positioning accuracy of the GPS sensor 12B installed for each vehicle type.

[0043] The control unit 23 may, when it receives an image and location information p1(x1,y1) of a first point p1 from multiple vehicles 10 of the same type traveling in the same area, calculate the differences d1 and d2 between the location information p1(x1,y1) and the location information p2(x2,y2) for each vehicle, and then substitute the average values ​​of the calculated differences d1 and d2 into d1 in equation (3) and d2 in equation (4) to correct the location information p1(x1,y1) of the first point p1 for each of the multiple vehicles 10 traveling in the same area.

[0044] S108: The control unit 23 determines whether the number of times the generated correction function f has been updated is greater than or equal to a predetermined number. If the number of updates is greater than or equal to the predetermined number n, the process proceeds to S109. If the number of updates is less than the predetermined number n, the process returns to S101 and the information processing is repeated.

[0045] The reason for setting the number of updates for the correction function f to a predetermined number n or more is to ensure the reliability of the correction function f.

[0046] S109: The control unit 23 registers the updated correction function f.

[0047] S110: After registering the correction function f, when the control unit 23 receives an image of the first point p1 and location information p1(x1,y1) from the vehicle 10, it corrects the location information p1(x1,y1) of the first point p1 using the registered correction function f.

[0048] Refer to Figure 3 again. As shown in Figure 3, it becomes possible to correct the position information p1(x1,y1) of the vehicle 10 by using the correction function f without performing image comparison.

[0049] As described above, the information processing device 20 according to this embodiment receives an image and location information of a first location from a moving vehicle 10, selects an image of a location within a predetermined distance from the first location from an image database 22A that stores multiple images embedded with location information of a higher accuracy than a predetermined accuracy, compares the image of the first location with the image of the selected location to calculate the similarity of the images, identifies the selected location as the second location if the similarity is above a threshold, and generates a correction function to correct the location information of the first location based on the difference between the location information of the first location and the location information of the second location.

[0050] With this configuration, a correction function is generated to correct the positional information of the first location. Therefore, even if the accuracy of the positional information of the first location is low, it is possible to reduce the processing cost of comparing the image of the first location with the image of the second location.

[0051] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art may make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided into two.

[0052] For example, in the embodiment described above, it is also possible to have an embodiment in which the configuration and operation of the information processing device 20 are distributed among multiple computers that can communicate with each other. Also, in the embodiment described above, the image database 22A is provided in the information processing device 20, but it may be provided in a server or the like that is connected to the information processing device 20 via the network 2 so as to be able to communicate with it.

[0053] Furthermore, it is also possible to implement an embodiment in which a general-purpose computer functions as the information processing device 20 according to the above embodiment. Specifically, a program describing the processing content that realizes each function of the information processing device 20 according to the above embodiment is stored in the memory of the general-purpose computer, and the processor reads and executes the program. Therefore, this disclosure can also be implemented as a program that can be executed by a processor, or as a non-temporary computer-readable medium that stores said program. [Explanation of symbols]

[0054] 1 System 2 Network 3,3A―3N Artificial satellite (GPS satellite) 10 vehicles 11 Communications Department 12 Measurement section 12A Camera (Vehicle-mounted camera) 12B GPS sensor 13 Storage section 14 Control Unit 20 Information Processing Devices 21 Communications Department 22 Memory section 22A Image Database (Image DB) 23 Control Unit

Claims

1. Information processing device comprising: receiving an image of a first location taken by a camera mounted on a moving vehicle and location information of the first location determined by a GPS sensor mounted on the vehicle; selecting an image of a location within a predetermined distance from the first location from an image database storing multiple images in which location information with a higher accuracy than the positioning accuracy of the GPS sensor is embedded; comparing the image of the first location with the image of the selected location to calculate the similarity of the images; identifying the selected location as a second location if the similarity is greater than or equal to a threshold; and generating a correction function to correct the location information of the first location based on the difference between the location information of the first location and the location information of the second location.

2. Information processing device comprising: receiving an image and location information of a first location from a moving vehicle; selecting an image of a location within a predetermined distance from the first location from an image database storing multiple images in which location information with a higher accuracy than a predetermined accuracy is embedded; comparing the image of the first location with the image of the selected location to calculate the similarity of the images; identifying the selected location as a second location if the similarity is greater than or equal to a threshold; and generating a correction function to correct the location information of the first location based on the difference between the location information of the first location and the location information of the second location.

3. An information processing apparatus according to claim 2, The control unit is an information processing device that divides the area in which the vehicle travels into multiple areas, classifies the vehicle according to the same vehicle type, and generates a correction function for correcting the location information of the first point for each group consisting of a combination of the divided area and the classified vehicle type.

4. An information processing apparatus according to claim 3, The control unit, upon receiving images and location information of the first location from multiple vehicles of the same type traveling in the same area, calculates the difference between the location information of the first location and the location information of the second location for each vehicle, and generates a correction function to correct the location information of the first location based on the average value of the calculated differences.

5. An information processing apparatus according to claim 2, The control unit registers the updated correction function when the generated correction function has been updated a predetermined number of times or more, and after the registration of the correction function, when it receives an image and location information of the first point from the vehicle, it corrects the location information of the first point using the registered correction function.

6. Information processing equipment Receiving images and location information of the first point from a moving vehicle, Select an image of a location within a predetermined distance from the first location from an image database that stores multiple images, each embedded with location information of a higher accuracy than a predetermined accuracy. The image of the first location and the image of the selected location are compared to calculate the similarity between the images, If the similarity is equal to or greater than the threshold, the selected location is identified as the second location. Calculate the difference between the location information of the first point and the location information of the second point, Based on the calculated difference, a correction function is generated to correct the position information of the first point, An information processing method that performs [this action].

Images