Location estimation system, information terminal used therein, and location estimation method
The system improves position estimation by using wireless signal strength and distance measurements to enhance accuracy in urban environments, ensuring precise vehicle stops and reducing shocks in challenging conditions.
Patent Information
- Application Number
- JP2024126200
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-08-01
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2040-07-31
AI Technical Summary
Existing position estimation methods, such as those using GPS or wireless LAN access points, suffer from inaccuracies due to signal interference and reception limitations, particularly in urban environments, which can be problematic in situations like hailing a taxi where precise location is crucial.
A position estimation system using information terminals that transmit and receive wireless signals with identification information, measuring reception strength and distance traveled to estimate the relative positional relationship between terminals, allowing for improved accuracy through parameter fitting and control of vehicle speed for precise meeting points.
Enhances position estimation accuracy by utilizing wireless signal strength and distance measurements to accurately determine the location of meeting partners, reducing shocks and improving the precision of vehicle stops in challenging environments.
Smart Images

Figure 2024156855000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a position estimation system using an information terminal. [Background technology]
[0002] Background art in the technical field related to obtaining location information is disclosed in Patent Document 1. In Patent Document 1, the user's location is estimated from the location of a nearby wireless LAN (Local Area Network) access point or location measured by a GPS (Global Positioning System). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2005-351859 A Summary of the Invention [Problem to be solved by the invention]
[0004] The position estimation method of Patent Document 1 may not provide sufficient accuracy in position estimation. For example, this may be the case when GPS radio waves are disturbed by reflection from a building, when they cannot be received inside a building, when the user is far away from a wireless LAN access point, or when radio waves from the wireless LAN access point cannot be received. Examples of cases where high position estimation accuracy is desirable include when calling a taxi in a place with a lot of pedestrian and vehicular traffic, when the appearance of the other party is unknown and there are multiple possible people and cars.
[0005] In view of the above-mentioned problems, an object of the present invention is to provide a position estimation system using an information terminal with improved position accuracy. [Means for solving the problem]
[0006] As one example, the present invention is a position estimation system comprising a first information terminal having a function of transmitting a wireless signal accompanied by identification information and a second information terminal having a function of measuring the reception strength of the wireless signal, wherein the second information terminal measures the reception strength of the wireless signal transmitted by the first information terminal, and at least one of the first information terminal and the second information terminal measures the traveled distance, and at least one of the first information terminal and the second information terminal estimates the relative positional relationship between the first information terminal and the second information terminal from the change in the reception strength of the wireless signal with respect to the traveled distance. Effect of the Invention
[0007] According to the present invention, it is possible to provide a position estimation system using an information terminal with improved position accuracy. [Brief description of the drawings]
[0008] [Figure 1] 1 is a schematic configuration diagram of a position estimation system according to a first embodiment. [Diagram 2] FIG. 11 is a diagram illustrating a meeting support operation in a situation where a passenger is waiting for a vehicle such as a taxi on the side of the road in the first embodiment. [Figure 3A] FIG. 11 is a diagram showing the positional relationship between information terminal 1A and information terminal 1B in the first embodiment when information terminal 1B does not move. [Figure 3B] FIG. 11 is a diagram showing the positional relationship between information terminal 1A and information terminal 1B in the first embodiment when information terminal 1B also moves. [Figure 3C] 3A and 3B. This is a projection diagram in which each element is projected in the direction of the movement line 110A on a vertical plane perpendicular to the movement line 110A in FIG. 3A and FIG. 3B. [Figure 4A] 13 is a profile of a reception intensity P0 when there is no reflection relative to a relative movement distance x in the first embodiment. [Figure 4B] 13 is a profile of the reception intensity P including the reflected wave with respect to the relative movement distance x in the first embodiment. [Figure 5A] FIG. 11 is a diagram showing a change in the estimated value x0E of the closest travel distance in the first embodiment. [Figure 5B]10 is a diagram showing measured values of reception intensity P including reflected waves versus relative movement distance x in the first embodiment. FIG. [Figure 6A] 6A to 6C are diagrams illustrating an example of control of acceleration change with respect to position information in the first embodiment. [Figure 6B] 11 is a diagram illustrating an example of control of a speed change with respect to position information in the first embodiment. [Figure 7] 2 illustrates an example of the external configuration of an HMD as an example of an information terminal in the first embodiment. [Figure 8] 8 is a functional block diagram of the information terminal (HMD) of FIG. 7. [Figure 9] FIG. 11 is a control flow diagram for meeting support in the first embodiment. [Figure 10A] FIG. 11 is a diagram for explaining internal data in which reception strength is recorded with respect to measurement time in the first embodiment. [Figure 10B] FIG. 11 is a diagram for explaining internal data in which a travel distance is recorded with respect to a measurement time in the first embodiment. [Figure 10C] 10 is a diagram for explaining internal data in which reception strength is recorded with respect to relative movement distance in the first embodiment. FIG. [Figure 10D] 11 is a diagram for explaining internal data in which an estimated value of the closest movement distance with respect to a relative movement distance in the first embodiment is recorded. FIG. [Figure 11A] FIG. 11 is an explanatory diagram showing an estimated distance to a partner information terminal displayed on a display screen of an information terminal in the second embodiment; [Figure 11B] FIG. 11 is an explanatory diagram showing an AR object mark displayed at an estimated position of a partner information terminal on a display screen of an information terminal in the second embodiment. [Figure 11C] FIG. 11 is an explanatory diagram showing an AR object mark indicating the partner information terminal itself displayed on the display screen of the information terminal in the second embodiment. [Figure 12A] 13 is a diagram showing the positional relationship of each element on a vertical plane including an information terminal 1B in the second embodiment and perpendicular to a movement line 110A. FIG. [Figure 12B] 12B is a diagram showing the positional relationship when each element in FIG. 12A is projected vertically onto a ground surface 111. FIG. [Figure 13A]FIG. 11 is an explanatory diagram of a hardware configuration on a vehicle side in a third embodiment. [Figure 13B] FIG. 13 is an explanatory diagram of another configuration of the vehicle-side hardware in the third embodiment. [Figure 14A] FIG. 11 is an explanatory diagram showing an estimated distance of a passenger displayed on a display screen of an information terminal in the third embodiment. [Figure 14B] FIG. 13 is an explanatory diagram showing an AR object mark displayed at an estimated position of a passenger on a display screen of an information terminal in the third embodiment. [Figure 14C] FIG. 13 is an explanatory diagram showing an AR object mark indicating a passenger displayed on a display screen of an information terminal in the third embodiment. [Figure 15A] FIG. 11 is a diagram showing the positional relationship of each element on a vertical plane including an information terminal 1A in the third embodiment and perpendicular to a movement line 110B. [Figure 15B] 15B is a diagram showing the positional relationship when each element in FIG. 15A is projected vertically onto the ground surface 111. FIG. [Figure 16] FIG. 13 is an explanatory diagram for explaining a method for evaluating the influence of buildings that contribute to radio wave reflection in the fourth embodiment. [Figure 17A] FIG. 13 is an explanatory diagram for switching fitting functions according to the amount of wall surfaces of a building in the fourth embodiment. [Figure 17B] FIG. 13 is an explanatory diagram for switching a fitting function depending on the distance to the meeting place in the fourth embodiment. [Figure 17C] FIG. 13 is an explanatory diagram for switching a predetermined distance TP for switching a fitting function depending on the amount of wall surface of a building in the fourth embodiment. [Figure 18A] FIG. 23 is a diagram for explaining how a car approaching in front at the time when the reception strength becomes maximum is photographed as a waiting car in the sixth embodiment. [Figure 18B] 23 is a diagram for explaining how an image captured from a frontal position is displayed on the screen of an information terminal and a mark is superimposed on the image of the target vehicle in the sixth embodiment. FIG. [Figure 19A] 13 shows a display screen of information terminal 1C in a state in which the other party's information terminal 1A is still far away from the meeting place and information terminal 1C is not receiving a wireless signal in the seventh embodiment. [Figure 19B] 13 shows a display screen of information terminal 1C in a state where portable information terminal 1A in the seventh embodiment is approaching a meeting place and information terminal 1C is receiving a wireless signal. [Figure 20] FIG. 23 is an explanatory diagram showing how the display is changed according to the reception intensity in the seventh embodiment. [Figure 21] 13 is a display screen showing an explanatory message indicating that the position is unknown in the seventh embodiment. [Figure 22] 13 is a display screen displaying a service switching input box in the seventh embodiment. [Diagram 23] FIG. 23 is an explanatory diagram of a voice notification corresponding to the reception strength of a wireless signal in the seventh embodiment. [Figure 24] FIG. 23 is an explanatory diagram of vibration notification corresponding to the reception strength of a wireless signal in the seventh embodiment. [Diagram 25] 13 is a display screen of an information terminal 1A in the seventh embodiment. [Figure 26A] 13 is a display example of a counterpart information terminal image of an information terminal 1C in the eighth embodiment. [Figure 26B] 13 is a display example in which an image of a partner information terminal is superimposed on a map display screen of an information terminal 1C in the eighth embodiment. [Figure 27] FIG. 23 is a diagram illustrating a modified example of the person identification method in the eighth embodiment. [Figure 28] 13 is a display example of a recognition result of an information terminal 1C by a counterpart information terminal 1A in the eighth embodiment. [Figure 29] 13 is a display format for listing person images for correcting a person to be met in the eighth embodiment. [Diagram 30] 13 is a display screen of an information terminal 1A in the eighth embodiment. [Diagram 31] FIG. 23 is a control flow diagram for meeting support in the eighth embodiment. [Diagram 32] 13 is a display screen of an information terminal 1B in a ninth embodiment. [Diagram 33] FIG. 13 is an explanatory diagram of transmission information of an optical signal of a light emitting device in a ninth embodiment. [Diagram 34]FIG. 13 is a control flow diagram for meeting support in the ninth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. EXAMPLES
[0010] In this embodiment, a location estimation system is described by assuming a case where a passenger calls a taxi and provides meeting support to confirm the other party. In the following description, an information terminal may be simply referred to as a terminal. Also, meeting support may be simply referred to as support.
[0011] Fig. 1 is a schematic diagram of a position estimation system according to the present embodiment. In Fig. 1, a wireless signal source with identification information is held by an information terminal 1A on the vehicle side. Note that the vehicle itself may be the information terminal 1A, or the information terminal 1A may be mounted on the vehicle.
[0012] A passenger has at least one of the following information terminals: information terminal 1B which is an HMD (Head Mount Display), information terminal 1C which is a smartphone, and information terminal 1D which is a smartwatch. The following operations of the passenger side terminals may be performed by these terminals alone or in cooperation with each other. Here, an example is shown in which information terminal 1B which is an HMD operates alone as the passenger side terminal. Also, information terminals 1A to 1D are collectively referred to as information terminal 1.
[0013] The server 2 performs processing on behalf of each information terminal 1, mediates the transmission and reception of information between each information terminal 1, and provides necessary information via a communication network 9. The server 2 may be, for example, a local server, a cloud server, an edge server, a net service, or the like, and its form is not important.
[0014] FIG. 2 is a diagram for explaining the meeting support operation in a situation where a passenger is waiting for a vehicle such as a taxi on the side of the road in this embodiment. In FIG. 2, information terminal 1A is the vehicle itself or an information terminal mounted on the vehicle. Information terminal 1A moves along a movement line 110A on the road and approaches information terminal 1B carried by the passenger. Here, movement line 110A refers to the path that information terminal 1A has taken in the past and the path that it is scheduled to take in the future. For the sake of explanation, the passenger is assumed to be stationary at first, but a generalized case where the passenger moves along movement line 110B on the side of the road will also be described later.
[0015] The information terminal 1A continues to transmit a wireless signal accompanied by identification information while moving. Here, the wireless signal accompanied by identification information may be a signal that can be identified by directly transmitting and receiving a wireless signal without going through a repeater or the like. For example, it may be a beacon signal, or a signal such as Bluetooth (registered trademark), BLE (Bluetooth low energy), WiFi (registered trademark), or a point-to-point communication signal of local 5G. In addition, this wireless signal may be used to communicate between the information terminals 1.
[0016] The moving information terminal 1 measures its own position. The measurement may be performed by a positioning system using radio waves such as GPS, a positioning system based on a position reference point, or by integrating the output of an acceleration sensor. The position may be recorded as the distance traveled from an appropriately determined starting point for each time. It is also possible to calculate the speed from the data of the distance traveled for each time. Hereinafter, the position of the information terminal 1A will be expressed as the distance traveled from the starting point. When this position information is to be used by the other information terminal 1B, it is provided from the information terminal 1A to the information terminal 1B. Meanwhile, the information terminal 1B receives the wireless signal and measures the reception strength.
[0017] When both information terminals 1 move, the change in the relative moving distance is used from the moving distance of both. Either information terminal may be the transmitting side of the wireless signal. Furthermore, both may transmit a wireless signal and both may receive it.
[0018] Fig. 3A is a diagram showing the positional relationship between information terminal 1A and information terminal 1B in this embodiment when information terminal 1B does not move. Fig. 3A is a projection diagram in which each element is projected vertically onto a horizontal plane, and hereafter, unless otherwise specified, when it is expressed as a projection diagram onto a horizontal plane, it means a diagram in which each element is projected vertically. Also, the symbols of each element may be the same as the symbols before projection on the projection diagram. Also, movement lines 110A, 110B are approximated by straight lines. Also, only the positions of information terminals 1A, 1B are shown.
[0019] In FIG. 3A, the moving distance 114A from the starting point 112A of the information terminal 1A is x A The starting point 112A may be determined arbitrarily. For example, the position of the information terminal 1A at the time when the information terminal 1A and the information terminal 1B start communication for rendezvous is determined as the starting point 112A. Even if a point that has never actually been there is determined as the starting point 112A, the travel distance from the starting point 112A is expressed here as 114A.
[0020] Also, the distance (closest movement distance) that information terminal 1A must move from the starting point until it reaches the position (closest position) closest to information terminal 1B on movement line 110A is denoted as x0. Before arriving at the closest position, the closest position is estimated from data on the reception strength of the wireless signal versus the movement distance (details will be described later). Also, the distance 113 between information terminal 1B and movement line 110A is denoted as d.
[0021] FIG. 3B is a diagram showing the positional relationship between information terminal 1A and information terminal 1B in this embodiment when information terminal 1B also moves, and is a projection onto a horizontal plane similar to FIG. 3A.
[0022] In FIG. 3B, information terminal 1B moves toward information terminal 1A along movement line 110B. Let x be the movement distance 114B of information terminal 1B along movement line 110B from an arbitrarily determined starting point 112B. BIt is expressed as follows. The movement line 110A and the movement line 110B are approximately parallel. The distance between the starting point 112A and the starting point 112B is the closest movement distance x0. Note that the movement distance here is an amount used to express the distance from the starting point, and depending on how the starting point is defined, it does not necessarily match the distance that the information terminal 1 actually moves while the meeting support operation is being performed.
[0023] FIG. 3C is a projection diagram in which each element is projected in the direction of the movement line 110A on a vertical plane perpendicular to the movement line 110A in FIG. 3A and FIG. 3B. Hereinafter, unless otherwise specified, when the expression is expressed as a projection diagram on a vertical plane perpendicular to the movement line, it means a diagram in which each element is projected in the direction of the movement line. This diagram is common to both the case where the information terminal 1B moves and the case where it does not move. The ground surface 111 is approximated as a horizontal plane. The distance 115A between the information terminal 1A and the ground surface 111 is h A and the distance 115B between the information terminal 1B and the ground surface 111 is expressed as h B It is written as follows.
[0024] In the following discussion, it is generalized and assumed that the information terminal 1B also moves. The relative movement distance x is defined by the following equation (1), and the profile of the received signal strength with respect to the relative movement distance is considered. x=x A +x B …(1) If the information terminal 1B does not move, press x. B = 0.
[0025] 3C, 120 indicates the path of the direct wave of the wireless signal, and 121 indicates the path of the reflected wave reflected by the ground surface 111. The received wireless signal includes signals reflected by the ground and nearby structures, but first consider the profile of the reception strength that does not include the reflected wave.
[0026] The received strength P0 of a wireless signal (direct wave) when there is no reflection is inversely proportional to the square of the distance between the transmission source (information terminal 1A) and the receiver (information terminal 1B), and is therefore expressed by the following equation (2). P0 = k0 / {(x-x0) 2 +g0 2} …(2) Here, k0 is a constant that depends on the signal source strength and the receiver sensitivity, and g0 is defined by the following equation (3). g0={d 2 +(h A -h B ) 2} 1 / 2 …(3) g0 is the closest distance in the direct wave path 120.
[0027] For example, h A and B When d is 5m and x0 is 50m, the profile of P0 is as shown in Figure 4A. g0 is 5m. Here, it is normalized so that P0 = 1 at the closest position (x = x0). As can be seen from Figure 4A, P0 is a profile with a sharp peak at the closest position.
[0028] In reality, the reception intensity profile differs from P0 because there are components reflected by the ground surface 111 and nearby buildings. The position at which the reception intensity of the reflected components is at its maximum depends on the position of the reflector, but on average it can be expected to be near the closest position. Also, since the path of the reflected wave is longer than the direct wave path 120, when the reflected wave is also taken into account, the profile will effectively have a larger closest distance. Therefore, the reception intensity profile including the reflected wave will also be explained by approximating it using the following formula (4). P=k / {(x-x0) 2 +g 2} …(4) Here, k is a constant that depends on the signal source strength and the receiver sensitivity, and g is the effective closest distance. If this effective closest distance is the closest distance of the reflection path 121 reflected by the ground surface 111, which is likely to be the strongest reflected wave, then the following equation (5) is obtained. g={d 2 +(h A +h B ) 2} 1 / 2 …(5) Here, d=5m, h A =h B= 1.6 m, then g = 5.9 m, and the closest distance in this case is not significantly different from the closest distance for a direct wave. This value of g is calculated as an estimate (see below), but it may also be used as an approximation of d and g0.
[0029] Figure 4B shows the profile of equation (4). There is no significant difference in the profile compared to Figure 4A. For this reason, we believe that the approximation of replacing the closest distance with the effective distance in the function form of the direct wave is appropriate as the actual received intensity profile.
[0030] Next, a method for estimating a position will be described based on the reception strength profile in Fig. 4B. Specifically, the parameters in equation (4) are estimated from the measured reception strength on the way to the closest position by parameter fitting using the least squares method or the like. As an example, the reciprocal of the reception strength is a quadratic function of the relative movement distance x, so fitting is performed using this function. 1 / P=(1 / k){(x-x0) 2 +g 2}=Ax 2 +Bx+C …(6) Equation (6) is linear with respect to parameters A, B, and C, and is calculated from the reception strength data by the least squares method. Here, when calculating by the least squares method, the measurement error of the reception strength P is assumed to be constant, and the error of 1 / P is P -2 As a result, the estimated values of each parameter in equation (4) are calculated as follows: k=1 / A …(7) x0=-B / (2A) …(8) g = (4AC-B 2 ) 1 / 2 / (2A) …(9) The parameter fitting method is not limited to the above method.
[0031] The values of the parameters k, x0, and g are updated as the measurement data of the reception strength is added while moving, and errors due to noise are reduced.
[0032] The estimated closest moving distance x0 is recalculated as x0E Let x 0E The change in is shown in FIG. 5A. FIG. 5B shows the measured value 200 of the reception strength and the reception strength profile 201 estimated by fitting. For example, the point at x=30 m on the graph in FIG. 5A is the x value estimated from the measured value of the reception strength P up to x=30 m shown in FIG. 5B. 0E It is not necessary to use all past measurements for estimation. When approaching the closest position and the accumulated measurement values with small errors increase, the measurement values with large errors at the beginning of the measurement are denoted by x 0E It may be possible not to use it in estimating
[0033] Estimated value x from reception strength 0E While the noise effect on x is large, the vehicle will control its speed to target the location specified at the time of reservation. As it approaches the nearest position, the estimated value x 0E After the noise effect on the vehicle becomes small, the vehicle controls its speed to stop at the closest position to the passenger. The vehicle can be controlled by the driver or an automatic driving device.
[0034] The control that is desirable for the vehicle control device is as follows. First, information on the position where the radio signal reception strength is maximum (x 0E ) and the location information of both information terminals 1 (x A , x B ) is provided to the vehicle control system. Based on this information, the vehicle control system performs control to stop the vehicle at the closest position by monotonically decreasing the magnitude of speed and acceleration in a continuous period up to and including the point of stopping, with the goal of stopping the vehicle at the closest position. This allows the vehicle to stop accurately at the closest position, while also lowering the magnitude of acceleration when the vehicle is stopped, thereby reducing the shock felt by passengers when the vehicle is stopped. An example of control is shown in Figures 6A and 6B. Figure 6A shows the acceleration change, and Figure 6B shows the speed change.
[0035] An example of speed control is shown below, assuming that a passenger approaches at a constant speed. In order to reduce the shock when the car stops, the speed is controlled with the goal of making the acceleration zero at the same time as the speed of the car becomes zero. Therefore, as an example, it is assumed that the passenger's moving speed is constant from a certain point onwards, and the acceleration of the car is kept constant, and the car control device sets the acceleration at that point so that the speed and acceleration of the car become zero at the closest moving distance position. If the speed of the information terminal 1A is v A , acceleration is a A Let the speed of information terminal 1B be v B Then, a A Specifically, the setting is as follows: a A =-2v A (v A / 3+v B ) / (x 0E -x A -x B ) …(10) With this acceleration setting, the vehicle controller controls the vehicle speed.
[0036] At this time, the distance the car travels to the nearest position is x R (hereafter referred to as the target distance) is x R =(x 0E -x A -x B ) / (1+3v B / v A ) …(11) This results in equation (11).
[0037] When passengers are stopped, B = 0. Also, the passenger speed is not always constant, and x 0E Since the value of may also change, we obtain the value of the right hand side of equations (10) and (11) at each time.
[0038] When the driver controls the vehicle, the information terminal 1A detects a target distance x R The driver may then use this value as a guide to control the speed.
[0039] For safety reasons, the acceleration a is set to avoid sudden acceleration or deceleration. A A predetermined upper limit may be set for the absolute value of
[0040] Here, there may be an estimation error or a prediction error, such as when the user with information terminal 1B stops at first but starts to approach as information terminal 1A approaches. In this case, the reception intensity profile is checked, and if the peak value is passed, the car should be stopped immediately by controlling the speed within a safe range.
[0041] As a result, it is possible to estimate the location of the meeting place more precisely based on the reception strength of the wireless signal.
[0042] Fig. 7 shows an example of the external configuration of an HMD as an example of an information terminal in this embodiment. In Fig. 7, the HMD is provided with a display device including a display surface 11 in a glasses-like housing 10. This display device is, for example, a transmissive display device, and a real image of the outside world is transmitted to the display surface 11, and an image is superimposed and displayed on the real image. A controller, a camera 12, a distance measuring sensor 13, other sensor units 14, etc. are mounted on the housing 10.
[0043] The camera 12 has, for example, two cameras arranged on the left and right sides of the housing 10, and captures an image by capturing an area including the front of the HMD. The distance measurement sensor 13 is a sensor that measures the distance between the HMD and an object in the outside world. The distance measurement sensor 13 may use a TOF (Time Of Flight) type sensor, a stereo camera, or other types. The sensor unit 14 includes a group of sensors for detecting the position and orientation state of the HMD. On the left and right sides of the housing 10, there are provided an audio input device 18 including a microphone, an audio output device 19 including a speaker and an earphone terminal, and the like.
[0044] The information terminal 1 may be provided with an operation device 20 such as a remote controller. In this case, the HMD performs, for example, short-distance wireless communication with the operation device 20. A user can input instructions related to the functions of the HMD and move a cursor on the display surface 11 by operating the operation device 20 with his or her hand. The HMD may communicate with an external smartphone, PC, or the like to cooperate with the HMD. For example, the HMD may receive AR (Augmented Reality) image data from an application on the smartphone.
[0045] The information terminal 1 may display a virtual image such as an AR image on the display surface 11. For example, the information terminal 1 generates a virtual image for guiding the user and displays it on the display surface 11.
[0046] Fig. 8 is a functional block configuration example of the information terminal 1 (HMD) in Fig. 7. Basically, other information terminals 1 have the same configuration. The information terminal 1 includes a processor 101, a memory 102, a camera 12, a distance measuring sensor 13, a sensor unit 14, a display device 103, a communication device 104, a voice input device 18 including a microphone, a voice output device 19 including a speaker or the like, a vibration generating device 1100 such as a vibrator, a light emitting device 3000, a wireless transmitting device 105, a wireless receiving device 106, an operation input unit 107, and a battery 108. These elements are connected to each other via a bus or the like.
[0047] The processor 101 is composed of a CPU, a ROM, a RAM, etc., and constitutes a controller of the HMD. The processor 101 executes processes according to a control program 31 and an application program 32 in the memory 102, thereby realizing functions of an OS, middleware, application, etc., and other functions. The memory 102 is composed of a non-volatile storage device, etc., and stores various data and information handled by the processor 101, etc. The memory 102 also stores images and detection information acquired by the camera 12, etc., as temporary information.
[0048] The camera 12 converts light incident from a lens into an electrical signal using an imaging element to obtain an image. When the distance measurement sensor 13 uses, for example, a TOF (Time Of Flight) sensor, it calculates the distance to an object from the time it takes for light emitted to the outside world to hit the object and return. The sensor unit 14 includes, for example, an acceleration sensor 141, a gyro sensor (angular velocity sensor) 142, a geomagnetic sensor 143, and a GPS receiver 144. The sensor unit 14 detects the position, orientation, movement, and other conditions of the HMD using detection information from these sensors. The HMD is not limited to these, and may also include an illuminance sensor, a proximity sensor, an air pressure sensor, and the like.
[0049] The display device 103 includes a display drive circuit and a display surface 11, and displays a virtual image or the like on the display surface 11 based on image data of the display information 34. Note that the display device 103 is not limited to a transmissive display device, and may be a non-transmissive display device or the like.
[0050] The communication device 104 includes a communication processing circuit, an antenna, and the like corresponding to various predetermined communication interfaces. Examples of the communication interface include a mobile network, Wi-Fi (registered trademark), BlueTooth (registered trademark), infrared, and the like. The communication device 104 performs wireless communication processing between other information terminals 1 and access points. The communication device 104 also performs short-range communication processing with a controller.
[0051] The wireless transmitting device 105 transmits a wireless signal accompanied by an identification signal used for position estimation. Here, the wireless signal accompanied by identification information is, for example, a signal such as Bluetooth. The wireless receiving device 106 receives the wireless signal accompanied by the identification signal and measures the reception strength. Information communication may be performed using the wireless transmitting device 105 and the wireless receiving device 106.
[0052] Moreover, in the case of an information terminal 1 that only transmits or only receives wireless signals, it is sufficient if it is equipped with the necessary devices.
[0053] The voice input device 18 converts input voice from a microphone into voice data. The voice output device 19 outputs voice from a speaker or the like based on the voice data. The voice input device may have a voice recognition function. The voice output device may have a voice synthesis function.
[0054] The vibration generator 1100 has a function of generating vibrations to notify the user. The light emitting device 3000 has a function of transmitting a code by intensity-modulated light. Note that near-infrared light may be used so as to be unnoticeable.
[0055] The operation input unit 107 is a part that accepts operation inputs to the HMD, such as power on / off, volume adjustment, etc., and is composed of hardware buttons, touch sensors, etc. The battery 108 supplies power to each unit.
[0056] The controller by the processor 101 has, as an example of a configuration of functional blocks realized by processing, a communication control unit 101A, a display control unit 101B, a data processing unit 101C, and a data acquisition unit 101D.
[0057] The memory 102 stores a control program 31, an application program 32, setting information 33, display information 34, position estimation information 35, etc. The control program 31 is a program for realizing an estimation of the relative positional relationship between the information terminals 1. The application program 32 is a program for realizing a guidance function for the user. The setting information 33 includes system setting information and user setting information related to each function. The display information 34 includes image data and position coordinate information for displaying a virtual image on the display surface 11. The position estimation information 35 is information related to the moving distance of the information terminal 1 and the reception strength of a wireless signal for performing position estimation.
[0058] The communication control unit 101A controls communication processing using the communication device 104, such as when communicating with another information terminal 1. The display control unit 101B controls the display of a virtual image or the like on the display surface 11 of the display device 103, using the display information 34.
[0059] The data processing unit 101C reads and writes the position estimation information 35, and performs operations such as estimating the relative positional relationship between the terminal itself and the other terminal.
[0060] The data acquisition unit 101D acquires the strength of a wireless signal from the wireless receiving device 106, and acquires each detection data from various sensors such as the camera 12, the distance measuring sensor 13, and the sensor unit 14. The data acquisition unit 101D estimates its own position and measures the travel distance from the detection data of the various sensors.
[0061] FIG. 9 is a control flow diagram of the rendezvous support in this embodiment. In FIG. 9, before this flow, the approximate location of the rendezvous is decided, and the receiving information terminal 1 acquires the identification information of the wireless signal. Either information terminal may be the transmitting side, and wireless signals may be transmitted from both. When both information terminals 1 enter the vicinity of the rendezvous place, first, in steps S1A and S1B, communication between the information terminals 1 is established, and a trigger is set to start the rendezvous support. In addition, the moving information terminal 1 sets starting points 112A and 112B for measuring the moving distance. This communication may be via a communication network, or may be direct communication between the information terminals 1. After that, a position estimation loop is entered.
[0062] In the position estimation loop, first, in steps S2A and S2B, the transmitting information terminal 1 transmits a wireless signal, and the receiving information terminal 1 receives the wireless signal and measures the reception strength. As a result, the receiving information terminal 1 generates internal data that records the reception strength at the measurement time, for example, as shown in FIG. 10A. When both information terminals transmit wireless signals, two radio wave strength profiles are obtained as a result. Accordingly, two types of position estimation are possible, and both estimation results may be used, or only one result may be used. Furthermore, the moving information terminal 1 measures the movement distance, and generates movement distance data, for example, as shown in FIG. 10B.
[0063] In steps S3A and S3B, each information terminal 1 transmits reception intensity data and movement distance data to the other information terminal 1. Data does not need to be transmitted to information terminals 1 that do not perform position estimation.
[0064] In steps S4A and S4B, the information terminal 1 performing the position estimation generates a reception strength profile for the relative movement distance as shown in FIG. 10C from the reception strength data and the movement distance data. Then, the information terminal 1 performing the position estimation generates an estimated value x of the closest movement distance as shown in FIG. 10D, for example, by the above-mentioned method. 0E In this case, if both information terminals 1 transmit wireless signals and the other information terminal 1 receives the wireless signals, two types of reception intensity profiles are obtained. In this case, both types may be used, or only one may be used. Note that the location estimation in the information terminal 1 starts after at least the amount of data required to execute the location estimation has been accumulated.
[0065] In steps S5A and S5B, the information terminal 1 that has performed the position estimation transmits the estimated value x of the closest movement distance calculated by the own terminal to the other information terminal 1. 0E If the other terminal does not need the information, it does not need to be transmitted.
[0066] In steps S6A and S6B, each information terminal 1 provides information for supporting the rendezvous to the user or the vehicle control device. The manner in which this information is provided will be described in detail in the following embodiment.
[0067] In steps S7A and S7B, when both parties confirm the other party's terminal, i.e., the waiting car or the waiting passenger, and the need for meeting support is eliminated, the meeting support is terminated, but if necessary, a branch is made to continue the support.
[0068] As described above, according to this embodiment, it is possible to support rendezvous with improved positional accuracy.
[0069] The method of this embodiment is not limited to meeting people and cars, but can also be applied to meeting people. In this embodiment, the wireless signal source with the identification information is described as being held by the information terminal, but it may be a case where the information terminal does not have the function of an information terminal and only has a wireless signal source. For example, it can be used to support a child who is lost by holding only a wireless signal source. EXAMPLES
[0070] In this embodiment, the support provided to the user on the passenger side, that is, on the information terminal 1B, will be described.
[0071] The information terminal 1B acquires information necessary for support according to embodiment 1. The support contents according to this embodiment are also the same when the information terminal 1A is mounted in a car and the driver receives support by looking at the display of the information terminal 1A.
[0072] First, as a simple method, the information terminal 1B displays the estimated distance of the other information terminal 1A on the display screen to provide the user with a clue to find the car. FIG. 11A shows this. In FIG. 11A, 500 is the display screen of the information terminal 1B, and 501 is an explanatory message for support. The estimated distance x d is given by the following equation (12). x d =x 0E -x A -x B …(12) If you have a see-through HMD, you can see the road as it is. If you have other information terminals, you can take pictures of the road with a camera and display them.
[0073] 11B, the information terminal 1B may display an AR object mark 502 at the estimated position of the car, and may simultaneously display an explanatory message 503.
[0074] Furthermore, if there are no other candidate cars or if the car of the person to be met can be specified from the characteristics of the car (model, color, license plate number, etc.), the information terminal 1B may display an AR object mark 504 representing the car itself as shown in Fig. 11C. Also, an explanatory message 505 may be displayed at the same time. Furthermore, if there are multiple candidates, the AR object marks 504 may be displayed on all of the candidate cars.
[0075] Furthermore, when communication between the information terminals is started, or when it is estimated that the distance between the two is equal to or less than a predetermined distance, the information terminal 1B may notify the user by vibration, etc. By doing so, the user does not need to pay attention to the location of the car until the car approaches.
[0076] The above is an example of support for passengers, but similar support is possible for the driver of the car (described in detail later). In that case, the image on the display of the information terminal 1A will be the passenger. Also, characteristic information of the passenger (e.g., a photo) may be used as an aid for estimation.
[0077] Additionally, external characteristic information of the passenger or vehicle acquired from the server may be used in conjunction with position estimation on the display screen.
[0078] In order to display the marks as in the cases of FIG. 11B and FIG. 11C, the estimated distance x d It is necessary to estimate the position on the road corresponding to the vehicle. Specifically, the position is estimated as follows. The following estimation method is also used to estimate the passenger's position from the vehicle side. In that case, each geometric element is replaced with the geometric element of the other side.
[0079] First, the information terminal 1B measures the distance d (113 in FIG. 3A) between the moving line 110A (FIG. 3A) of the information terminal 1A and the information terminal itself. B (115B in FIG. 3C) is assumed to be known by the user's settings, etc. B Instead of the value of h, the value of h is calculated from the user's height data recorded on the information terminal 1 or the server. BThe value of may be estimated.
[0080] Distance d is estimated as shown in FIG. 12A. FIG. 12A is a diagram including information terminal 1B, showing the positional relationship of each element on a vertical plane perpendicular to movement line 110A. Movement line 110A is projected vertically onto ground surface 111 to form a projection line 130A. Projection line 130A cannot be determined accurately unless the vehicle is identified, but as the vehicle is assumed to be driving in the center of the lane, it is unlikely that there will be a large error, so the center line of the lane is used as projection line 130A. Information terminal 1B estimates that perpendicular line 131B is drawn from its own terminal to projection line 130A, and calculates angle 133B (φ B The angle φ is measured based on the camera image and the attitude information of the information terminal 1B. B Knowing this, the distance d is d=h B tanφ B …(13) It is calculated using equation (13).
[0081] When this distance d is known, the relationship between a point along the projection line 130A and the direction of that point can be known as follows. FIG. 12B is a diagram showing the positional relationship of each element projected vertically onto the ground surface 111. In FIG. 12B, point 1B represents the point where the position of the information terminal 1B is projected vertically onto the ground surface 111. p 0B (134B) is the foot of the perpendicular line drawn from 1B to the projection line 130A. And, on the ground 111 of FIG. 12B, 1B and p 0B The line segment (137B) connecting the above and the estimated position p B The angle (139B) between the line segment (138B) and 1B is θ B Let us assume that.
[0082] By the quantities defined above, p 0B and point p on projection line 130A B Distance between (135B) x pB is given by the following equation (14). x pB = d tan θ B =hB tanφ B tan θ B …(14) Conversely, the direction of the other party's information terminal 1A can be found using this equation (14).
[0083] According to the first embodiment, the information terminal 1B estimates the distance x d The estimated position p of the other party's information terminal 1A on the projection line 130A can be obtained. B teeth, x pB =x d …(15) This is the point where equation (15) holds.
[0084] By simultaneously solving the equations (12), (14), and (15), the angle θ B is calculated as follows: θ B =tan -1 {(x 0E -x A -x B )} / h B tanφ B} …(16) Or, φ B If it is difficult to measure, use the estimated value of g (equation (9)) as an approximation of d, θ B =tan -1 {(x 0E -x A -x B )} / g} …(17) As shown in equation (17), θ B You may ask for:
[0085] In FIG. 11B, the information terminal 1B is B The AR object mark 502 may be displayed superimposed on a point on the road in the direction of the arrow.
[0086] In FIG. 11C, the information terminal 1B is BA search is made for cars that are waiting for a meeting near a point on the road in the direction of the arrow, and when a candidate car is identified, an AR object mark 504 is displayed superimposed on the car.
[0087] According to this embodiment, the vehicle to be met can be confirmed before it arrives. EXAMPLES
[0088] In this embodiment, details of the meeting support on the vehicle side will be described.
[0089] As a hardware configuration on the vehicle side, as shown in Fig. 13A, a vehicle control device has the functions of the information terminal 1A, and the vehicle control device displays information to the user (driver). Alternatively, as shown in Fig. 13B, a configuration is available in which the control information of the vehicle and information used for display to the user are obtained from the information terminal 1A. Note that the driver may view the display of the information terminal 1A.
[0090] When the vehicle control device controls the vehicle speed, the control is performed in the manner described in the first embodiment.
[0091] The vehicle control device or information terminal 1A may present information to the vehicle passengers. For example, the display may be as shown in Figs. 14A, 14B, and 14C. Figs. 14A, 14B, and 14C are the same as those in the second embodiment, so details will be omitted. 600 is a display screen of a display device, which displays an estimated distance of a passenger, etc. The state of the outside world may be displayed as a video image, or may be visible on a HUD (Head Up Display). 601, 603, and 605 are explanatory messages superimposed on the screen. 602 and 604 are marks of AR objects.
[0092] Furthermore, external characteristic information of the passenger obtained from the server may also be used in estimating the position on the display screen.
[0093] Angle θ of the estimated passenger position from the vehicle side AFrom the same consideration as in the second embodiment, is given by the following equation (18). θ A =tan -1 {(x 0E -x A -x B )} / h A tanφ A} …(18) Here, φ A is defined as shown in FIG. 15A.
[0094] FIG. 15A is a diagram including information terminal 1A, showing the positional relationship of each element on a vertical plane perpendicular to movement line 110B (a line in the case where information terminal 1B is not moving, in which case it is assumed that information terminal 1B moves parallel to movement line 110A of information terminal 1A). A projection line 130B is defined as a line obtained by vertically projecting movement line 110B onto ground surface 111. Information terminal 1A projects perpendicular line 131A from its own terminal to projection line 130B, and calculates angle 133A (φ A ) is measured. A The measurement is performed using the camera image and the camera attitude information. Here, when the information terminal 1B, i.e., the passenger, cannot be identified, the projection line 130B is estimated assuming a position where the passenger would normally stand on the side of the road. For example, it is acceptable to assume that the center line of the sidewalk on the side of the road is the projection line 130B since this does not result in a large error.
[0095] In addition, θ A is defined as shown in FIG. 15B. FIG. 15B is a diagram showing the positional relationship of each element projected vertically onto ground surface 111. In FIG. 15B, point 1A represents the point where information terminal 1A is projected onto ground surface 111. p 0A (134A) is the foot of the perpendicular line drawn from 1A to the projection line 130B. And, on the ground 111 of FIG. 15B, 1A and p 0A and the estimated position p of the other party's information terminal 1B on the projection line 130B. A The angle (139A) of the line segment (138A) connecting 1A and A It is.
[0096] Also, hA is the height of the information terminal 1A, more specifically, the height of the wireless signal transmission source of the information terminal 1A from the ground 111. A This information may be stored in the information terminal 1 A, or may be held in the server 2 .
[0097] Or, θ A As in the case of the formula (17) in the second embodiment, the formula calculated by the following formula (19) may be used. θ A =tan -1 {(x 0E -x A -x B )} / g} …(19) If there is no measurement error, θ A =θ B It becomes.
[0098] Also, similarly to the second embodiment, when communication between the information terminals 1 is started, or when it is estimated that the distance between the two is equal to or less than a predetermined distance, the information terminal 1A may notify the user (driver) by voice or the like. By doing so, the user does not need to pay attention to the position of the passenger until the user is close to the passenger.
[0099] According to this embodiment, it is possible to confirm passengers who are waiting before arriving. EXAMPLES
[0100] In this embodiment, correction of the estimated value depending on the situation of the meeting place will be described.
[0101] When there are many buildings and the reflected components of the wireless signal are large, the reception strength profile may be distorted from the functional form of the above-mentioned formula (4). To incorporate the effect of this distortion, in places with many buildings, the reception strength profile is set to the following formula (20) to estimate the closest position. P=k / {α(x-x0) 3 +(x-x0) 2 +g 2} …(20) In equation (20), the third order term represents the effect of distortion.
[0102] Hereinafter, the function form of equation (4) is called L0 type, and the function form of equation (20) is called L1 type. Parameter fitting of L1 type is also performed for 1 / P. As in the first embodiment, when calculating by the least squares method, the measurement error of the reception strength P is assumed to be constant, and the error of 1 / P is assumed to be P -2 It is also possible to perform weighted processing by assuming that the .lambda. 1 / P=(1 / k) {α(x-x0) 3 +(x-x0) 2 +g 2} =A1x 3 +B1x 2 +C1x+D1…(21) Below, we estimate the closest position, that is, x where the reception strength is maximized, and the extreme point x where the reception strength is maximized is the extreme point x where the reception strength is minimized in equation (21). Since the first-order differential of equation (21) is given by the following equation (22), it can be seen that the first-order differential value of equation (21) is 0 at x0. (1 / P)´=(1 / k) {3α(x-x0) 2 +2(x-x0)} …(22) Therefore, among the estimated values of x0 below, the x0 for which the second derivative of equation (21) is positive is the x for which the received signal strength is maximized.
[0103] Now, equation (21) is linear with respect to parameters A1, B1, C1, and D1, and is found from the reception intensity data by the least squares method. The relationship between parameters A1, B1, C1, and D1 and the parameters in equation (21) is as follows: α / k=A1…(23) (1-3αx0) / k=B1…(24) -(2x0-3αx0 2 ) / k=C1…(25) (x0 2 +g 2 -αx0 3 ) / k=D1…(26) By solving equations (23), (24), and (25) simultaneously, we get two x0 values as shown below. x0={-B1±(B12 -3A1C1) 1 / 2} / (3A1) …(27) On the other hand, the second derivative of (21) is given by the following equation (28). (1 / P)´´=6A1x+2B1…(28) The second derivative expressed by the inner equation (28) is taken as positive as follows: x0={-B1+(B1 2 -3A1C1) 1 / 2} / (3A1) …(29) Where: J=B1 2 -3A1C1…(30) When J>0, x0 is an extreme value. When J≦0, this means that equation (21) does not have an extreme value, so L1 fitting cannot be used.
[0104] With x0 being the value of equation (29), the values of the other parameters α and g are shown in equations (31) and (32). α = (1 / 3)(2x0-C1 / B1) / (x0 2 -(C1 / B1)x0) …(31) g = (αx0 3 -x0 2 -αD1 / A1) …(32) Hereafter, we use x0 in equation (29) as the estimate of x that gives the maximum received signal strength, assuming that J>0. 0E As such, the processing and control in the other embodiments are carried out.
[0105] In addition, since the L1 type approximation becomes poor in the range of x where the absolute value of x-x0 is large, the reception strength data used in fitting using the L1 type may be limited to data close to the estimated x0.
[0106] Next, a method for evaluating the amount of the wall surface of a building that contributes to the reflection of radio waves (hereinafter, referred to as the amount of the building) will be described.
[0107] FIG. 16 is an explanatory diagram for explaining a method for evaluating the influence of buildings that contribute to radio wave reflection in this embodiment. The influence of buildings is evaluated using map information, for example. FIG. 16 is map information showing the state of buildings 4 around the meeting place 3. The map information may be acquired from the server 2 or may be stored in the terminal itself. In FIG. 16, the buildings 4 around the meeting place 3 include 4A to 4F, and the influence of the buildings 4 is evaluated based on the length of the wall facing the route of the car 1A within a predetermined range 5 from the meeting place 3, since the influence of the buildings 4 is problematic in that they reflect radio signals.
[0108] The predetermined range 5 may be set wider for places with high importance. For example, the range may be divided into 5A, 5B, 5C, and 5D depending on whether the vehicle is approaching the meeting place 3, whether it is in front of the meeting place 3, and whether it is along the road on the meeting place side or on the opposite side. Since the passenger position is estimated with the vehicle 1A in front, the structure 4 in front of the meeting place 3 is important, and in terms of distance, the structure 4 along the road on the meeting place 3 side is important.
[0109] The wall lengths 6 facing the vehicle route within the set predetermined range 5 are 6A, 6B, 6C, 6D, and 6E in the case of Figure 16. The sum of these wall lengths is the amount of building walls that contribute to the reflection of radio waves.
[0110] As one control method, as shown in Fig. 17A, depending on the condition of the building 4, when the amount of the wall surface of the building 4 is less than a predetermined amount, an L0 type fitting function is used, and when it exceeds the predetermined amount, an L1 type fitting function is used. Alternatively, as shown in Fig. 17B, since the approximation of the L1 type becomes poor in the range of x where the absolute value of x-x0 is large, a method may be used in which the L0 type is used when the distance to the meeting place 3 is greater than a predetermined value, and the L1 type is used when it is less than the predetermined value. Note that when switching the fitting function depending on the position of the car, since the estimated distance to the passenger may change significantly due to the switching, a position-weighted average of the estimated values by the two fitting functions for a certain section may be used.
[0111] Alternatively, the above two methods may be combined. For example, the amount of wall surface of a building M B The fitting function is switched by a predetermined distance T P may be changed. T p =f(M B ) …(33) Since the greater the amount of wall surface of a building, the greater the possibility that the distortion of the reception intensity profile will be, f is set to a monotonically increasing function in a broad sense. Alternatively, threshold control as shown in FIG. 17C may be used. Here, T P1 <T P2 …(34) Set it so that:
[0112] When using an L1 type fitting function, if the value of J (equation (30)) is 0 or negative, this means that the fitted function does not have an extreme value. Therefore, regardless of the above conditions, the estimate using the L0 type fitting function will be used.
[0113] According to this embodiment, the estimated value can be corrected depending on the situation at the meeting place. EXAMPLES
[0114] In this embodiment, a description will be given of a method for estimating the distance to the meeting point based on information about the direction in which the meeting point appears when the meeting point can be identified before the meeting point arrives.
[0115] In this embodiment, the case where the vehicle-side information terminal 1A identifies the passengers will be described, but the reverse is also true. The derivation of the relationships between the quantities is the same as in the third embodiment. The symbols correspond to those in Figs. 15A and 15B.
[0116] First, the passenger direction θ A Measure the angle θ A The measurement is performed using the camera image and the camera's attitude information. Aand the distance to the closest position x d The relationship is the same as that considered in the third embodiment, and is expressed by the following equation (35) based on the equation (18). x d =x 0E -x A -x B =h A tanφ A tan θ A …(35) This gives us an estimate of the closest travel distance x0 based on the direction measurement. 0E is required.
[0117] In the first embodiment, x is calculated from the change in the received radio wave strength. 0E However, the x 0E Instead of x calculated by equation (35), 0E The vehicle speed is controlled using the above method. In order to prevent shocks in the vehicle speed control when switching from the control based on the radio wave reception strength to the control based on the camera measurement, the switching period is set to x, which is estimated by using both methods. 0E Alternatively, a weighted average of the values of 1 to 3 may be used to gradually switch the values.
[0118] Furthermore, the vehicle control device adjusts the stopping position target by x so that the car door is in front of the passenger at the stopping position. 0E Therefore, the position may be shifted by the difference in the position between information terminal 1A and the car door in the travel direction.
[0119] In addition, if the passenger can locate the vehicle, the information terminal 1B will display the 0E may be estimated and the information transmitted to the vehicle, and the vehicle's controller may use the value to control the vehicle.
[0120] When the passenger side performs the camera measurement, the following relational expression (36) is used in accordance with the second embodiment in correspondence with the expression (16). x d =x 0E -x A -x B =h B tanφ B tan θ B…(36) Here, the angle θ B The measurement is performed in information terminal 1B using the camera image and the camera attitude information.
[0121] As a result, more precise control of the stopping position can be achieved. EXAMPLES
[0122] In this embodiment, a procedure for dealing with a case where the vehicle fails to stop in front of a passenger will be described.
[0123] If the vehicle fails to stop in front of the passenger, the vehicle that is in front when the reception strength reaches its maximum is the vehicle that is waiting, so that vehicle is photographed as shown in Fig. 18A and the photographed image is used for subsequent guidance. The image photographed from the front position is displayed on the screen of information terminal 1B as shown in Fig. 18B, and if the vehicle can be tracked by the camera, a mark is superimposed on the image of the target vehicle.
[0124] Once the vehicle's control system detects that the passenger has passed in front of it, it will slow down the vehicle as far as is safe and bring it to a stop.
[0125] In this way, even if the vehicle cannot stop directly in front of the passenger, the passenger can be guided to the vehicle. EXAMPLES
[0126] In this embodiment, a modified example of the user interface will be described.
[0127] In this embodiment, when the smartphone, which is the information terminal 1C of the passenger who called the taxi, displays the location of the taxi on a map, the mark displaying the location of the taxi, i.e., the other party's information terminal 1A, is changed depending on whether the direct communication wireless signal is not received or received. Here, the direct communication wireless signal is a wireless signal such as a beacon signal accompanied by the above-mentioned identification signal, and may be simply referred to as a wireless signal in this embodiment. Also, it is assumed that the other party's information terminal 1A transmits a wireless signal, and the information terminal 1C receives the wireless signal.
[0128] FIG. 19A is a display screen of the information terminal 1C in this embodiment when the position of the other party's information terminal 1A is still far from the meeting place and the information terminal 1C is not receiving a wireless signal. At this time, the information terminal 1C displays a mark drawn with a dotted line as the mark 1000 that displays the other party's information terminal 1A on the map. Meanwhile, FIG. 19B is a display screen of the information terminal 1C in this embodiment when the other party's information terminal 1A approaches the meeting place and the information terminal 1C is receiving a wireless signal. At this time, the information terminal 1C displays a mark drawn with a solid line as the mark 1001 that displays the other party's information terminal 1A on the map. Note that in the above, the mark 1000 and the mark 1001 are differentiated by a dotted line and a solid line, but this is not limited to this. For example, any difference in display may be used, such as blurring, changing the color, or surrounding the mark with a frame. Note that in FIG. 19A and FIG. 19B, the mark 1010 represents the position of the own information terminal.
[0129] As for the method of displaying the mark, once the other party's information terminal 1A approaches the meeting place and information terminal 1C starts receiving a wireless signal, the mark 1001 may continue to be displayed, or if for some reason information terminal 1C is no longer able to receive a wireless signal, the mark may revert to being displayed as mark 1000.
[0130] Before the information terminal 1C is able to receive wireless signals, the information terminal 1C acquires the display position information of the other party's information terminal 1A on the map from the other party's information terminal 1A by communication other than direct communication, such as via the Internet. The other party's information terminal 1A measures its own position using a GPS or the like. On the other hand, after the information terminal 1C is able to receive wireless signals, the information terminal 1C may determine the display position of the other party's information terminal 1A on the map by the above-mentioned method of estimating from the change in reception strength of the wireless signal. Note that, in order to prevent the display position of the other party's information terminal 1A from changing suddenly before and after the start of reception of the wireless signal, the information terminal 1C may gradually change the display position of the other party's information terminal 1A from the position acquired from the other party's information terminal 1A to the position determined by the method of estimating from the change in reception strength of the wireless signal.
[0131] Furthermore, the information terminal 1C may change the mark representing the other information terminal 1A in stages depending on the strength of the wireless signal reception, not just on the presence or absence of wireless signal reception. For example, as shown in Fig. 20, the information terminal 1C displays the marks (a) to (d) by lengthening the length of the bars as the reception strength increases.
[0132] In addition, when information terminal 1A is unable to measure its own location due to an inability to receive GPS radio waves or the like, and is also unable to estimate its location using direct communication radio signals, information terminal 1C may display an explanatory message 1005 indicating that its location is unknown, as shown in FIG. 21, instead of displaying a mark indicating the location of information terminal 1A on a map.
[0133] Furthermore, information terminal 1C not only displays the position of the other information terminal 1A, but also, when the other information terminal 1A approaches, can switch to a service in which the user can visually recognize the other information terminal 1A through a camera image or by looking directly at the other information terminal 1A, as shown in Fig. 11A to Fig. 11C. This service switching may be performed automatically by information terminal 1C, or a user instruction may be received through input box 1007 shown in Fig. 22. Also, information terminal 1C and information terminal 1B may cooperate to start a new service on another information terminal 1B (e.g., HMD).
[0134] If the information terminal owned by the user is an information terminal 1D such as a watch, the notification corresponding to the reception strength of the wireless signal from the other information terminal 1A may be made by voice or vibration instead of displaying a mark. That is, in the case of voice notification, as shown in Fig. 23, the information terminal 1D notifies the other information terminal 1A (taxi) that it is approaching or notifies the change in reception strength when it receives a wireless signal of direct communication. In addition, in the case of vibration notification, as shown in Fig. 24, the information terminal 1D generates vibrations of a preset vibration pattern when the reception strength changes.
[0135] In the above description, the other party's information terminal 1A transmits a wireless signal and the information terminal 1C receives the wireless signal, but the information terminal 1C may transmit a wireless signal and the information terminal 1A may receive the wireless signal. In this case, the information terminal 1A obtains the wireless signal reception strength information from the information terminal 1C by direct communication or other communication means.
[0136] Furthermore, the type of information terminal is not limited to the illustrated one. For example, the taxi's information terminal may be the local information terminal 1A, and the customer's smartphone may be the other party's information terminal 1C, and the information terminal 1A may display a mark according to the reception strength, or notify by voice or vibration. An example of the display on the information terminal 1A is shown in FIG. 25. Mark 1200 indicates the position of the local information terminal 1A, and mark 1210 indicates the position of the other party's information terminal 1C. The information terminal 1A then changes the shape of the mark 1210 in response to the reception strength of the wireless signal. The specific changes are basically the same as those described above, and therefore a detailed description will be omitted.
[0137] As described above, if the user knows whether or not a wireless signal is being received and the reception strength, the user can visually grasp the distance from the user's own information terminal to the other information terminal 1A for which he is waiting. Also, if the user knows that a wireless signal can be received and that the other information terminal 1A has come close, it is suitable for the user to grasp the timing of switching services, such as switching to a meeting support display as described in other embodiments. Furthermore, the information terminal 1C can also control the contents of meeting support according to the position of the other information terminal 1A in a way that is suitable for the user, improving user convenience. EXAMPLES
[0138] In this embodiment, a modified example of a meeting support service that uses camera images will be described.
[0139] The information terminal 1C acquires an image (still image or video, hereafter referred to as a partner information terminal image) of the direction in which the own information terminal 1C is located, taken by the partner information terminal 1A based on the estimated position of the own information terminal 1C, and identifies the position of the own information terminal 1C in the image. The information terminal 1C then transmits position information of the identified partner information terminal image of the own information terminal 1C to the partner information terminal 1A, and the partner information terminal 1A identifies the position of the information terminal 1C from the received position information. Here, the position in the image means the position in the image drawing area.
[0140] A detailed procedure will be described. First, the information terminal 1C displays a partner information terminal image 2000 as shown in FIG. 26A. Note that as shown in FIG. 26B, the partner information terminal image 2000 may be displayed superimposed on the map display screen in the seventh embodiment. In this example, the partner information terminal image 2000 shows a person 2001a who is a user who holds the information terminal 1C as seen from the information terminal 1A, and a person 2001b who is different from the user. The information terminal 1C accepts a user input that identifies a person in the image who is a user or a meeting position. The user input that identifies a person or a meeting position is, for example, an input that specifies a position on the display screen by touching with the user's finger, and specifies a person or a meeting position by the specified position. Note that the input method by touching is one example, and is not limited to touching as long as it can specify a person or a position. The same applies to the following input method for specifying a person or a position. Also, the position where a person is located may be specified as a meeting position by an input that specifies a person, and the person at the meeting position may be specified by an input that specifies a meeting position. Alternatively, the information terminal 1C may identify the user 2001a by image analysis, without relying on the user's input of identifying the person. When analyzing the image, the information terminal 1C may use an image of the user taken by the information terminal 1C at the time of the meeting for identifying the user.
[0141] After identifying the person to be met (i.e. the user himself / herself) or the meeting location, information terminal 1C displays a person identification mark 2010 or a meeting location identification mark 2011 by superimposing it on the other party's information terminal image 2000. Information terminal 1C may display both marks. After identifying the person or the meeting location, information terminal 1C transmits position information of the identified target in other party's information terminal image 2000 to other party's information terminal 1A. Other party's information terminal 1A recognizes the meeting person and the meeting location from the position information in the image acquired from information terminal 1C.
[0142] As a variation of the person identification method, a method as shown in FIG. 27 may be used. That is, the information terminal 1C displays an image (2021a, 2021b) of a person part extracted from the other party's information terminal image, and accepts the user's input action of identifying the person. In this example, the image 2021a is a person image corresponding to the user, and the information terminal 1C displays, for example, an identification mark 2020 indicating that the image has been identified after accepting the user's action of identifying the image 2021a. After accepting the user's action of identifying the person image, the information terminal 1C notifies the other party's information terminal 1A of the identification result. The other party's information terminal 1A recognizes which person is the other party to the meeting based on the identification result of the person image notified from the information terminal 1C. The extraction of the images (2021a, 2021b) may be performed by the other party's information terminal 1A, may be performed by the information terminal 1C, or may be performed by the server 2.
[0143] Next, a method for the user carrying information terminal 1C to confirm the recognition result of the meeting person and the meeting location by the other party's information terminal 1A will be described.
[0144] The other party's information terminal 1A recognizes the person to be met and the meeting location by position estimation based on the reception strength of the wireless signal of direct communication or the above-mentioned location information acquired from the information terminal 1C. The information terminal 1C acquires information on the recognition result by the other party's information terminal 1A and displays the recognition result as shown in Fig. 28. In the display example of Fig. 28, the information terminal 1C displays a specific mark 2030 indicating the person to be met recognized by the other party's information terminal and a specific mark 2031 indicating the meeting location superimposed on the other party's information terminal image 2100. The superimposition of the marks may be performed by the information terminal 1C, or the other party's information terminal image 2100 may be acquired by the information terminal 1A as an image on which the marks are superimposed.
[0145] The user of the information terminal 1C can judge whether the recognition result is correct by looking at the specific mark 2030 of the person to be met recognized by the other party's information terminal 1A and the specific mark 2031 of the meeting place. After presenting the other party's information terminal image 2100 to the user, the information terminal 1C accepts correction input from the user.
[0146] If correction is necessary, the user inputs the correction. The correction input is the same as in FIG. 26A, and the user inputs the user's own position in the image or the meeting location by touching or other means. FIG. 28 shows an example of a case where the meeting target is person 2001a, but the other party's information terminal 1A erroneously recognizes the meeting target as person 2001b. After receiving correction input from the user, such as that the meeting target is person 2001a (i.e. the user himself), information terminal 1C notifies the other party's information terminal 1A of the correction information. Information terminal 1A corrects the meeting target person and the meeting location using the correction information notified by information terminal 1C.
[0147] The person to be met may be corrected by a display format that lists images of people as shown in Fig. 29. In the list display of images of people, a method such as superimposing a specific mark 2040 may be used to display the image of the person that the other party's information terminal 1A recognizes as the person to be met, so that the user of the information terminal 1C can determine whether the recognition result is correct. When correcting, the information terminal 1C accepts input of the correct image of the person, as in Fig. 27.
[0148] The type of information terminal is not limited to the illustrated one. For example, the information terminal on the taxi side may be the own information terminal 1A, and the customer's HMD may be the other party's information terminal 1B, and the information terminal 1A may determine whether the customer information terminal 1B recognizes the vehicle to be met. An example of display on the information terminal 1A is shown in FIG. 30. The information terminal 1A displays the other party's information terminal image 2200, in which the vehicle 2051a and the vehicle 2051b are shown. In addition, a specific mark 2060 indicating that the vehicle is recognized as the vehicle to be met by the information terminal 1B is also superimposed and displayed. The vehicle corresponding to the information terminal 1A is 2051a, but in this example, the information terminal 1B erroneously recognizes a different vehicle 2051b as the vehicle to be met. In correcting the erroneous recognition of the vehicle to be met, the information terminal 1A may make a correction judgment by means of image analysis or the like of the information terminal itself, or may accept a correction input by the user, as described above. In addition, since the user of the information terminal 1A may be driving a car, the user may input corrections by vocalizing the identification symbols (2052a, 2052b) for identifying the car that are added and superimposed by the information terminal 1A, instead of by touching or other actions. Other embodiments are basically the same as those described above, and detailed explanations will be omitted.
[0149] Fig. 31 is a control flow diagram of meeting support in this embodiment. Fig. 31 is based on the flow diagram of Fig. 9, and the processing from steps S51A, S51B onwards to steps S54A, S54B is different, so those parts will be explained. The other steps are the same as Fig. 9, so the explanation will be omitted.
[0150] In steps S51A and S51B, each information terminal 1 provides information to support the meeting to the user or the vehicle control device. In addition to displaying the support information on its own information terminal, the information terminal transmits the support information displayed on its own information terminal to the other information terminal.
[0151] In steps S52A and S52B, if there is input from the user to correct information such as the vehicle or person to be met, the input is accepted.
[0152] In steps S53A and S53B, if there is any correction information input by the user or determined by the information terminal, that information is transmitted to the other information terminal.
[0153] In steps S54A and S53B, if there is any correction information, the support information is corrected.
[0154] As a result, by acquiring an image of the other party's information terminal, it is possible to confirm that the other party's information terminal recognizes the location of the own information terminal and to notify the other party's information terminal of the location of the own information terminal, thereby enabling more accurate support for rendezvous. EXAMPLES
[0155] In this embodiment, a modified example will be described in which the information terminal 1 can be easily identified in an image.
[0156] In the above-mentioned embodiments, when the information terminal 1 determines a person or a car in an image that is a meeting target, the information terminal 1 receives a determination input from a user or uses a determination based on image analysis. In this embodiment, instead of these methods, the information terminal 1 identifies a person or a car based on identification information transmitted as an optical signal using the light-emitting device 3000.
[0157] 32 shows a display screen 500 of an information terminal 1B, which is an HMD in this embodiment. In the case of a see-through HMD, the user can see the outside world directly. In the case of a video see-through HMD, the user can see the outside world as a video image. In the case of a see-through HMD, it is assumed that the outside world is also captured by a camera.
[0158] FIG. 32 shows a situation where the user faces a direction in which it is estimated from the reception strength of the wireless signal of the direct communication that the vehicle to be met is located. In this example, there are two candidate vehicles (3001a, 3001b) within the user's field of vision. Light-emitting devices 3000a, 3000b are installed on the vehicle bodies, respectively. In the case of the information terminal 1A, when the information terminal 1A is mounted on a vehicle, the light-emitting device 3000 is separated and installed in a position visible from the outside. In this situation, the information terminal 1A transmits its own identification information as an optical signal by the light-emitting device 3000. This identification information may be the same as the identification information of the wireless signal of the direct communication, but the information terminal 1A notifies the other information terminal 1B in advance. Alternatively, the information terminal 1B may specify the identification information.
[0159] The transmission of identification information by optical signals using the light emitting device 3000 is performed by modulating the light emission intensity, for example, as shown in FIG. 33. FIG. 33 shows communication by a bit string in which light emission corresponds to "1" and no light emission corresponds to "0", but this is not limiting. For example, multi-value modulation may be performed. The modulation speed of the light emission intensity is set slower than the frame rate of the camera on the imaging side so that the camera can reliably capture the modulation of the light emission intensity. For example, when the frame rate of the camera is 60 frames / second, the modulation speed is set to about 20 baud.
[0160] The information terminal 1B captures a video with a camera and extracts the areas of the light-emitting devices 3000a and 3000b by image analysis. The information terminal 1B then restores the information transmitted from each of the cars (3001a and 3001b) from the change in brightness of the area. The information terminal 1B then collates the identification information of the information terminal 1A of the meeting target, which it has been notified of in advance or has specified, to identify which car is the meeting target. In FIG. 33, the car of the meeting target is 3001a, and the information terminal 1B displays a specific mark 3050 by an AR object indicating that the car 3001a is the meeting target car in a superimposed manner. Here, if there is a target that happens to transmit the same identification information as the identification information of the information terminal 1 of the meeting target, the information terminal 1B may request the information terminal 1A to change the identification information to be transmitted.
[0161] The identification information may be transmitted from both information terminals 1, and each may confirm the other information terminal 1. The combination of types of information terminals 1 is not limited to the above. The information terminal 1 that transmits information from the light-emitting device 3000 by optical signals may receive an image of the other information terminal 1 and confirm the information transmitted by its own light-emitting device 3000. Furthermore, the server 2 may restore the information transmitted from the light-emitting device 3000 in the image, and the server 2 may notify the information terminal 1 of which person, car, or information terminal 1 is the person, car, or information terminal 1 that is the meeting target in the image.
[0162] The technique of this embodiment may be combined with other embodiments as a means for identifying the person, the car, and the information terminal 1 to be met.
[0163] Fig. 34 is a control flow diagram of the meeting support in this embodiment. Based on the flow diagram in Fig. 9, the above-mentioned processing is added as steps S71A and S71B. The other steps are the same as those in Fig. 9, so the explanation will be omitted.
[0164] In steps S71A and S71B, one information terminal 1 transmits its own identification information to the other information terminal 1 using the light-emitting device 3000. The information terminal 1 that receives the transmitted identification information captures a video in the direction in which it presumes the transmitting information terminal 1 is located. Here, both information terminals 1 may be the senders of the identification information. This video is analyzed by the information terminal 1 that receives the identification information, or the information terminal 1 that transmits the identification information, or the server 2, and the transmitting information terminal 1 in the image is identified.
[0165] As described above, by transmitting identification information by the information terminal 1 using a light-emitting device and analyzing a video captured of the information terminal 1 transmitting the identification information, the exact location of the other information terminal 1 can be identified, thereby improving the reliability of the meeting support system.
[0166] Although the above-mentioned embodiments have been described in detail to easily explain the present invention, the present invention is not necessarily limited to those having all the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. In addition, the above-mentioned functions, etc. may be implemented in part or in whole by hardware, or may be implemented by software program processing. The programs and data constituting the functions, etc. may be stored in a computer-readable storage medium, or may be stored in a device on a communication network. [Explanation of symbols]
[0167] 1, 1A to 1D: information terminal, 2: server, 3: meeting place, 4: building, 5: specified range, 6: wall length, 9: communication network, 10: housing, 11: display surface, 101: processor, 102: memory, 103: display device, 107: operation input unit, 110A, 110B: movement line, 111: ground surface, 112A, 112B: starting point, 113, 115A, 115B: distance, 114A, 114B: movement distance, 120: direct wave path, 130A, 130B: projection line, 500, 600: display screen, 501, 503, 505, 601, 603, 605, 1005: explanatory message ,502, 504, 602, 604: AR object mark, 1000, 1001, 1010, 1200, 1210: mark, 1007: input box, 1100: vibration generator, 2000, 2100, 2200: other party's information terminal image, 2001a, 2001b: person, 2010, 2011, 2020, 2030, 2031, 2040, 2060, 3050: specific mark, 2021a, 2021b: image, 2051a, 2051b, 3001a, 3001b: car, 2052a, 2052b: identification symbol, 3000, 3000a, 3000b: light-emitting device
Claims
1. A first information terminal has a function of transmitting a radio signal accompanied by identification information, and a second information terminal has a function of measuring the reception strength of the radio signal, The second information terminal measures the reception strength of the wireless signal transmitted by the first information terminal; At least one of the first information terminal and the second information terminal measures a moving distance; A position estimation system characterized in that at least one of the first information terminal and the second information terminal estimates the relative positional relationship between the first information terminal and the second information terminal from a change in the reception strength of the radio signal with respect to the travel distance.
2. 2. The position estimation system of claim 1, A position estimation system, wherein one of the first information terminal or the second information terminal is a car.
3. 2. The position estimation system of claim 1, A position estimation system, characterized in that the relative positional relationship is estimated by parameter fitting using measured values of the reception strength with respect to the movement distance, assuming that the reception strength is proportional to a function of the movement distance.
4. 4. The position estimation system of claim 3, A position estimation system, wherein the function is an inverse of a second-order polynomial of the movement distance.
5. 4. The position estimation system of claim 3, A position estimation system, characterized in that the function is switched depending on the condition of surrounding buildings.
6. 2. The position estimation system of claim 1, one of the first information terminal or the second information terminal is a vehicle or is mounted in a vehicle; A position estimation system, characterized in that a control device of the vehicle controls the speed of the vehicle based on the estimated value of the relative position relationship.
7. 2. The position estimation system of claim 1, A position estimation system characterized in that one of the first information terminal and the second information terminal has an imaging function, and based on the estimated relative positional relationship, the position on an image of the other information terminal, or a user possessing the information terminal, or a vehicle equipped with the information terminal is estimated.
8. 8. The position estimation system of claim 7, A position estimation system characterized in that external characteristic information of the user or the vehicle acquired from a server is used in conjunction with the position estimation on the image.
9. 2. The position estimation system of claim 1, one of the first information terminal and the second information terminal has a photographing function; A position estimation system, comprising: a position estimation unit that estimates the relative positional relationship from the angular direction of the other information terminal, and uses the estimated relative positional relationship from the received strength of radio waves in combination with the estimated relative positional relationship from the angular direction of the other information terminal.
10. 2. The position estimation system of claim 1, A position estimation system, characterized in that an information terminal located in front of one of the first information terminal and the second information terminal at the time when the reception strength is at its maximum is estimated to be the other information terminal.
11. 2. The position estimation system of claim 1, the second information terminal displays the position of the first information terminal with a mark on a map; A position estimation system, characterized in that the shape of the mark is changed according to the strength of the reception strength.
12. 2. The position estimation system of claim 1, one of the first information terminal and the second information terminal has a photographing function; A position estimation system characterized in that a captured image is displayed on a partner information terminal.
13. 2. The position estimation system of claim 1, one of the first information terminal and the second information terminal has a photographing function; the other of the first information terminal and the second information terminal has a function of transmitting an optical signal; A position estimation system characterized by restoring identification information transmitted by an optical signal using a captured image, and identifying the position in the image of the first information terminal or the second information terminal transmitting the optical signal.
14. A position estimation method in a position estimation system including a first information terminal having a function of transmitting a wireless signal accompanied by identification information and a second information terminal having a function of measuring a reception strength of the wireless signal, comprising: A step in which the second information terminal measures a reception strength of a radio wave transmitted by the first information terminal; At least one of the first information terminal and the second information terminal measures a travel distance; a step in which at least one of the first information terminal and the second information terminal estimates a relative positional relationship between the first information terminal and the second information terminal from a change in the reception strength of the radio signal with respect to the moving distance.
15. An information terminal in a position estimation system including a first information terminal having a function of transmitting a radio signal accompanied by identification information and a second information terminal having a function of measuring a reception strength of the radio signal, Measure the reception strength at your own terminal or receive information on the reception strength measured at another terminal, Measure the travel distance by the own terminal or receive the travel distance measured by another terminal; An information terminal characterized in that a relative positional relationship between said first information terminal and said second information terminal is estimated from a change in the reception strength of said wireless signal with respect to said moving distance.
Citation Information
Patent Citations
Information processing device, method for controlling information processing device, and program
JP2015005798A
Information processor, program, and information processing method
JP2019191914A
Information processing device and information processing method
JP2019194836A
Navigation information device, navigation information processing method, and navigation information processing program
WO2020021936A1
Wireless communication system and communication terminal device
JP2005351859A
