Data processing system, user terminal device, and data processing method

Abstract

To provide a reference position for augmented reality in a simple and highly accurate manner.SOLUTION: A data processing system has: a transmission device 30 which is installed in association with a location where a movement speed of a user decreases and transmits a beacon signal to a surrounding area; and a user terminal device 40 which is carried by the user. The user terminal device 40 comprises: a beacon receiving unit 42 which receives the beacon signal; an augmented reality processing unit 45 which displays a virtual object while superposing on an image obtained by photographing the surroundings; a movement detection unit 43 which detects a movement direction and a movement speed of the user terminal device 40 itself; and a reference position determination unit 44 which, when the movement speed detected by the movement detection unit 43 decreases and the beacon signal is received by the beacon receiving unit 42, sets a predetermined position located at a predetermined distance in the movement direction from a current position of the user terminal device 40 as a reference position of the augmented reality processing unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a data processing system, a user terminal device, and a data processing method. [Background technology]

[0002] Conventionally, there is a technology described in JP 2021-100222 A (Patent Document 1) for realizing augmented reality (AR). This publication states that "This contributes to improving the accuracy of capturing and image recognition when viewing AR services using 5G from the spectator seats of a large facility such as a stadium." and "The terminal device has a transceiver, a camera, a display, and a processor. The processor determines a first synchronization signal whose wireless quality meets a predetermined threshold from among multiple synchronization signals beamformed and transmitted from the base station, and reports it to the base station. The processor displays the virtual object corrected by the correction information on the display by superimposing it on the image captured by the camera. The correction information is information for indicating the position of the real object in the area covered by the first synchronization signal. The correction information includes information regarding the orientation of the virtual object to be displayed on the display in the area and the distance from the real object to the area." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-100222 A Summary of the Invention [Problem to be solved by the invention]

[0004] When placing a virtual object, it is important to determine the reference position. If the reference position is not determined properly, the virtual object will appear unnatural, such as sinking into the ground or floating in the air. Furthermore, in services that guide users outdoors, sufficient accuracy is also required in the horizontal direction. The above conventional technology can obtain the relative angle to the AR object, but does not consider performing positioning with high accuracy. In addition, because expensive and high-performance base stations are used, there is a problem that the use is limited and versatility is low. For example, to provide various augmented reality services to users of widely used smartphones, a technology that can determine the reference position with high accuracy with a simple configuration is required.

[0005] Therefore, an object of the present invention is to provide a reference position for augmented reality simply and with high accuracy. [Means for solving the problem]

[0006] In order to achieve the above-mentioned object, one representative data processing system of the present invention comprises a transmitting device that is installed in correspondence with a location where a user's movement speed slows down and emits a beacon signal in the surrounding area, and a user terminal device carried by the user, wherein the user terminal device is characterized in that it comprises a beacon receiving unit that receives the beacon signal, an augmented reality processing unit that displays a virtual object superimposed on an image captured of the surrounding area, a movement detection unit that detects the movement direction and movement speed of the user terminal device itself, and a reference position determination unit that, when the movement speed detected by the movement detection unit slows down and the beacon signal is received by the beacon receiving unit, sets a predetermined position located a predetermined distance in the movement direction from the current position of the user terminal device as the reference position of the augmented reality processing unit. Furthermore, one representative user terminal device of the present invention is a user terminal device carried by a user, and is characterized in that it comprises a beacon receiving unit that receives a beacon signal from a transmitting device installed in correspondence with a location where the user's movement speed slows down, an augmented reality processing unit that displays a virtual object superimposed on an image captured of the surroundings, a movement detection unit that detects the movement direction and movement speed of the user terminal device itself, and a reference position determination unit that, when the movement speed detected by the movement detection unit slows down and the beacon signal is received by the beacon receiving unit, sets a predetermined position located a predetermined distance in the movement direction from the current position of the user terminal device as the reference position of the augmented reality processing unit. Furthermore, one representative data processing method of the present invention includes a movement detection step in which a user terminal device carried by a user detects the movement direction and movement speed of the user terminal device itself, a beacon signal reception step in which the user terminal device receives a beacon signal from a transmitting device installed in correspondence with a location where the user's movement speed decreases, a reference position determination step in which, when the movement speed detected by the movement detection step decreases and the user terminal device receives the beacon signal by the beacon signal reception step, the reference position is set to a predetermined position located a predetermined distance from the current position of the user terminal device in the movement direction, and a step in which the user terminal device performs augmented reality processing in which a virtual object is superimposed on an image captured of the surroundings based on the reference position. Effect of the Invention

[0007] According to the present invention, it is possible to provide a reference position for augmented reality easily and with high accuracy. Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram of the provision of a reference position according to an embodiment; [Diagram 2] Data processing system configuration diagram [Diagram 3] Illustrative diagram of reference position determination [Figure 4] Flowchart showing the procedure for determining a reference position [Diagram 5] Flowchart showing the reference position calculation process [Figure 6] Flowchart showing the processing steps of the augmented reality service [Figure 7] FIG. 13 is a block diagram of a user terminal device according to a modified example. [Figure 8] A flowchart showing a procedure for determining a reference position according to a modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment will be described with reference to the drawings. EXAMPLES

[0010] FIG. 1 is an explanatory diagram of providing a reference position according to an embodiment. The data processing system according to the embodiment includes a user terminal device 40 and a calling device 30 . The transmitting device 30 is installed near a place where the movement speed of the user 10 slows down. For example, if a guide board 20 is set up, the user 10 who approaches the guide board 20 will stop and read its contents. Therefore, the guide board 20 becomes a place where the movement speed of the user 10 slows down. In addition, doors, gates, service counters, etc. can also be used as places where the movement speed of the user 10 slows down. The transmitting device 30 emits a beacon signal of Bluetooth (registered trademark) Low Energy (BLE) in the surrounding area.

[0011] The user terminal device 40 includes a camera and a display unit, and is carried by the user 10. The user terminal device 40 is, for example, a smartphone. The user terminal device 40 provides an augmented reality service that displays virtual objects superimposed on a captured image of the surroundings, thereby presenting the virtual objects as if they exist in a real space.

[0012] The user terminal device 40 receives the beacon signal from the transmitting device 30 and determines the reference position of the augmented reality. This reference position serves as a reference for where in the captured image of the surroundings a virtual object is to be placed. If the reference position is not positioned properly, the virtual object will appear to be embedded in the ground or floating in the air, resulting in an unnatural display. Furthermore, in services that guide users outdoors, etc., sufficient accuracy is also required in the horizontal direction.

[0013] The user terminal device 40 detects the moving direction and moving speed of the user terminal device 40 itself, and determines the reference position of the augmented reality using the moving direction and moving speed. Specifically, when the user 10 approaches the guide board 20 and stops, if the reception strength of the beacon signal of the transmitting device 30 is equal to or greater than a predetermined signal strength threshold, the user terminal device 40 determines a position a predetermined distance away from the current position Pr of the user terminal device 40 in the same moving direction as before as the reference position Pv. The condition when the user 10 approaches the guide board 20 and stops may be detected when the movement speed of the user terminal device 40 becomes equal to or greater than a first speed threshold and then becomes less than a second speed threshold that is smaller than the first speed threshold. The predetermined distance may be a fixed value or may be a value according to the strength of the beacon signal. If the predetermined distance is a value according to the strength of the beacon signal, the higher the strength of the beacon signal, the smaller the predetermined distance should be.

[0014] When the user terminal device 40 receives a beacon signal from one transmitting device 30, the strength of the beacon signal can be converted into the proximity of the distance to the transmitting device 30, thereby estimating the distance from the user terminal device 40 to the transmitting device 30. However, in a state where a beacon signal is being received from one transmitting device 30, it is not possible to identify in which direction the transmitting device 30 is located from the user terminal device 40. Therefore, a beacon signal is transmitted from the transmitting device 30 placed near a position where the speed of the user 10 is sufficiently reduced, and the direction in which the user 10 approaches that position is determined as the relative direction of the transmitting device 30 with respect to the user terminal device 40, so that the user terminal device 40 specifies its positional relationship with the transmitting device 30. Using this positional relationship, the reference position Pv is determined, so that the user terminal device 40 can easily and accurately determine the reference position for augmented reality. The user terminal device 40 displays, for example, a virtual object 60 at a reference position Pv. Note that the virtual object 60 does not necessarily need to be displayed at the reference position Pv, ​​and the display position can be appropriately set based on the reference position Pv.

[0015] 2 is a configuration diagram of the data processing system. The user terminal device 40 includes a camera 41, a beacon receiving unit 42, a movement detecting unit 43, a reference position determining unit 44, an augmented reality processing unit 45, a server communication unit 46, and a display unit 47.

[0016] The camera 41 is a photographing unit that photographs the surroundings of the user terminal device 40 . The beacon receiver 42 is a BLE communication interface, and receives a beacon signal from the transmitting device 30. The beacon receiver 42 can identify the transmitting device 30 by reading the identification information of the received beacon signal.

[0017] The movement detection unit 43, the reference position determination unit 44, and the augmented reality processing unit 45 may be realized by, for example, a CPU (Central Processing Unit). The movement detection unit 43 is a processing unit that detects the moving direction and moving speed of the user terminal device 40 itself. The movement detection unit 43 analyzes the image captured by the camera 41 in real time, observes the camera coordinates, and stores them. The movement detection unit 43 detects the moving direction and moving speed of the user terminal device 40 from the change in the camera coordinates. For convenience, the moving direction and moving speed of the user terminal device 40 are called the terminal moving direction and terminal moving speed. If the sampling period of the camera coordinates by the movement detection unit 43 is short, the estimation accuracy of the moving direction and moving speed becomes high, but the shorter the sampling period, the larger the capacity becomes. Therefore, the movement detection unit 43 may limit the time for recording the camera coordinates. In addition, if the user terminal device 40 itself does not move, it is difficult to detect the direction of movement. In that case, it can be assumed that the direction of movement of the user is the direction in front of the camera when the application is started. In addition, content that encourages the user to move may be displayed on the user terminal device 40. Also, when an unintended movement such as the user terminal device 40 moving unsteadily from side to side is detected, it is effective to display a warning to "move straight toward a predetermined position" on the user terminal device 40. For example, a warning message may be displayed in text on the user terminal device 40, or a virtual object may be used to call attention.

[0018] When the terminal moving speed becomes equal to or greater than the first speed threshold and then falls below the second speed threshold, the reference position determination unit 44 compares the reception strength of the beacon signal received by the beacon receiving unit 42 with the signal strength threshold. The second speed threshold is preset to a value sufficiently smaller than the first speed threshold. When the reception strength of the beacon signal is equal to or greater than a signal strength threshold, the reference position determination unit 44 determines a position located a predetermined distance from the current position of the user terminal device 40 in the terminal movement direction as a reference position for augmented reality processing. The reference position determination unit 44 can use a fixed distance value as the predetermined distance. For example, the signal strength threshold may be set to a value corresponding to 1 m in distance conversion, and the predetermined distance may be set to 50 cm. The reference position determination unit 44 can also use a value corresponding to the reception strength as the predetermined distance. In this case, the higher the strength of the beacon signal, the shorter the predetermined distance is set.

[0019] The augmented reality processing unit 45 performs processing to superimpose a virtual object on the image captured by the camera 41 and display it on the display unit 47. When superimposing a virtual object on the image, the augmented reality processing unit 45 controls the position of the virtual object based on the reference position determined by the reference position determination unit 44. The virtual object may be data held or stored by the augmented reality processing unit 45, or may be acquired from outside.

[0020] The server 50 includes an augmented reality service providing unit and is capable of transmitting virtual objects to the user terminal device 40 . The server communication unit 46 of the user terminal device 40 is a communication interface that communicates with the server 50 wirelessly.

[0021] The augmented reality processing unit 45 transmits, for example, the identification number of the beacon signal received by the beacon receiving unit 42 to the server 50, and receives a virtual object corresponding to the identification number. Therefore, the augmented reality processing unit 45 can receive a virtual object to be displayed in the vicinity of the transmitting device 30 specified by the identification number of the beacon signal. In addition, the augmented reality processing unit 45 can receive data for a virtual object that specifies the positional relationship between the position where the virtual object is displayed and a reference position, and the behavior of the virtual object when the positional relationship between the virtual object and the user terminal device 40 changes.

[0022] FIG. 3 is an explanatory diagram of the determination of the reference position. In Fig. 3, the user 10 approaches the guide board 20 and stops at a position Pr. Since the user terminal device 40 is carried by the user 10, the moving direction and moving speed of the user 10 coincide with the terminal moving direction and terminal moving speed. As already described, the terminal moving direction and terminal moving speed are obtained by the user terminal device 40. The terminal moving direction and terminal moving speed indicate the terminal moving vector of the user terminal device 40.

[0023] If the position Pr is within the range 30a where the strength of the beacon signal from the transmitting device 30 is equal to or greater than the signal strength threshold, the user terminal device 40 places the reference position Pv on the extension of the terminal movement vector from the current position Pr. The transmitting device 30 is not necessarily on the extension of the terminal movement vector, but since the terminal movement vector is obtained when approaching the transmitting device 30, it is expected that the distance between the transmitting device 30 and the reference position Pv can be made sufficiently small by appropriately setting the distance from the current position Pr to the reference position Pv. The distance between the transmitting device 30 and the reference position Pv being sufficiently small means that the error is small enough to make the service practical, that is, to make the position and behavior of the object presented as virtual reality not unnatural. In addition, since the terminal movement vector was obtained when approaching the originating device 30, it is possible to estimate the relative direction of the originating device 30 with respect to the user terminal device 40. Furthermore, since the movement of the user terminal device 40 occurs due to the movement of the user 10, the terminal movement direction is horizontal to a surface (ground, floor, etc.) that is the reference for the movement of the user 10. Therefore, the positional relationship between the horizontal direction for the user terminal device 40 and the position of the originating device 30 can be specified.

[0024] 4 is a flowchart showing the procedure for determining a reference position. When determining a reference position, the user terminal device 40 sequentially executes the processes of steps S101 to S108. Step S101 The movement detection unit 43 starts observing the camera coordinates and storing the camera coordinates. The camera coordinate observation is a process of analyzing in real time the image captured by the camera 41 and observing the camera coordinates. After that, the process proceeds to step S102.

[0025] Step S102 The movement detection unit 43 detects the terminal movement direction and terminal movement speed of the user terminal device 40 from the change in the camera coordinates. After that, the process proceeds to step S103. Step S103 The reference position determination unit 44 compares the terminal movement speed with the first speed threshold. If the terminal movement speed is less than the first speed threshold (step S103; No), the process returns to step S102. If the terminal movement speed is equal to or greater than the first speed threshold (step S103; Yes), the process proceeds to step S104.

[0026] Step S104 The movement detection unit 43 detects the terminal movement direction and terminal movement speed of the user terminal device 40 from the change in the camera coordinates. After that, the process proceeds to step S105. Step S105 The reference position determination unit 44 compares the terminal movement speed with the second speed threshold. If the terminal movement speed is equal to or greater than the second speed threshold (step S105; No), the process returns to step S104. If the terminal movement speed is less than the second speed threshold (step S105; Yes), the process proceeds to step S106.

[0027] Step S106 The reference position determining unit 44 acquires the reception strength of the beacon signal received by the beacon receiving unit 42. After that, the process proceeds to step S107. Step S107 The reference position determination unit 44 compares the reception strength of the beacon signal with the signal strength threshold. If the reception strength of the beacon signal is less than the signal strength threshold (step S107; No), the process returns to step S104. If the reception strength of the beacon signal is equal to or greater than the signal strength threshold (step S107; Yes), the process proceeds to step S108. Step S108 The reference position determination unit 44 executes the reference position calculation process and ends the process.

[0028] FIG. 5 is a flowchart showing the procedure of the reference position calculation process. The reference position calculation process sequentially executes the processes of steps S201 to S208. Step S201 The reference position determination unit 44 identifies the current position of the camera coordinates. Then, the process proceeds to step S202. Step S202 The reference position determination unit 44 identifies the terminal movement direction. Then, the process proceeds to step S203. Step S203 The reference position determination unit 44 calculates a position that is a predetermined distance from the current position in the terminal movement direction. Then, the process proceeds to step S204. Step S204 The reference position determination unit 44 determines the calculated position as the reference position, outputs it to the augmented reality processing unit 45, and ends the process.

[0029] 6 is a flowchart showing a processing procedure of the augmented reality service. When providing the augmented reality service, the augmented reality processing unit 45 repeatedly executes the processes of steps S301 to S305. Step S301 The augmented reality processing unit 45 acquires an image from the camera 41. After that, the process proceeds to step S302. Step S302 The augmented reality processing unit 45 acquires the virtual object from the server 50. After that, the process proceeds to step S303. Step S303 The augmented reality processing unit 45 sets the reference position in the camera coordinates. Then, the process proceeds to step S304. Step S304 The augmented reality processing unit 45 overlays the content data on the camera image based on the reference position. After that, the process proceeds to step S305. Step S305 The augmented reality processing unit 45 displays the camera image with the content data overlaid on it on the display unit 47, and ends the processing.

[0030] 7 is a configuration diagram of a user terminal device according to a modified example. User terminal device 40a shown in FIG. The acceleration sensor 48 measures the acceleration applied to the user terminal device 40a and outputs it as acceleration data. The GPS unit 49 receives signals from GPS satellites, measures the coordinates of the user terminal device 40a, and outputs the coordinates as GPS position data. The movement detection unit 43a uses the acceleration measurement result and output by the acceleration sensor 48 and the position information by the GPS unit 49 to detect the movement direction and movement speed of the user terminal device 40a. Other configurations are similar to those of the user terminal device 40 shown in FIG. 2, so the same components are given the same reference numerals and the description thereof is omitted.

[0031] 8 is a flowchart showing a procedure for determining a reference position according to a modified example. When determining a reference position, the user terminal device 40a sequentially executes the processes of steps S401 to S408. Step S401 The movement detection unit 43a acquires acceleration data and GPS position data (step S401), and then proceeds to step S402.

[0032] Step S402 The movement detection unit 43a detects the terminal movement direction and terminal movement speed of the user terminal device 40 from the acceleration data and the GPS position data. After that, the process proceeds to step S403. Step S403 The reference position determination unit 44 compares the terminal movement speed with the first speed threshold. If the terminal movement speed is less than the first speed threshold (step S403; No), the process returns to step S402. If the terminal movement speed is equal to or greater than the first speed threshold (step S403; Yes), the process proceeds to step S404.

[0033] Step S404 The movement detection unit 43a detects the terminal movement direction and terminal movement speed of the user terminal device 40 from the acceleration data and the GPS position data. After that, the process proceeds to step S405. Step S405 The reference position determination unit 44 compares the terminal movement speed with the second speed threshold. If the terminal movement speed is equal to or greater than the second speed threshold (step S405; No), the process returns to step S404. If the terminal movement speed is less than the second speed threshold (step S405; Yes), the process proceeds to step S406.

[0034] Step S406 The reference position determining unit 44 acquires the reception strength of the beacon signal received by the beacon receiving unit 42. After that, the process proceeds to step S407. Step S407 The reference position determination unit 44 compares the reception strength of the beacon signal with the signal strength threshold. If the reception strength of the beacon signal is less than the signal strength threshold (step S407; No), the process returns to step S404. If the reception strength of the beacon signal is equal to or greater than the signal strength threshold (step S407; Yes), the process proceeds to step S408. Step S408 The reference position determination unit 44 executes the reference position calculation process and ends the process.

[0035] As described above, the disclosed system comprises a transmitting device 30 that is installed in correspondence with a location where the user's movement speed decreases and emits a beacon signal in the surrounding area, and a user terminal device 40 carried by the user, and the user terminal device 40 is characterized in that it comprises a beacon receiving unit 42 that receives the beacon signal, an augmented reality processing unit 45 that displays a virtual object superimposed on an image captured of the surrounding area, a movement detection unit 43 that detects the movement direction and movement speed of the user terminal device 40 itself, and a reference position determination unit 44 that, when the movement speed detected by the movement detection unit 43 decreases and the beacon signal is received by the beacon receiving unit 42, sets a predetermined position located a predetermined distance in the movement direction from the current position of the user terminal device 40 as the reference position of the augmented reality processing unit. The components and operations described above provide a reference position for augmented reality in a simple and highly accurate manner.

[0036] In addition, when the movement speed detected by the movement detection unit 43 becomes equal to or greater than a first speed threshold and then falls below a second speed threshold, the reference position determination unit 44 compares the reception strength of the beacon signal with a signal strength threshold, and determines the reference position if the reception strength of the beacon signal is equal to or greater than the signal strength threshold. Therefore, the approach and stopping of the user can be easily determined, and the reference position can be calculated with high accuracy by utilizing the approach of the user.

[0037] In one example, the reference position determination unit 44 uses a fixed distance value as the predetermined distance. In this operation, the reference position can be obtained more easily. In one example, the reference position determination unit 44 uses a value corresponding to the reception strength as the predetermined distance. In this operation, the reference position can be determined with higher accuracy.

[0038] In one example, the movement detection unit 43 detects the moving direction and moving speed of the user terminal device itself by analyzing the video of the surroundings in real time. In this operation, the reference position can be obtained with a simpler configuration. In one example, the user terminal device 40a further includes an acceleration sensor 48 that measures the acceleration applied to the user terminal device 40a, and the movement detection unit 43a detects the movement direction and movement speed of the user terminal device 40a itself using the output of the acceleration sensor 48. In this operation, the reference position can be obtained with higher accuracy. Also, if the user terminal device 40a is a smartphone or the like, an acceleration sensor or the like provided for other functions can be effectively utilized.

[0039] The present invention is not limited to the above-mentioned embodiment, and various modifications are included. For example, the above-mentioned embodiment is described in detail to easily explain the present invention, and is not necessarily limited to the embodiment having all the described configurations. Moreover, the present invention is not limited to the deletion of the configurations, and it is also possible to replace or add the configurations. For example, in the above embodiment, a case where a BLE beacon signal is used has been illustrated, but the present invention can be used with any beacon signal such as UWB (Ultra Wide Band). In the above embodiment, a configuration in which movement detection is performed using camera coordinates and an acceleration sensor is exemplified, but any method can be used for movement detection. As shown in the above embodiment, when the user approaches the transmitting device in a substantially straight line, the positional relationship (distance, relative direction, horizontal direction) between the user terminal device 40 and the transmitting device 30 can be obtained with high accuracy. The user terminal device 40 may provide guidance so that the user approaches the transmitting device in a substantially straight line. [Explanation of symbols]

[0040] 10: user, 20: guide board, 30: transmitter, 40: user terminal device, 41: camera, 42: beacon receiver, 43: movement detector, 44: reference position determiner, 45: augmented reality processor, 46: server communication unit, 47: display unit, 48: acceleration sensor, 49: GPS unit

Claims

1. A transmitting device that is installed in association with a location where the user's moving speed decreases and transmits a beacon signal to the surrounding area; A user terminal device carried by the user, The user terminal device a beacon receiving unit for receiving the beacon signal; an augmented reality processing unit that displays virtual objects overlaid on a captured image of the surroundings; A movement detection unit that detects a moving direction and a moving speed of the user terminal device itself; a reference position determination unit that, when the moving speed detected by the movement detection unit decreases and the beacon receiving unit receives the beacon signal, sets a predetermined position located a predetermined distance in the moving direction from a current position of the user terminal device as a reference position of the augmented reality processing unit. A data processing system comprising:

2. 2. The data processing system of claim 1, The data processing system is characterized in that the reference position determination unit compares the reception strength of the beacon signal with a signal strength threshold when the movement speed detected by the movement detection unit becomes equal to or greater than a first speed threshold and then falls below a second speed threshold, and determines the reference position when the reception strength of the beacon signal is equal to or greater than the signal strength threshold.

3. 2. The data processing system of claim 1, A data processing system according to claim 1, wherein the reference position determination unit uses a fixed distance value as the predetermined distance.

4. 2. The data processing system of claim 1, The data processing system according to claim 1, wherein the reference position determining unit uses a value corresponding to the reception strength as the predetermined distance.

5. 2. The data processing system of claim 1, The data processing system is characterized in that the movement detection unit detects the direction and speed of movement of the user terminal device itself by analyzing video of the surroundings in real time.

6. 2. The data processing system of claim 1, The user terminal device further includes an acceleration sensor that measures acceleration applied to the user terminal device, The data processing system according to claim 1, wherein the movement detection unit detects a direction and speed of movement of the user terminal device itself by using an output from the acceleration sensor.

7. A user terminal device carried by a user, a beacon receiving unit that receives a beacon signal from a transmitting device that is installed in association with a location where the user's moving speed decreases; an augmented reality processing unit that displays virtual objects overlaid on a captured image of the surroundings; A movement detection unit that detects a moving direction and a moving speed of the user terminal device itself; a reference position determination unit that, when the moving speed detected by the movement detection unit decreases and the beacon receiving unit receives the beacon signal, sets a predetermined position located a predetermined distance in the moving direction from a current position of the user terminal device as a reference position of the augmented reality processing unit. A user terminal device comprising:

8. A movement detection step in which a user terminal device carried by a user detects a movement direction and a movement speed of the user terminal device itself; a beacon signal receiving step in which the user terminal device receives a beacon signal from a transmitting device installed in association with a location where the user's moving speed decreases; a reference position determination step of determining a reference position to be a predetermined position located a predetermined distance in the moving direction from a current position of the user terminal device when the moving speed detected by the moving detection step decreases and the user terminal device receives the beacon signal by the beacon signal receiving step; a step of performing an augmented reality process in which the user terminal device displays a virtual object by superimposing it on an image of the surroundings based on the reference position; 13. A data processing method comprising:

Images