Information processing system, information processing apparatus, program, and information processing method

The system facilitates the easy and cost-effective generation of motion information for virtual objects by using a terminal device to recognize and process identification information, addressing the challenge of expensive motion capture systems.

JP2024160553AActive Publication Date: 2024-11-14AKATSUKI INC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023075678
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-01
Publication Date
2024-11-14
Estimated Expiration
2043-05-01

AI Technical Summary

Technical Problem

Generating motion information for AR objects attached to objects moving over time is challenging due to the need for expensive and complex motion capture systems.

Method used

A system and method that utilizes a terminal device to recognize identification information associated with virtual objects, calculate position information, and generate motion information based on this recognition, allowing for easy generation of motion information without specialized equipment.

Benefits of technology

Enables easy and cost-effective generation of motion information for virtual objects by using a terminal device to recognize and process identification information, reducing the need for expensive motion capture systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024160553000001_ABST
    Figure 2024160553000001_ABST
Patent Text Reader

Abstract

To provide an information processing system, an information processing apparatus, a program, and an information processing method that easily generate motion information of an AR object.SOLUTION: An information processing system executes: first position information calculation processing of recognizing, by a terminal device, identification information associated with information for displaying a virtual object on the terminal device, thereby calculating first position information including the coordinates and angle of the identification information with reference to the terminal device; display processing of displaying, on the terminal device, a virtual object corresponding to the identification information on the basis of the first position information calculated in the first position information calculation processing; second position information calculation processing of calculating second position information including the coordinates and angle of the terminal device with reference to the identification information with which the virtual object displayed in the display processing is associated; and generation processing of generating motion information related to the virtual object on the basis of the second position information in a predetermined period, which is obtained by causing the terminal device to recognize the identification information over the predetermined period with the virtual object displayed thereon.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing device, a program, and an information processing method. [Background technology]

[0002] In recent years, technologies have been proposed to provide an augmented reality space (AR space), an extension of real space, allowing users to enjoy harmony between the real space and the image by overlaying computer graphics images onto the image of the real space that they can perceive.

[0003] Patent Document 1 discloses a technology that allows a terminal device to recognize an AR marker, display an AR object on the AR marker, and capture images of the AR object from multiple viewpoints to render the AR object as if it is moving. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2020-95645 Summary of the Invention [Problem to be solved by the invention]

[0005] When generating motion information including an AR object, the movement of an object (e.g., a person) having multiple AR markers attached thereto is captured by a camera (motion capture camera) for a predetermined period of time, and the motion information including the AR object is generated based on the captured information including the movement of the object.

[0006] In this case, it is necessary to secure space in which the object with the AR marker attached can move, and motion capture systems including cameras that capture the object's movements are expensive and not easy to introduce, so there are concerns that generating motion information including AR objects will be difficult.

[0007] The present invention has been made in consideration of the above circumstances, and has an objective of providing an information processing system, an information processing device, a program, and an information processing method that can easily generate motion information related to a virtual object. [Means for solving the problem]

[0008] In order to achieve the above object, an information processing system according to the present invention executes a first position information calculation process in which a terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information based on the terminal device; a display process in which the terminal device displays the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process in which second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; and a generation process in which motion information related to the virtual object is generated based on the second position information for the predetermined period obtained by having the terminal device recognize the identification information for the predetermined period with the virtual object displayed.

[0009] According to this, motion information can be generated based on the second position information calculated by having the terminal device recognize the identification information, so that motion information can be easily generated without requiring a special environment, equipment, or device, or performing complicated operations.

[0010] In order to achieve the above object, an information processing device according to the present invention executes, in the generation process, a complementation process for complementing a movement of the terminal device during a period during which the terminal device is unable to recognize the identification information within the predetermined period, based on information during a period during which the terminal device recognizes the identification information. 3. The information processing system according to claim 1, further comprising: a first position information calculation process for calculating first position information including coordinates and an angle of the identification information based on the terminal device by the terminal device recognizing the identification information associated with information for displaying a virtual object on the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process of generating motion information related to the virtual object based on the second position information during a predetermined period of time, the motion information being obtained by causing the terminal device to recognize the identification information for the predetermined period of time while the virtual object is displayed; The control unit executes the above.

[0011] A program according to the present invention for achieving the above object includes a first position information calculation process for calculating first position information including coordinates and an angle of the identification information based on the terminal device by having the terminal device recognize identification information associated with information for displaying a virtual object on the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process of generating motion information related to the virtual object based on the second position information during a predetermined period of time, the motion information being obtained by causing the terminal device to recognize the identification information for the predetermined period of time while the virtual object is displayed; The present invention is directed to a computer that executes the above.

[0012] In order to achieve the above object, an information processing method according to the present invention includes a first position information calculation process in which a terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and the first position information including coordinates and an angle of the identification information relative to the terminal device is calculated; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process of generating motion information related to the virtual object based on the second position information during a predetermined period of time, the motion information being obtained by causing the terminal device to recognize the identification information for the predetermined period of time while the virtual object is displayed; The above is executed by a computer. Effect of the Invention

[0013] According to this invention, it is possible to easily generate motion information regarding a virtual object. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram illustrating an outline of a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a block diagram illustrating an outline of the configuration of a user terminal of the information processing system according to the present embodiment. [Diagram 3]FIG. 1 is a block diagram for explaining an outline of the functions of a user terminal of the information processing system according to the present embodiment. [Figure 4] FIG. 13 is a block diagram for explaining an outline of processing in the functions of a user terminal of the information processing system according to the present embodiment. [Diagram 5] FIG. 13 is a block diagram for explaining an outline of processing in the functions of a user terminal of the information processing system according to the present embodiment. [Figure 6] FIG. 13 is a block diagram for explaining an outline of processing in the functions of a user terminal of the information processing system according to the present embodiment. [Figure 7] FIG. 2 is a block diagram illustrating an outline of the configuration of a server in the information processing system according to the present embodiment. [Figure 8] FIG. 2 is a block diagram for explaining an outline of the functions of the server of the information processing system according to the present embodiment. [Figure 9] FIG. 13 is a diagram for explaining an outline of virtual object information of the information processing system according to the present embodiment. [Figure 10] 10 is a flowchart outlining the processing of the information processing system according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Next, an information processing system according to an embodiment of the present invention will be described with reference to FIGS.

[0016] 1 is a block diagram illustrating an outline of a configuration of an information processing system according to an embodiment of the present invention. As shown in the figure, the information processing system 10 mainly comprises a user terminal 20 which is a terminal device that recognizes an AR marker M which is identification information, and a server 30 which is an information processing device.

[0017] In this embodiment, this information processing system 10 includes a user terminal 20 owned by a user 1 and a server 30 provided at a business operator 2, which are connected to each other so as to be accessible to each other via a network N such as the Internet.

[0018] In this embodiment, the AR marker M is identification information associated with (stored in) information (e.g., drawing information) for displaying an AR object O (virtual object) on a terminal device, and when the AR marker M is recognized by a user terminal 20 held by a user 1, the AR object O is displayed on the user terminal 20.

[0019] Next, a specific configuration of each part of the game system 10 of the present embodiment will be described.

[0020] In this embodiment, the user terminal 20 is implemented as a smartphone, which is a portable information terminal, but may also be implemented as, for example, a tablet computer, desktop computer, or notebook computer.

[0021] 2 is a block diagram illustrating an outline of the configuration of the user terminal 20. As shown in the figure, the user terminal 20 includes a control unit 21, a camera 22, a display 23, and sensors 24 as its main components.

[0022] In this embodiment, the control unit 21 controls each part of the user terminal 20, such as the camera 22, the display 23, and the sensors 24, and is composed of, for example, a processor, a memory, a storage, a transmission / reception unit, and the like.

[0023] In this embodiment, various applications that are activated on the user terminal 20 are stored in the control unit 21, and in this embodiment, an application that recognizes the AR marker M and processes the acquired information is stored in the control unit 21.

[0024] The camera 22 is disposed on the opposite side of the display 23 facing the user 1 (the back surface of the user terminal 20), and in this embodiment operates as an interface that recognizes the AR marker M by photographing it in cooperation with an application stored in the control unit 21. The camera 22 may be disposed on the front surface (the surface on which the display 23 is provided) or the side surface of the user terminal 20.

[0025] In this embodiment, the display 23 displays a screen interface of an application executed on the user terminal 20, and also renders an AR object O by recognizing the AR marker M.

[0026] The display 23 is a so-called touch panel that accepts information input by touching the display surface, and is implemented using various technologies such as a resistive film type or a capacitive type.

[0027] Information is input to the display 23 based on any action of the player 2 on the display 32 (for example, tapping, double tapping, long tapping, multi-touching, pinching in, pinching out, sliding, swiping the screen, or dragging and dropping an icon displayed on the screen, etc.).

[0028] In this embodiment, the sensors 24 are configured with a gyro sensor, an acceleration sensor, a GPS, etc., and detect the coordinate position and angle of the user terminal 20 in any space. The sensors 24 can detect the inclination of the user terminal 20 (angle with respect to the horizontal and vertical directions).

[0029] 3 is a block diagram illustrating an outline of functions of the user terminal 20. As illustrated, in this embodiment, the user terminal 20 includes a first position information calculation unit 20a, a generation unit 20b, a second position information calculation unit 20c, and a transmission / reception unit 20d.

[0030] The first position information calculation unit 20a, the generation unit 20b, the second position information calculation unit 20c, and the transmission / reception unit 20d are realized by causing the processor of the control unit 21 to execute a program stored in the memory of the control unit 21.

[0031] 4 is a diagram for explaining an outline of the process of the first position information calculation unit 20a. As shown in the figure, when the user terminal 20 recognizes the AR marker M via the camera 22, the first position information calculation unit 20a calculates, as first position information D1, position information including coordinates and an angle in a space in which the AR marker M is placed with respect to the user terminal 20 (first position information calculation process).

[0032] FIG. 5 is a diagram for explaining an outline of the process of the generating unit 20b. In this embodiment, the generating unit 20b draws an AR object O corresponding to the AR marker M on the display 23 of the user terminal 20 as shown in the figure based on the first position information D1 calculated in the first position information calculation process and AR object information (described later) associated with the AR marker M (display process). The AR object O is displayed on the display 23 at a position directly above the AR marker M and in a direction corresponding to the direction of the AR marker M (for example, a direction in which one side of the AR marker is in front of the AR object). When the user moves the user terminal 20, the relative positional relationship (three-dimensional position, direction) between the AR marker M and the user terminal 20 changes, and the AR object O displayed on the display 23 also changes in response to the change. That is, for example, when the user terminal 20 recognizes the AR marker M from one side of the AR marker M, an image of the front of the AR object O is displayed, and when the user terminal 20 recognizes the AR marker M from the opposite side of the AR marker M, an image of the back of the AR object O is displayed. The position and orientation of the AR object O relative to the AR marker M are basically constant, but may be changed according to a program stored in advance.

[0033] 6 is a diagram for explaining an outline of the process of the second position information calculation unit 20c. As shown in the figure, the second position information calculation unit 20c calculates, as second position information D2, position information including coordinates and an angle in the space of the user terminal 20 based on the AR marker M in which the drawing information of the AR object O drawn in the display process is stored (second position information calculation process).

[0034] In this embodiment, the second position information calculation unit 20c performs inverse calculation on the first position information D1 calculated in the first position information calculation process by a coordinate conversion process to calculate, as second position information D2, the relative coordinates and relative angle of the user terminal 20 with respect to the AR marker M. Specifically, for example, by a known coordinate conversion process using a coordinate conversion matrix or the like, the second position information D2 is calculated by converting a coordinate system having the position and orientation of the camera of the user terminal 20 as the origin into a coordinate system having the position and orientation of the AR marker M as the reference.

[0035] In this embodiment, the transmitting / receiving unit 20d shown in FIG. 3 executes a process of transmitting information such as the second location information D2 calculated in the second location information calculation process to the server 30 (transmission process). The transmitting / receiving unit 20d connects the user terminal 20 to the network N. This transmitting / receiving unit 20d may be equipped with a short-range communication interface such as Bluetooth (registered trademark) or BLE (Bluetooth Low Energy). In this embodiment, the first location information calculation process, the display process, the second location information calculation process, and the transmission process are repeatedly and continuously performed at predetermined intervals (for example, at a fixed interval such as every 0.1 seconds or 0.5 seconds). In addition, the order of the display process and the second location information calculation process is not limited, and either may come first, or they may be performed simultaneously.

[0036] In this embodiment, the server 30 shown in FIG. 1 is implemented by a desktop or notebook computer.

[0037] 7 is a block diagram illustrating an outline of the configuration of the server 30. As shown in the figure, the server 30 includes a processor 31, a memory 32, a storage 33, a transmission / reception unit 34, and an input / output unit 35, which are electrically connected to each other via a bus 36.

[0038] The processor 31 is an arithmetic device that controls the operation of the server 30, controls the transmission and reception of data between each element, and performs processes required for executing application programs.

[0039] In this embodiment, the processor 31 is, for example, a CPU (Central Processing Unit), and executes application programs stored in a storage 33 (to be described later) and deployed in a memory 32 to perform various processes.

[0040] In this embodiment, the memory 32 is implemented as a main storage device configured as a volatile storage device such as a dynamic random access memory (DRAM).

[0041] This memory 32 is used as a working area for the processor 31, and also stores a basic input / output system (BIOS) that is executed when the server 30 is started up, various setting information, and the like.

[0042] The storage 33 stores programs, information used for various processes, and the like.

[0043] The transceiver 34 connects the server 30 to the network N. The transceiver 34 may include a short-range communication interface such as Bluetooth (registered trademark) or BLE (Bluetooth Low Energy).

[0044] The input / output unit 35 is an interface to which input / output devices such as a keyboard, a mouse, or a display are connected.

[0045] The bus 36 transmits, for example, address signals, data signals, and various control signals between the connected processor 31, memory 32, storage 33, transmission / reception unit 34, and input / output unit 35.

[0046] 8 is a block diagram illustrating an outline of the functions of the server 30. As shown in the figure, the server 30 in this embodiment includes a storage unit 30a, a receiving unit 30b, and a generating unit 30c.

[0047] The storage unit 30a is realized by partitioning a memory area of ​​the storage 33. In this embodiment, the storage unit 30a stores second position information D2, AR object information D3, and motion information D4.

[0048] 9 is a diagram for explaining an outline of the AR object information D3. As shown in the figure, in this embodiment, the AR object information D3 is a database or library composed of a plurality of AR objects each assigned with identification information (such as an identification number) of an AR marker. The AR object is, for example, an image of any target object such as a three-dimensional character, item, building, plant, animal, person, etc., and the AR object information includes three-dimensional model data information. The AR object information is not limited to a stationary object, and may include a moving object.

[0049] The motion information D4 shown in FIG. 8 can be video information that records an image including an AR object O. The motion information D4 may include camera operation information including the position and angle of a user terminal (camera) for a predetermined period based on an AR marker. In this embodiment, the motion information D4 can be used as a material for realizing image processing of a character in a game executed on a computer such as a smartphone or a game terminal. More specifically, when generating an image including a character in a game, the camera operation information can be used as operation information of a virtual camera (viewpoint) in the game.

[0050] In this embodiment, the receiving unit 30b and the generating unit 30c are realized by causing the processor 31 to execute a program stored in the memory 32.

[0051] In this embodiment, the receiver 30b executes a reception process for receiving the second location information D2 transmitted from the user terminal 20.

[0052] In the present embodiment, the generation unit 30c executes a process of generating motion information D4 of an arbitrary AR object O based on the second position information D2 calculated in the second position information calculation process of the user terminal 20 (generation process).

[0053] In this embodiment, this generation process generates motion information D4 of the AR object O by writing out data that has been recorded (video recorded) in association with AR object information D3 associated with the AR marker M and second position information D2 for a predetermined period (converting the data format as appropriate).

[0054] Next, an overview of the processing of the information processing system 10 according to the present embodiment will be described.

[0055] 10 is a flowchart outlining the processing of the information processing system 10 according to this embodiment. As shown in the figure, first, in step S1, when the user 1 starts an application on the user terminal 20 and causes the application to recognize the AR marker M via the camera 22, first position information D1 in a space in which the AR marker M is located, relative to the user terminal 20, is calculated (first position information calculation processing).

[0056] In this embodiment, the first position information D1 includes position information including coordinates and angles in the space (real space) in which the AR marker M is placed. More specifically, the first position information D1 includes information on the three-dimensional coordinates and three-dimensional direction (angle) of the AR marker M on a user terminal coordinate system having a predetermined point on the user terminal 20 as the origin.

[0057] In the following step S2, image data of an AR object O (AR character) corresponding to the AR marker M is generated based on the first position information D1 calculated in the first position information calculation process and the AR object information D3, and is displayed on the display 23 of the user terminal 20 (display process). An AR image in which the AR object O is superimposed on the real space including the AR marker M is displayed on the display 23.

[0058] On the other hand, in step S3, second position information D2 in the space of the user terminal 20 is calculated based on the AR marker M in which the drawing information of the AR object O drawn in the drawing process is stored (second position information calculation process).

[0059] In this embodiment, the second position information calculation unit 20c includes the relative coordinates and the relative angle of the user terminal 20 with respect to the AR marker M. The second position information calculation unit 20c includes information on the three-dimensional coordinates and the three-dimensional direction (angle) of the user terminal 20 on an AR marker coordinate system whose origin is the center point of the AR marker, for example.

[0060] After calculating the second position information D2, the user terminal 20 transmits the calculated second position information D2 to the server 30 in step S4 (transmission process).

[0061] When the server 30 receives the second position information D2 transmitted from the user terminal 20 (reception process), in step S5, the server 30 associates the received second position information D2 with the AR object information D3 and stores them. In addition, in step S6, the server 30 generates motion information D4 regarding the AR object O based on the received data (stored data) (generation process).

[0062] With the AR object O displayed on the user terminal 20 by the display process, the user causes the user terminal 20 to recognize the AR marker M for an arbitrary predetermined period. As a result, the server 30 repeatedly executes the second position information calculation process for a predetermined period, calculates the second position information for the predetermined period, and generates motion information related to the movement of the user terminal 20 based on the AR marker M (on the AR marker coordinate system). The user may change the relative distance by moving the user terminal 20 closer to or farther away from the AR marker, i.e., the AR object, during the predetermined period, or may move the user terminal 20 around the AR marker to photograph the AR marker, i.e., the AR object, from various angles. This generates motion information in which the display size (size) and angle of the AR object change according to changes in the position and angle of the user terminal relative to the AR marker.

[0063] The predetermined period may be a period between a start operation and an end operation received from a user who uses the user terminal 20. The start operation may be, for example, an operation in which the user selects a shooting start icon displayed on the user terminal 20 in a state in which the AR object O is displayed on the user terminal 20 by the display process. The end operation may be an operation in which the user selects a shooting end icon displayed on the user terminal 20. Alternatively, the predetermined period may be a period during which the user continuously selects (long presses) the shooting icon displayed on the user terminal 20. In that case, the operation of starting to select the shooting icon (starting to touch the touch panel) is the start operation, and the operation of releasing the finger from the shooting icon is the end operation.

[0064] The generated motion information D4 can be used, for example, as material for implementing image processing of characters in image expressions such as games and animated movies.

[0065] In this way, since motion information D4 of the AR object O can be generated based on the second position information D2 calculated as a result of recognizing the AR marker M on the user terminal 20, the motion information D4 can be easily generated without securing special facilities or introducing special devices.

[0066] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit of the invention.

[0067] In the above embodiment, a case has been described in which the programs stored in the user terminal 20 and the server 30 execute information processing related to the AR object, but the information processing related to the AR object may be executed in the user terminal 20 without distributing the processing between the user terminal 20 and the server 30. In the above description, at least a part of the information processing executed in the user terminal 20 may be executed in the server 30, or a part of the information processing executed in the server 30 may be executed in the user terminal 20.

[0068] In this embodiment, a tilt information calculation process may be executed to calculate tilt information of the AR marker based on sensor information acquired from a sensor (sensors 24) capable of detecting the tilt of the user terminal 20. In this case, when generating motion information in the generation process, a tilt correction process may be executed to correct the tilt of at least one of the AR object and the virtual camera based on the tilt information. According to this, even if the AR marker is placed on an oblique surface inclined with respect to the horizontal plane, it is possible to generate the same motion data as when the AR marker is placed on a horizontal plane. As a result, it is possible to prevent the AR object displayed on the AR marker from being displayed tilted. When calculating the tilt information of the AR marker, the tilt of the AR marker with respect to the horizontal direction (or vertical direction) can be calculated based on the tilt of the user terminal 20 with respect to the horizontal direction (or vertical direction) and information on the relative angle of the AR marker with respect to the user terminal.

[0069] Furthermore, in this embodiment, the generation process may be configured to execute a complementation process that complements the motion of the terminal device during a period during which the user terminal 20 cannot recognize the AR marker in a predetermined period based on information during a period during which the terminal device recognizes the identification information. For example, the motion information during the period during which the AR marker cannot be recognized can be complemented by a process of connecting (connecting) the position of the terminal device immediately before the period during which the AR marker cannot be recognized with the position of the terminal device immediately after the period during which the AR marker cannot be recognized with a line. This makes it possible to generate motion information even if there is a period during which the AR marker cannot be recognized temporarily in the predetermined period.

[0070] The present invention has the following configuration. (Item 1) a first position information calculation process in which the terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information relative to the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process of generating motion information related to the virtual object based on the second position information during a predetermined period of time, the motion information being obtained by causing the terminal device to recognize the identification information for the predetermined period of time while the virtual object is displayed; An information processing system that executes the above. (Item 2) The second position information calculation process includes: Calculating the second position information by performing a coordinate conversion process on the first position information calculated in the first position information calculation process. Item 1. An information processing system according to item 1. (Item 3) the predetermined period is a period between a start operation and an end operation received from a user of the terminal device; generating the motion information based on AR object information regarding an AR object serving as the virtual object associated with the drawing information stored in the identification information and the second position information; 3. The information processing system according to item 1 or 2. (Item 4) The identification information is The virtual object is an AR marker associated with information of the AR object. 3. The information processing system according to item 1 or 2. (Item 5) a tilt information calculation process for calculating tilt information of the identification information based on sensor information acquired from a sensor capable of detecting a tilt of the terminal device; a tilt correction process for correcting a tilt based on the tilt information when generating the motion information in the generation process; 3. The information processing system according to item 1 or 2. (Item 6) a complementation process for complementing a movement of the terminal device during a period during which the terminal device is unable to recognize the identification information within the predetermined period, based on information during a period during which the terminal device recognizes the identification information, in the generation process; 3. The information processing system according to item 1 or 2. (Item 7) a first position information calculation process in which the terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information relative to the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process of generating motion information related to the virtual object based on the second position information during a predetermined period of time, the motion information being obtained by causing the terminal device to recognize the identification information for the predetermined period of time while the virtual object is displayed; An information processing device comprising a control unit that executes the above. (Item 8) a first position information calculation process in which the terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information relative to the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process of generating motion information related to the virtual object based on the second position information during a predetermined period of time, the motion information being obtained by causing the terminal device to recognize the identification information for the predetermined period of time while the virtual object is displayed; A program that causes a computer to execute the following. (Item 9) a first position information calculation process in which the terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information relative to the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process of generating motion information related to the virtual object based on the second position information during a predetermined period of time, the motion information being obtained by causing the terminal device to recognize the identification information for the predetermined period of time while the virtual object is displayed; An information processing method implemented by a computer. [Explanation of symbols]

[0071] 1 User 2 Business operators 10. Information Processing Systems 20 User terminal (terminal device) 30 Server (information processing device)

Claims

1. a first position information calculation process in which the terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information relative to the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process for generating motion information related to the virtual object based on the second position information during a predetermined period, the second position information being obtained by causing the terminal device to recognize the identification information for the predetermined period while the virtual object is displayed; An information processing system that executes the above.

2. The second position information calculation process includes: calculating the second position information by subjecting the first position information calculated in the first position information calculation process to a coordinate conversion process; The information processing system according to claim 1 .

3. the predetermined period is a period between a start operation and an end operation received from a user of the terminal device; generating the motion information based on AR object information regarding an AR object serving as the virtual object associated with the drawing information stored in the identification information and the second position information; 3. The information processing system according to claim 1 or 2.

4. The identification information is The virtual object is an AR marker associated with information of the AR object.

3. The information processing system according to claim 1 or 2.

5. a tilt information calculation process for calculating tilt information of the identification information based on sensor information acquired from a sensor capable of detecting a tilt of the terminal device; a tilt correction process for correcting a tilt based on the tilt information when generating the motion information in the generation process; 3. The information processing system according to claim 1 or 2.

6. a complementation process for complementing a movement of the terminal device during a period during which the terminal device is unable to recognize the identification information within the predetermined period, based on information during a period during which the terminal device recognizes the identification information, in the generation process; 3. The information processing system according to claim 1 or 2.

7. a first position information calculation process in which the terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information relative to the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process for generating motion information related to the virtual object based on the second position information during a predetermined period, the second position information being obtained by causing the terminal device to recognize the identification information for the predetermined period while the virtual object is displayed; An information processing device comprising a control unit that executes the above.

8. a first position information calculation process in which the terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information relative to the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process for generating motion information related to the virtual object based on the second position information during a predetermined period, the second position information being obtained by causing the terminal device to recognize the identification information for the predetermined period while the virtual object is displayed; A program that causes a computer to execute the following.

9. a first position information calculation process in which the terminal device recognizes identification information associated with information for displaying a virtual object on the terminal device, and calculates first position information including coordinates and an angle of the identification information relative to the terminal device; a display process of displaying, on the terminal device, the virtual object corresponding to the identification information based on the first position information calculated in the first position information calculation process; a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device based on the identification information associated with the virtual object displayed in the display process; a generation process for generating motion information related to the virtual object based on the second position information during a predetermined period, the second position information being obtained by causing the terminal device to recognize the identification information for the predetermined period while the virtual object is displayed; An information processing method implemented by a computer.

Citation Information

Patent Citations

  • Image processing apparatus, terminal device, image processing method, and program

    JP2014071850A

  • Display method, display program and display device

    JP2015204079A

  • Mixed reality system, program, method, and mobile terminal device

    JP2019133507A

  • Display program, display device, and display method

    JP2020095645A