Information processing system, information processing apparatus, program, and information processing method
The information processing system simplifies the generation of AR object motion information by using standard devices to recognize and calculate location information, eliminating the need for expensive motion capture systems.
Patent Information
- Application Number
- JP2023180675
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-10-20
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2043-05-01
AI Technical Summary
Generating motion information for AR objects is challenging due to the need for multiple AR markers and expensive motion capture systems, making it difficult to introduce and operate.
An information processing system that transfers a virtual object to a terminal device, recognizing identification information to calculate first and second location information, and generates motion information based on this data without requiring specialized equipment.
Enables easy generation of motion information for AR objects using standard devices, reducing the need for expensive motion capture systems and simplifying the process.
Smart Images

Figure 2024160671000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing system, an information processing device, a program, and an information processing method. [Background technology]
[0002] In recent years, computer graphics images have been added to images in real space that users can perceive. An extension of the real world that allows you to enjoy the harmony between the real world and the image by overlaying them. Technologies for providing a real space (AR space) have been proposed.
[0003] In Patent Document 1, an AR object is displayed on an AR marker by recognizing the AR marker with a terminal device. By displaying an AR object and capturing images of the AR object from multiple viewpoints, The technology disclosed here allows AR objects to be rendered as if they are moving. do. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent Publication No. 2020-95645 Summary of the Invention [Problem to be solved by the invention]
[0005] By the way, when generating motion information including AR objects, The movement of an object (e.g. a person) to which a mosquito is attached is recorded by a camera (motion capture camera) in a predetermined manner. AR objects are created based on the information captured over a period of time, including the object's movements. The motion information including the above will be generated.
[0006] In this case, it is necessary to secure a space for the object with the AR marker to operate. Moreover, the motion capture system, which includes a camera that captures the object's movements, is expensive. Therefore, the generation of motion information including AR objects requires a There are fears that this will be difficult.
[0007] The present invention has been made in consideration of the above circumstances, and relates to a motion control method for a virtual object. Information processing system, information processing device, program, and method for easily generating application information An object of the present invention is to provide an information processing method. [Means for solving the problem]
[0008] In order to achieve the above object, an information processing system according to the present invention provides a virtual object to a terminal. The terminal device recognizes identification information associated with information to be displayed on the device. and calculating first position information including the coordinates and angle of the identification information based on the terminal device. and a first position information calculation process based on the first position information calculated in the first position information calculation process. a display process for displaying the virtual object corresponding to the identification information on the terminal device and based on the identification information associated with the virtual object displayed in the display process. a second position information calculation process for calculating second position information including the coordinates and angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information and generating motion information relating to the virtual object. That is why.
[0009] According to this, based on the second location information calculated by having the terminal device recognize the identification information, Since motion information can be generated by using the It allows users to easily generate motion information without having to perform complex operations or Cut.
[0010] In order to achieve the above object, the information processing device according to the present invention comprises, in the generation process: The movement of the terminal device during a period in which the terminal device cannot recognize the identification information during a predetermined period A complementation process for complementing the identification information based on information on a period during which the terminal device recognizes the identification information. Execute, 3. The information processing system according to claim 1, wherein the virtual object is displayed on a terminal device. The terminal device recognizes the identification information associated with the information for calculating first position information including coordinates and an angle of the identification information based on the device; Information calculation processing; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; The control unit executes the above.
[0011] The program according to the present invention for achieving the above object is a program for displaying a virtual object on a terminal device. The terminal device recognizes identification information associated with information to be displayed. A first position information calculating unit that calculates first position information including coordinates and an angle of the identification information based on the terminal device. A position information calculation process; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; The present invention is directed to a computer that executes the above.
[0012] In order to achieve the above object, an information processing method according to the present invention includes: The terminal device recognizes identification information associated with information to be displayed on the calculating first position information including coordinates and an angle of the identification information based on the terminal device; 1. Location information calculation process; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; The above is executed by a computer. Effect of the Invention
[0013] According to the present invention, it is possible to easily generate motion information regarding a virtual object. can. [Brief description of the drawings]
[0014] [Figure 1] 1 is a block diagram illustrating an outline of a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 1 is a block diagram illustrating an outline of the configuration of a user terminal of the information processing system according to the present embodiment. [Diagram 3] FIG. 1 is a block diagram for explaining an outline of the functions of a user terminal of the information processing system according to the present embodiment. [Figure 4] FIG. 13 is a block diagram for explaining an outline of processing in the functions of a user terminal of the information processing system according to the present embodiment. [Diagram 5] FIG. 13 is a block diagram for explaining an outline of processing in the functions of a user terminal of the information processing system according to the present embodiment. [Figure 6] FIG. 13 is a block diagram for explaining an outline of processing in the functions of a user terminal of the information processing system according to the present embodiment. [Figure 7] FIG. 2 is a block diagram illustrating an outline of the configuration of a server in the information processing system according to the present embodiment. [Figure 8] FIG. 2 is a block diagram for explaining an outline of the functions of the server of the information processing system according to the present embodiment. [Figure 9]FIG. 13 is a diagram for explaining an outline of virtual object information of the information processing system according to the present embodiment. [Figure 10] 10 is a flowchart outlining the processing of the information processing system according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Next, an information processing system according to an embodiment of the present invention will be described with reference to FIGS. explain.
[0016] FIG. 1 is a block diagram illustrating an outline of the configuration of an information processing system according to an embodiment of the present invention. As shown in the figure, the information processing system 10 recognizes an AR marker M, which is identification information. The main components are a user terminal 20 which is a terminal device for performing the operations and a server 30 which is an information processing device. Prepare.
[0017] In this embodiment, the information processing system 10 includes a user terminal 20 owned by a user 1. The server 30 is provided in the business operator 2, and these are connected to a network N such as the Internet. , and are accessible to each other via .
[0018] In this embodiment, the AR marker M is an AR object O (virtual object) that is displayed on a terminal. Identification information associated with (stored in) information to be displayed on the device (e.g., drawing information) When the information is recognized by the user terminal 20 held by the user 1, the AR object O is It is displayed on the user terminal 20.
[0019] Next, a specific configuration of each part of the game system 10 of the present embodiment will be described.
[0020] In this embodiment, the user terminal 20 is implemented as a smartphone, which is a portable information terminal. For example, tablet computers, desktop or notebook computers It may be implemented by a computer.
[0021] FIG. 2 is a block diagram for explaining the outline of the configuration of the user terminal 20. As shown in the figure, The user terminal 20 mainly comprises a control unit 21, a camera 22, a display 23, and sensors 24. Prepare as a member.
[0022] In this embodiment, the control unit 21 controls a camera 22, a display 23, sensors 24, etc. The device controls each part of the user terminal 20, such as a processor, memory, storage, etc. It is composed of a transmitter / receiver unit, etc.
[0023] In this embodiment, the control unit 21 controls various applications that are started on the user terminal 20. In this embodiment, the AR marker M is recognized and the acquired information is processed. The applications to be used are stored.
[0024] The camera 22 is positioned on the opposite side of the display 23 facing the user 1 (the rear surface of the user terminal 20). ) and in this embodiment, it cooperates with the application stored in the control unit 21. It acts as an interface that recognizes the AR marker M by photographing it. Camera 2 2 is provided on the front surface (the surface on which the display 23 is provided) or the side surface of the user terminal 20. Good too.
[0025] In this embodiment, the display 23 displays the application executed by the user terminal 20. The screen interface, such as the application, is displayed, and the AR marker M is recognized. Thus, the AR object O is rendered.
[0026] The display 23 is a so-called touch panel that accepts information input by touching the display surface. touch panels implemented using various technologies such as resistive and capacitive touch panels .
[0027] The input of information to the display 23 is performed by any movement of the player 2 relative to the display 32. (e.g. tap the screen, double tap, long tap, multi-touch, pinch in, You can also move the screen by opening, sliding, swiping, or dragging icons displayed on the screen. The input is based on the action of dropping the
[0028] In this embodiment, the sensors 24 include a gyro sensor, an acceleration sensor, a GPS, etc. The sensor detects the coordinate position, angle, etc. of the user terminal 20 in any space. 24 can detect the inclination of the user terminal 20 (angle with respect to the horizontal and vertical directions). Cut.
[0029] FIG. 3 is a block diagram for explaining an outline of the functions of the user terminal 20. As shown in the figure, In this embodiment, the user terminal 20 includes a first position information calculation unit 20a, a generation unit 20b, a second position information calculation unit 20c, and a The device includes a location information calculation unit 20c and a transmission / reception unit 20d.
[0030] The first location information calculation unit 20a, the generation unit 20b, the second location information calculation unit 20c, and the transmission and reception The reception unit 20d executes a program stored in the memory of the control unit 21 by the processor of the control unit 21. This is achieved by executing it.
[0031] FIG. 4 is a diagram for explaining an outline of the process of the first position information calculation unit 20a. As shown in the figure, When the user terminal 20 recognizes the AR marker M via the camera 22, the first position information calculation unit 2 0a is the coordinates and angle in the space where the AR marker M is placed, based on the user terminal 20. The position information including the degree is calculated as first position information D1 (first position information calculation process).
[0032] FIG. 5 is a diagram for explaining an outline of the process of the generation unit 20b. In this embodiment, the first position information D1 calculated in the first position information calculation process is associated with the AR marker M. Based on the AR object information (to be described later) thus obtained, as shown in the figure, The AR object O is rendered on the display 23 of the user terminal 20 (display process). The spray 23 is located directly above the AR marker M and is oriented in a direction (e.g. For example, the AR marker is oriented so that one side of the AR object faces the AR object. When the user moves the user terminal 20, the AR marker M and the user terminal 20 are displayed. The relative position (three-dimensional position, direction) changes, and the display 23 responds accordingly. The displayed AR object O also changes. That is, for example, from one side of the AR marker M When the user terminal 20 recognizes the AR marker M from the AR object O, an image of the front of the AR object O is displayed. When the user terminal 20 recognizes the AR marker M from the opposite side of the AR marker M, the AR object The image of the back of the AR object O is displayed. The position and orientation are basically constant, but may be changed according to a pre-stored program. stomach.
[0033] FIG. 6 is a diagram for explaining an outline of the process of the second position information calculation unit 20c. As shown in the figure, The second position information calculation unit 20c stores the drawing information of the AR object O drawn in the display process. A position including coordinates and an angle in the space of the user terminal 20 based on the AR marker M The information is calculated as second position information D2 (second position information calculation process).
[0034] In this embodiment, the second position information calculation unit 20c calculates the first position information in the first position information calculation process. The first position information D1 is then calculated by a coordinate conversion process to obtain the AR marker M as a reference. The relative coordinates and the relative angle of the user terminal 20 in this case are calculated as the second position information D2. Specifically, for example, the user can use a known coordinate conversion process using a coordinate conversion matrix or the like. A coordinate system with the position and orientation of the camera of the terminal 20 as the origin is defined as the position and orientation of the AR marker M. The second position information D2 is calculated by converting the coordinate system into a reference coordinate system.
[0035] In this embodiment, the transmitting / receiving unit 20d shown in FIG. 2. Execute a process of transmitting information such as the location information D2 to the server 30 (transmission process). The transmission / reception unit 20d connects the user terminal 20 to the network N. Bluetooth (registered trademark) and BLE (Bluetooth Low Energy) The present embodiment may be provided with a short-range communication interface. Then, the first location information calculation process, the display process, the second location information calculation process, and the transmission process are performed for a predetermined period ( For example, this is repeated every 0.1 seconds, 0.5 seconds, or other fixed period. The order of the process and the second position information calculation process is not limited, and either process may be performed first. , or simultaneously.
[0036] In this embodiment, the server 30 shown in FIG. It is implemented by a computer.
[0037] FIG. 7 is a block diagram for explaining the outline of the configuration of the server 30. As shown in the figure, the server 30 includes a processor 31, a memory 32, a storage 33, a transmission / reception unit 34, and an input / output unit 3 5, which are electrically connected to each other via a bus 36.
[0038] The processor 31 controls the operation of the server 30 and controls the transmission and reception of data between each element. The CPU is a computing device that performs the necessary control and processing to execute application programs.
[0039] In this embodiment, the processor 31 is, for example, a CPU (Central Processor). The data is stored in the storage 33 (described later) and expanded in the memory 32. The CPU 10 executes the application program and performs each process.
[0040] In this embodiment, the memory 32 is a dynamic random access memory (DRAM). It is implemented in a main memory consisting of a volatile memory device such as an ESS Memory.
[0041] The memory 32 is used as a working area for the processor 31 and is also used for starting up the server 30. BIOS (Basic Input / Output System), which runs during startup, Also various setting information etc. are stored.
[0042] The storage 33 stores programs, information used for various processes, and the like.
[0043] The transmitting / receiving unit 34 connects the server 30 to the network N. The transmitting / receiving unit 34 is Bluetooth (registered trademark) and BLE (Bluetooth Low Energy) The communication device may be provided with a short-range communication interface such as a wireless communication device.
[0044] The input / output unit 35 is connected to input / output devices such as a keyboard, a mouse, or a display. It is an interface that can be used.
[0045] The bus 36 connects a processor 31, a memory 32, a storage 33, a transceiver 34, and and the input / output unit 35, for example, address signals, data signals, and various control signals are transmitted. Reach.
[0046] FIG. 8 is a block diagram for explaining an outline of the functions of the server 30. As shown in the figure, the server In this embodiment, the device 30 includes a storage unit 30a, a receiving unit 30b, and a generating unit 30c.
[0047] The memory unit 30a is realized by partitioning the memory area of the storage 33. In this embodiment, the storage unit 30a stores second position information D2, AR object information D3, and motion information D4 is stored.
[0048] FIG. 9 is a diagram for explaining an outline of the AR object information D3. In this embodiment, the object information D3 is the identification information (identification number, etc.) of the AR marker. A database or library that consists of multiple AR objects assigned to each AR objects are, for example, 3D characters, items, buildings, plants, etc. It is an image of any object such as an object, animal, or person, and the AR object information is a 3D model The AR object information is not limited to static objects, but also includes moving objects. It may contain objects.
[0049] The motion information D4 shown in FIG. 8 is video information that records a video including the AR object O. The motion information D4 may be a user motion data for a predetermined period based on the AR marker. It may also include camera operation information including the position and angle of the terminal (camera). In this embodiment, the information D4 is stored in a computer such as a smartphone or a game terminal. It can be used as a material for realizing image processing of characters in games executed on More specifically, when generating images including characters in a game, Camera operation information can be used as operation information of the virtual camera (viewpoint).
[0050] In this embodiment, the receiving unit 30b and the generating unit 30c receive the program stored in the memory 32. This is realized by executing the RAM on the processor 31.
[0051] In this embodiment, the receiving unit 30b receives the second location information D2 transmitted from the user terminal 20. Execute the reception process to receive the message.
[0052] In this embodiment, the generation unit 30c calculates the second position information in the second position information calculation process of the user terminal 20. Generate motion information D4 of an arbitrary AR object O based on the second position information D2. The process to generate the file is executed (generation process).
[0053] In this embodiment, this generation process is performed by generating an AR object associated with the AR marker M. The recorded data is written out by associating the information D3 with the second location information D2 for a predetermined period. By converting the data format as necessary, the motion information of the AR object O is Execute the process to generate D4.
[0054] Next, an overview of the processing of the information processing system 10 according to the present embodiment will be described.
[0055] FIG. 10 is a flow chart for explaining an outline of the processing of the information processing system 10 according to the present embodiment. As shown in the figure, first, in step S1, a user 1 accesses a user terminal 20. When the application is started and the AR marker M is recognized via the camera 22, the user Calculating first position information D1 in a space in which the AR marker M is placed, with the terminal 20 as a reference (first position information calculation process).
[0056] In this embodiment, the first position information D1 is a position information of the space (real space) in which the AR marker M is placed. More specifically, the user terminal includes position information including coordinates and angles in the vicinity of the target location. The three-dimensional coordinates of the AR marker M on the user terminal coordinate system with a specific point on 20 as the origin. Includes information on the origin (angle).
[0057] In the next step S2, the first position information D1 calculated in the first position information calculation process and the AR Based on the object information D3, an AR object O (AR Generate image data of the character and display it on the display 23 of the user terminal 20 ( The display 23 displays an AR object O in a real space including an AR marker M. An AR image with superimposed will be displayed.
[0058] On the other hand, in step S3, the drawing information of the AR object O drawn in the drawing process is recorded. Calculate second position information D2 of the user terminal 20 in space based on the stored AR marker M. (second position information calculation process).
[0059] In this embodiment, the second position information calculation unit 20c calculates the position information based on the AR marker M. The second position information calculation includes the relative coordinates and the relative angle of the user terminal 20. The unit 20c is configured to calculate the user end on the AR marker coordinate system having the center point of the AR marker as the origin, for example. Includes information on the last 20 three-dimensional coordinates and three-dimensional direction (angle).
[0060] After calculating the second position information D2, the user terminal 20 calculates the calculated second position information D2 in step S4. The second location information D2 is transmitted to the server 30 (transmission process).
[0061] The server 30 receives the second location information D2 transmitted from the user terminal 20 (reception process ), and in step S5, the received second position information D2 and AR object information D3 In step S6, the received data (stored data) is stored in association with the received data. ) based on the motion information D4 about the AR object O (generation process) .
[0062] In a state where the AR object O is displayed on the user terminal 20 by the display process, the user The AR marker M is recognized by the user terminal 20 for an arbitrary predetermined period of time. The server 30 repeatedly executes the second position information calculation process for a predetermined period of time. The second position information is calculated based on the AR marker M (in the AR marker coordinate system). The user can generate motion information regarding the movement of the terminal 20. In this embodiment, the user terminal 20 is moved closer to or farther away from the AR marker, i.e., the AR object. The relative distance may be changed by moving the AR marker. Move the device 20 to capture the AR marker, i.e., the AR object, from various angles. According to this, the position and angle of the user terminal relative to the AR marker can be changed. The motion information that changes the display size and angle of the AR object is generated accordingly. will be done.
[0063] The predetermined period is the period between a start operation and an end operation received from a user who uses the user terminal 20. The start operation may be, for example, a period during which the AR object O When the image capturing start button is displayed on the user terminal 20, the user presses the image capturing start button displayed on the user terminal 20. The end operation can be an operation of selecting the icon. The shooting end icon displayed on the camera can be selected. During this time, the user continues to select (long press) the shooting icon displayed on the user terminal 20. In that case, select the shooting icon (touch panel) to start The above-mentioned start operation is the operation of touching the camera icon, and the operation of releasing the finger from the camera icon is This is the ending operation.
[0064] The generated motion information D4 is used for video expression such as games and animation movies. It can be used as a material to realize image processing of characters in.
[0065] In this way, the first time t1 is calculated as a result of the user terminal 20 recognizing the AR marker M. 2. Based on the position information D2, motion information D4 of the AR object O can be generated. This means that it is possible to operate the machine without having to secure special facilities or introduce special equipment. This makes it possible to easily generate the application information D4.
[0066] The present invention is not limited to the above-described embodiment, and may be modified without departing from the spirit and scope of the invention. Various modifications are possible.
[0067] In the above embodiment, the programs stored in the user terminal 20 and the server 30 are AR The case where information processing regarding objects is executed has been described. 0 and execute information processing related to AR objects on the user terminal 20 without distributing processing In the above description, the information processing executed by the user terminal 20 may be configured to At least a part of the processing may be executed by the server 30, or the information processing executed by the server 30 may be executed by the server 30. A part of the process may be executed on the user terminal 20.
[0068] In this embodiment, a sensor capable of detecting the inclination of the user terminal 20 (sensor 2 4) Calculate the tilt information of the AR marker based on the sensor information obtained from In this case, the motion information may be generated in the generation process. When the AR object is captured, the tilt of at least one of the AR object and the virtual camera is calculated based on the tilt information. In this way, the AR marker is horizontally aligned. Even if it is placed on a diagonal surface that is inclined to the horizontal plane, it will have the same motion as if it were placed on a horizontal plane. As a result, the AR object displayed on the AR marker can be generated. This can prevent the object from being displayed tilted. When the user terminal 20 is tilted in the horizontal direction (or vertical direction) and the Based on the information on the relative angle of the AR marker, the horizontal (or vertical) ) can be calculated.
[0069] In the present embodiment, in the generation process, if the user terminal 20 The behavior of the terminal device during the period when the R marker cannot be recognized is compared with the period when the terminal device recognizes the identification information. For example, a supplementary process may be performed based on information about a predetermined period of time. The position of the terminal device immediately before the period in which the AR marker cannot be recognized and the The AR marker is placed by connecting the position of the terminal device immediately after the period when the AR marker is not in use with ... It is possible to complement the motion information of the period that cannot be recognized. Even if there is a period during which the AR marker cannot be recognized temporarily, motion information can be generated. can be done.
[0070] The present invention has the following configuration. (Item 1) The identification information associated with information for displaying a virtual object on a terminal device is The terminal device recognizes the coordinates and angle of the identification information based on the terminal device. a first position information calculation process for calculating first position information including the first position information; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; An information processing system that executes the above. (Item 2) The second position information calculation process includes: The first position information calculated in the first position information calculation process is subjected to coordinate conversion processing. and calculating the second position information. Item 1. An information processing system according to item 1. (Item 3) The predetermined period is the period between a start operation and an end operation received from a user of the terminal device. The period between As the virtual object associated with the drawing information stored in the identification information based on AR object information related to the AR object and the second position information Generate motion information; 3. The information processing system according to item 1 or 2. (Item 4) The identification information is An AR marker to which information of an AR object as the virtual object is associated. be, 3. The information processing system according to item 1 or 2. (Item 5) The identification is performed based on sensor information acquired from a sensor capable of detecting the inclination of the terminal device. A gradient information calculation process for calculating gradient information of the information; When generating the motion information in the generation process, the tilt information is used to generate the motion information. and executing a tilt correction process to correct the tilt of the image. 3. The information processing system according to item 1 or 2. (Item 6) In the generation process, if the terminal device cannot recognize the identification information within the predetermined period, The movement of the terminal device during a period in which the terminal device recognizes the identification information is Execute a completion process to complete based on 3. The information processing system according to item 1 or 2. (Item 7) The identification information associated with information for displaying a virtual object on a terminal device is The terminal device recognizes the coordinates and angle of the identification information based on the terminal device. a first position information calculation process for calculating first position information including the first position information; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; An information processing device comprising a control unit that executes the above. (Item 8) The identification information associated with information for displaying a virtual object on a terminal device is The terminal device recognizes the coordinates and angle of the identification information based on the terminal device. a first position information calculation process for calculating first position information including the first position information; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; A program that causes a computer to execute the following. (Item 9) The identification information associated with information for displaying a virtual object on a terminal device is The terminal device recognizes the coordinates and angle of the identification information based on the terminal device. a first position information calculation process for calculating first position information including the first position information; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; An information processing method implemented by a computer. [Explanation of symbols]
[0071] 1 User 2 Business operators 10. Information Processing Systems 20 User terminal (terminal device) 30 Server (information processing device)
Claims
1. The identification information associated with information for displaying a virtual object on a terminal device is The terminal device recognizes the coordinates and angle of the identification information based on the terminal device. a first position information calculation process for calculating first position information including the first position information; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; An information processing system that executes the above.
2. The second position information calculation process includes: The first position information calculated in the first position information calculation process is subjected to coordinate conversion processing. and calculating the second position information. The information processing system according to claim 1 .
3. The predetermined period is the period between a start operation and an end operation received from a user of the terminal device. The period between As the virtual object associated with the drawing information stored in the identification information based on the AR object information and the second position information Generate motion information; 3. The information processing system according to claim 1 or 2.
4. The identification information is An AR marker to which information of an AR object as the virtual object is associated. be, 3. The information processing system according to claim 1 or 2.
5. The identification is performed based on sensor information acquired from a sensor capable of detecting the inclination of the terminal device. A gradient information calculation process for calculating gradient information of the information; When generating the motion information in the generation process, the tilt information is used to generate the motion information. and executing a tilt correction process to correct the tilt of the image.
3. The information processing system according to claim 1 or 2.
6. In the generation process, if the terminal device cannot recognize the identification information within the predetermined period, The movement of the terminal device during a period in which the terminal device recognizes the identification information is Execute a completion process to complete based on 3. The information processing system according to claim 1 or 2.
7. The identification information associated with information for displaying a virtual object on a terminal device is The terminal device recognizes the coordinates and angle of the identification information based on the terminal device. a first position information calculation process for calculating first position information including the first position information; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; An information processing device comprising a control unit that executes the above.
8. The identification information associated with information for displaying a virtual object on a terminal device is The terminal device recognizes the coordinates and angle of the identification information based on the terminal device. a first position information calculation process for calculating first position information including the first position information; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; A program that causes a computer to execute the following.
9. The identification information associated with information for displaying a virtual object on a terminal device is The terminal device recognizes the coordinates and angle of the identification information based on the terminal device. a first position information calculation process for calculating first position information including the first position information; The first position information is calculated based on the first position information calculated in the first position information calculation process. A display process of displaying the virtual object on the terminal device; The virtual object displayed in the display process is displayed based on the associated identification information. a second position information calculation process for calculating second position information including coordinates and an angle of the terminal device; The identification information is displayed on the terminal device for a predetermined period of time while the virtual object is displayed. Based on the second location information during the predetermined period obtained by having the user recognize information generating motion information relating to the virtual object; An information processing method implemented by a computer.
Citation Information
Patent Citations
Image processing apparatus, terminal device, image processing method, and program
JP2014071850A
Mixed reality system, program, method, and mobile terminal device
JP2019133507A
Program, electronic apparatus, and data recording method
JP2020119334A
Display program, display device, and display method
JP2020095645A
