Object display program, object display system, and object display method
By offsetting virtual objects in AR displays from real space objects in a predetermined axial direction, the method addresses flickering issues, providing stable and realistic AR display.
Patent Information
- Application Number
- JP2023076543
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-08
- Publication Date
- 2024-11-20
- Estimated Expiration
- Not applicable · inactive patent
Smart Images

Figure 2024161675000001_ABST
Abstract
Description
[Technical field]
[0001] At least one embodiment of the present invention relates to an object display program, an object display system, and an object display method. [Background technology]
[0002] There is an AR display system that displays content superimposed on real space. For example, Patent Document 1 describes an AR additional information content display system. In the AR additional information content display system, a spatial point, which is an object indicating a predetermined coordinate point in a three-dimensional space, is displayed on a display screen of a terminal device on which an AR marker is displayed. The AR additional information content display system includes a spatial point generation unit that generates a group of spatial points based on the AR marker on the display screen, and a spatial point and two-dimensional information link unit that links a designated spatial point with two-dimensional information displayed in two dimensions on the display screen and makes the spatial point linked to the two-dimensional information have an appearance different from that of a spatial point not linked to the two-dimensional information. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2021-144407 Summary of the Invention [Problem to be solved by the invention]
[0004] There is an occlusion function when displaying AR. Occlusion refers to a state in which an object in front hides an object behind it. By using the occlusion function, the display system can grasp the front-to-back relationship of objects, making it possible to display AR in a state in which an object in front hides an object behind it, resulting in a more realistic AR display.
[0005] However, when an AR display system places a virtual object so that it comes into contact with a surface such as a detection surface detected from real space, flickering due to the occlusion function can occur. This is because the coordinates of an object detected from real space and the coordinates of a virtual object match, and when there is a slight error or shaking in the measurement, the front-back relationship between the surface such as a detection surface detected from real space and the virtual object frequently changes. There was a latent need to suppress this kind of flickering due to the occlusion function in as simple a way as possible.
[0006] An object of at least one embodiment of the present invention is to solve the above problem and suppress flickering caused by the occlusion function in a simple manner. [Means for solving the problem]
[0007] From a non-limiting perspective, an object display program according to one embodiment of the present invention causes a computer to realize an offset function that offsets a virtual object from an object in real space in a predetermined axial direction in accordance with coordinates of a viewpoint relative to a virtual object, the coordinates being based on the virtual object, and a display function that displays the offset virtual object superimposed on real space.
[0008] From a non-limiting perspective, an object display system according to one embodiment of the present invention is an object display system equipped with at least a user terminal, and includes: an offset means for offsetting a virtual object from an object in real space in a predetermined axial direction in accordance with a coordinate in the predetermined axial direction of a viewpoint based on a virtual object; and a display means for displaying the offset virtual object superimposed on real space.
[0009] From a non-limiting perspective, an object display method according to one embodiment of the present invention is a method for displaying objects by a computer, and includes an offset process for offsetting a virtual object from an object in real space in a predetermined axial direction in accordance with a coordinate of a viewpoint in the predetermined axial direction, the coordinate being based on a virtual object, and a display process for displaying the offset virtual object superimposed on real space. Effect of the Invention
[0010] Each embodiment of the present application addresses one or more of the deficiencies. [Brief description of the drawings]
[0011] [Figure 1] 1 is a block diagram showing an example of a configuration of an object display system corresponding to at least one of the embodiments of the present invention. [Diagram 2] FIG. 2 is a block diagram showing a configuration of a server corresponding to at least one of the embodiments of the present invention. [Diagram 3] 11 is a flowchart illustrating an example of processing of an object display program corresponding to at least one of the embodiments of the present invention. [Figure 4] FIG. 13 is a diagram showing a display example on a display device. [Diagram 5] 11 is a conceptual diagram illustrating an offset process performed by an offset unit. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, examples of embodiments of the present invention will be described with reference to the drawings. Note that the various components in the examples of the embodiments described below can be appropriately combined to the extent that no inconsistencies or the like arise. In addition, the contents described as an example of one embodiment may be omitted in other embodiments. In addition, the contents of operations and processes that are not related to the characteristic parts of each embodiment may be omitted. Furthermore, the order of various processes that make up the various flows and sequences described below is random to the extent that no inconsistencies or the like arise in the processing contents.
[0013] [Embodiment] An overview of an embodiment of the present invention will be described below. In the following, an object display program executed in a server, which is an example of a computer, will be described as an embodiment.
[0014] FIG. 1 is a block diagram showing an example of a configuration of an object display system 100 corresponding to at least one of the embodiments of the present invention. The object display system 100, which is an example of an object display system, includes a server 10 and a user terminal 20 used by a user. Each of the user terminals 20A, 20B, and 20C is an example of the user terminal 20. The configuration of the object display system 100 is not limited to this. For example, the object display system 100 may be configured such that a single user terminal is used by multiple users. The object display system 100 may include multiple servers.
[0015] The server 10 and the user terminal 20 are examples of computers. The server 10 and the user terminal 20 are each communicatively connected to a communication network 30 such as the Internet. The connection between the communication network 30 and the server 10 and the connection between the communication network 30 and the user terminal 20 may be a wired connection or a wireless connection. For example, the user terminal 20 may be connected to the communication network 30 by performing data communication with a base station managed by a communication carrier via a wireless communication line.
[0016] The object display system 100 includes a server 10 and a user terminal 20, and thereby realizes various functions for executing various processes in response to user operations.
[0017] The object display system 100 may also function as a game processing system. The game processed by the game processing system may be a video game, and may be a game that utilizes real space, with or without XR technology. When the object display system 100 also functions as a game processing system, the server 10 controls the progress of the game. The server 10 may control the progress of the game according to the operation of the player. The server 10 is managed by an administrator of the object display system 100, and has various functions for providing information on various processes to multiple user terminals 20. The server 10 may be configured by an information processing device such as a game server capable of rendering game images. The object display system and the game processing system may be configured by multiple servers.
[0018] The server 10 includes a processor 11, a memory 12, and a storage device 13. The processor 11 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and control. In addition, when the server 10 includes a GPU (Graphics Processing Unit), some of the various calculations and control may be performed by the GPU. The server 10 executes various information processing by the processor 11 using data read into the memory 12, and stores the obtained processing results in the storage device 13 as necessary.
[0019] The storage device 13 has a function as a storage medium for storing various information. The configuration of the storage device 13 is not particularly limited, but from the viewpoint of reducing the processing load on the user terminal 20, it is preferable that the storage device 13 is configured to be capable of storing all of the various information required for the control performed by the object display system 100. Examples of such a configuration include an HDD and an SSD. However, the storage device that stores the various information only needs to have a storage area accessible by the server 10, and may be configured to have a dedicated storage area outside the server 10, for example.
[0020] The user terminal 20 is equipped with hardware (e.g., a display device that displays a browser screen or a game screen according to coordinates) and software for connecting to the communication network 30 and executing various processes by communicating with the server 10. Note that each of the multiple user terminals 20 may be configured to be able to communicate directly with each other without going through the server 10.
[0021] The user terminal 20 may be configured as a communication terminal managed by a user and capable of playing a network distribution type game. Examples of communication terminals include mobile phone terminals, PDAs (Personal Digital Assistants), portable game devices, VR goggles, AR glasses, smart glasses, AR contacts, so-called wearable devices, etc. The configuration of the user terminal that may be included in the object display system 100 is not limited to these. Other examples of the configuration of the user terminal include a combination of various communication terminals, a personal computer, and a stationary game device.
[0022] A display device may be built into the user terminal 20. Also, the display device may be connected wirelessly or by wire to the user terminal 20. Note that the display device is a very common configuration, and is therefore not shown here.
[0023] When a user terminal is used for gaming purposes, the game screen is displayed by a display device as a composite image, for example, and the user recognizes this composite image. The game screen is displayed, for example, on a display, which is an example of a display device provided in the user terminal, or on a display, which is an example of a display device connected to the user terminal. Display devices include, for example, a holographic display device capable of holographic display, a projection device that projects an image (including the game screen) onto a screen, and a device that performs XR display. XR includes VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality), etc.
[0024] The user terminal 20 includes a processor 21, a memory 22, and a storage device 23. The processor 21 is, for example, a central processing unit such as a CPU (Central Processing Unit) that performs various calculations and control. In addition, if the user terminal 20 includes a GPU (Graphics Processing Unit), some of the various calculations and control may be performed by the GPU. The user terminal 20 executes various information processing in the processor 21 using data read into the memory 22, and stores the obtained processing results in the storage device 23 as necessary. The storage device 23 functions as a storage medium that stores various information.
[0025] The user terminal 20 may have a built-in input device. The input device may be connected to the user terminal 20 wirelessly or by wire. The input device accepts operational input by a user. In response to the operational input by the user, a processor provided in the server 10 or a processor provided in the user terminal 20 executes various control processes. Examples of the input device include a touch panel screen provided in a mobile phone terminal, a keyboard, a mouse, a game pad, a joystick, and other controllers. A camera built into or connected to the user terminal 20 may also correspond to the input device. The user performs operational input by making a gesture such as moving a hand in front of the camera (gesture input).
[0026] Additionally, the user terminal 20 may include other output devices such as a speaker, etc. The other output devices output various types of information such as sound, vibration, and the like to the user.
[0027] 2 is a block diagram showing a configuration of a server corresponding to at least one of the embodiments of the present invention. Server 10A, which is an example of the configuration of server 10, includes at least offset unit 101 and display unit 102. A processor included in server 10A refers to an object display program stored in a storage device and executes the program to functionally realize offset unit 101 and display unit 102.
[0028] The offset unit 101 has a function of offsetting a virtual object from an object in real space in an axial direction according to a coordinate in a predetermined axial direction for a viewpoint, with the virtual object as a reference.
[0029] The display unit 102 has a function of displaying the offset virtual object superimposed on the real space.
[0030] Next, a program execution process in the embodiment of the present invention will be described below. Fig. 3 is a flowchart showing an example of a process of an object display program corresponding to at least one of the embodiments of the present invention.
[0031] The offset unit 101 offsets the virtual object from the object in real space in an axial direction according to a coordinate in a predetermined axial direction of the viewpoint with respect to the virtual object as a reference (St11). The display unit 102 displays the offset virtual object superimposed on the real space (St12).
[0032] A virtual object is a virtual object that is to be displayed overlaid on a real space. Examples of virtual objects include a player character, an enemy character, an item such as a sword, and a vehicle such as a tank. Virtual objects are not limited to these.
[0033] The viewpoint refers to the position of a virtual camera or the user's viewpoint acquired by a user terminal such as VR goggles, AR glasses, smart glasses, or AR contacts.
[0034] The predetermined axis direction may typically be the upward direction in the predetermined space. For example, the Y axis direction in a Y-up coordinate space, the Z axis direction in a Z-up coordinate space, etc., are the predetermined axis direction. However, a direction other than the upward direction may be the predetermined axis direction.
[0035] The coordinates of a given axis direction of the viewpoint with respect to a virtual object as a reference means the coordinates of the viewpoint when a given position of the virtual object is assumed to have a given coordinate value (for example, zero coordinate, which is the origin). For example, when the virtual object is a humanoid figure, the coordinates of the figure's feet are assumed to have a coordinate value of 0. The coordinates of the viewpoint in this case correspond to the coordinates of the viewpoint with respect to the virtual object as a reference here. The given position of the object may be appropriately determined by the system designer or the like.
[0036] An object in real space means an object in real space such as a desk, a chair, or a floor.
[0037] To offset in the axial direction means to shift the display position in the axial direction.
[0038] [Examples of offsets] FIG. 4 is a diagram showing a display example by a display device. In this example, a smartphone is used as the user terminal and display device. Based on an image captured by the built-in camera of the user terminal, the object display system 100 detects a shelf and a mug as objects in real space. The object display system 100 recognizes the horizontal plane of the shelf as a detection plane PL, and displays a virtual object OBJ on the detection plane PL in AR. The AR display of a virtual object refers to displaying the virtual object superimposed on the real space.
[0039] In this example, the virtual object OBJ is a figure. When the occlusion function is used, flickering may occur at the feet of the figure. This is because the figure's feet are in contact with the detection surface PL of the object in real space, and so which of the figure's feet or the detection surface PL is in front frequently switches.
[0040] FIG. 5 is a conceptual diagram illustrating an offset process performed by the offset unit.
[0041] The object display system 100 detects an object in real space, and AR-displays a virtual object OBJ on the detection surface of the object in real space.
[0042] 5, reference symbol V denotes a viewpoint of a virtual camera, etc. As the viewpoint positions, P1 and P2 are illustrated.
[0043] 5 illustrates an example of a Y-up coordinate space. In this case, coordinates C1 and C2 are coordinates in the Y-axis direction with respect to the viewpoint V, based on the foot of the virtual object OBJ, that is, the part that contacts the detection plane PL.
[0044] In the object display system 100 of the present disclosure, the offset unit 101 offsets the virtual object OBJ in an axial direction from the object in real space according to the coordinate of a predetermined axial direction (Y-axis direction in FIG. 5) of the viewpoint based on the virtual object OBJ.
[0045] For example, when the viewpoint V is at a position P1, the virtual object OBJ is viewed from above, and flickering resulting from the occlusion function is likely to occur. To prevent this flickering, the offset unit 101 offsets the virtual object OBJ in the axial direction. In other words, the offset unit 101 slightly lifts the virtual object OBJ from the detection plane PL. The offset amount may be, for example, about 1.5 cm when converted into a length in real space. However, the offset amount may be a value other than 1.5 cm, and may be set appropriately depending on the environment.
[0046] When the viewpoint V is at the position of P2, the virtual object OBJ is viewed from the side, so flickering caused by the occlusion function hardly occurs. Therefore, the offset unit 101 does not offset the virtual object OBJ in the axial direction. In other words, the offset unit 101 grounds the virtual object OBJ so that it does not float above the detection plane PL. Note that, as an internal process, the offset process may be performed with the offset amount=0.
[0047] [Example of offset processing] The offset amount may be processed as a binary value of 0 or a predetermined value. The offset amount may be processed so as to take more than two values.
[0048] For example, the offset unit 101 may reduce the offset amount as the absolute value of the coordinate in the predetermined axis direction for the viewpoint is smaller. In other words, when the predetermined axis direction is the upward direction, the offset amount may be increased as the viewpoint moves to a position where the virtual object OBJ is viewed from further above, and the offset amount may be reduced as the viewpoint moves to a position where the virtual object OBJ is viewed from further to the side. More specifically, the offset unit 101 may set the offset amount to zero when the absolute value of the coordinate in the predetermined axis direction for the viewpoint is smaller than a predetermined value. This makes it possible to prevent the object OBJ from appearing to float above the detection plane PL.
[0049] The offset unit 101 may set the offset amount in stages based on a threshold value of 1 or more, for example, if the coordinate value is b or more, the offset amount = Y1, if the coordinate value is a or more and less than b, the offset amount = Y2, if the axis coordinate value is less than a, the offset amount = 0.
[0050] Further, the offset unit 101 may set the offset amount by a function that receives a value based on the coordinate value as input and outputs a value from which the offset amount can be derived.
[0051] An upper limit may be set for the offset amount.
[0052] According to one aspect of the embodiment, flickering caused by the occlusion function can be suppressed in a simple manner.
[0053] As one aspect of the embodiment, it is possible to prevent flickering and display a virtual object so that the virtual object does not appear to float above the detection surface of the object in real space.
[0054] According to one aspect of the embodiment, when a virtual object is placed on an object in real space, flickering caused by an occlusion function can be suppressed in a simple manner.
[0055] As described above, each embodiment of the present application solves one or more deficiencies. Note that the effects of each embodiment are non-limiting effects or examples of effects.
[0056] In each of the above-mentioned embodiments, the user terminal 20 and the server 10 execute the above-mentioned various processes according to various control programs (e.g., object display program) stored in a storage device provided in the user terminal 20 and the server 10. Also, other computers, not limited to the user terminal 20 and the server 10, may execute the above-mentioned various processes according to various control programs (e.g., object display program) stored in a storage device provided in the computer.
[0057] The configuration of the object display system 100 is not limited to the configuration described as an example of each of the above-mentioned embodiments. For example, the server may execute part or all of the processes described as processes executed by the user terminal, or the user terminal may execute part or all of the processes described as processes executed by the server. The user terminal may also be equipped with part or all of the storage unit (storage device) equipped in the server. In other words, the user terminal or the server in the object display system 100 may be equipped with part or all of the functions equipped in the other one.
[0058] Furthermore, the program may be configured to realize some or all of the functions described as examples of each of the above-mentioned embodiments in a standalone device that does not include a communication network.
[0059] [Note] The above-mentioned embodiments have been described so as to enable at least one person having ordinary skill in the art to which the invention pertains to practice the invention. [1] On the computer, an offset function that offsets the virtual object from an object in real space in a predetermined axial direction in accordance with a coordinate of a viewpoint in the predetermined axial direction, the coordinate being based on the virtual object; a display function of superimposing and displaying the offset virtual object on a real space; An object display program that realizes the above. [2] In the offset function, the smaller the absolute value of the coordinate, the smaller the offset amount. [1] An object display program according to the present invention. [3] In the offset function, if the absolute value is smaller than a predetermined value, the offset amount is set to zero. [2] An object display program according to the present invention. [4] The predetermined axial direction is an upward direction of the predetermined space. An object display program according to any one of [1] to [3]. [5] An object display system comprising at least a user terminal, an offset means for offsetting the virtual object from an object in real space in a predetermined axial direction in accordance with a coordinate of a viewpoint in the predetermined axial direction, the coordinate being based on the virtual object; a display means for displaying the offset virtual object in a superimposed state on a real space, Object display system. [6] A method for displaying objects by a computer, comprising the steps of: an offset process for offsetting the virtual object from an object in real space in a predetermined axial direction in accordance with a coordinate of a viewpoint in the predetermined axial direction, the coordinate being based on the virtual object; A display process of displaying the offset virtual object in a real space by superimposing the offset virtual object on the real space. How objects are displayed. [Industrial Applicability]
[0060] According to one embodiment of the present invention, an object display program and an object display system are useful for suppressing flickering caused by an occlusion function in a simple manner. [Explanation of symbols]
[0061] 10, 10A Server 11 Processors 12. Memory 13 Storage device 20, 20A, 20B User terminal 21 Processors 22 Memory 23 Storage device 30 Communication Network 100 Object Display System 101 Offset section 102 Display section
Claims
1. On the computer, an offset function that offsets the virtual object from an object in real space in a predetermined axial direction in accordance with a coordinate of a viewpoint in the predetermined axial direction, the coordinate being based on the virtual object; a display function of superimposing and displaying the offset virtual object on a real space; An object display program that realizes the above.
2. In the offset function, the smaller the absolute value of the coordinate, the smaller the offset amount.
2. The object display program according to claim 1.
3. In the offset function, if the absolute value is smaller than a predetermined value, the offset amount is set to zero.
3. The object display program according to claim 2.
4. The predetermined axial direction is an upward direction of the predetermined space.
4. The object display program according to claim 1.
5. An object display system comprising at least a user terminal, an offset means for offsetting the virtual object from an object in real space in a predetermined axial direction in accordance with a coordinate of a viewpoint in the predetermined axial direction, the coordinate being based on the virtual object; a display means for displaying the offset virtual object in a superimposed state on a real space, Object display system.
6. A method for displaying objects by a computer, comprising the steps of: an offset process for offsetting the virtual object from an object in real space in a predetermined axial direction in accordance with a coordinate of a viewpoint in the predetermined axial direction, the coordinate being based on the virtual object; A display process of displaying the offset virtual object in a real space by superimposing the offset virtual object on the real space. How objects are displayed.
Citation Information
Patent Citations
Ar additional information content display system
JP2021144407A
