Image processing apparatus, image processing method, and program

Abstract

To prevent a visual effect in a composite world from being impaired, while preventing an opportunity loss of crisis detection in a real world, when experiencing the composite world in which a virtual object is superimposed on the real world.SOLUTION: When a user is walking ahead, a virtual object is rearranged to a screen upper part, for raising the user's sight line upward for avoiding risk. When the user is moving ahead and downward such as descending stairs, the virtual object is rearranged to a screen lower part, for lowering the user's sight line downward for avoiding the risk.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, an image processing method, and a program.

Background Art

[0002] In recent years, technologies for providing a composite world in an environment that combines the real world and virtual objects, such as augmented reality (AR) and mixed reality (MR), have been developing. For example, by wearing a head-mounted display (HMD) on the user's head, it is possible to provide the user with a composite world in which virtual objects are superimposed on a video image of the real world in front of the eyes. Furthermore, the user's actions can be detected by various sensors and synchronized with the actions in the composite world, providing the user with an unprecedented experience.

[0003] While the user's sense of immersion is improved by freely moving around in the composite world as if it were the real world, there is a possibility that the user may become less attentive to the real world, and in dangerous situations, it is important to reduce the degree of attention to virtual objects. As a technique for reducing the degree of attention to virtual objects according to the situation, Patent Document 1 discloses a technique for estimating whether the user is stopped or moving and changing the transparency of a superimposed image during movement.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] As described above, there is a case where a person becomes obsessed with a virtual object and fails to notice a danger in the real world, resulting in an accident. For example, because the person is looking at a virtual object displayed on a screen in a hunched posture, they may not notice the presence of a car coming from the front. In the technology described in Patent Document 1, even if the transparency of the virtual object is increased, there is a possibility that the person will continue to gaze at the virtual object, resulting in the loss of an opportunity to detect a crisis.

[0006] In view of the above problems, an object of the present invention is to prevent the loss of an opportunity to detect a crisis in the real world and to avoid degrading the visual effect in the composite world when experiencing a composite world in which a virtual object is superimposed on the real world.

Means for Solving the Problems

[0007] An image processing apparatus according to the present invention is an image processing apparatus that generates an image of a composite world by superimposing a virtual object on an image of the real world, and includes an acquisition unit that acquires information related to the operation status of a user of the image processing apparatus Indicating the direction of progress in the real world and a determination unit that determines a position for arranging the virtual object on the image of the real world based on the acquired operation status, and an image processing unit that generates an image in which the virtual object is arranged at the determined position. It is characterized by that. Recording Based on the obtained operation status, From the position required by the application that generated the virtual object, to the target position that directs the user's line of sight in the direction of progress based on the acquired motion state on the image of the real world The the virtual object Re is arranged Target for position is determined The de determined Target position, the virtual object Is is arranged Done at the Of the composite world image is generated. Obtained It is characterized by that.

Effects of the Invention

[0008] According to the present invention, when experiencing a composite world in which a virtual object is superimposed on the real world, it is possible to prevent the loss of an opportunity to detect a crisis in the real world and to avoid degrading the visual effect in the composite world.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

MODE FOR CARRYING OUT THE INVENTION

[0010] (First Embodiment) Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an example of an image displayed by the image processing apparatus according to the present embodiment. In FIG. 1, image 101 is an example of an image when the user is stopped. In image 101, virtual objects 103, 104, and 105 are superimposed on the coordinates required by the application to provide a composite world, respectively. On the other hand, image 102 is an example of an image when the wearer (user) of the HMD device is walking forward. When the user starts walking, virtual objects 103, 104, and 105 are each repositioned to predetermined coordinates.

[0011] At that time, the repositioning is performed over a plurality of drawing frames, and the virtual object is displayed with a visual effect so as to gradually move from the coordinates shown in image 101 to the coordinates shown in image 102 when the user is walking forward. In the present embodiment, when walking forward, for the purpose of raising the user's line of sight upward for danger avoidance, the virtual object is repositioned to the upper part of the screen.

[0012] In the present embodiment, when the virtual object starts to move for repositioning, the size of the virtual object, the display method for the overlapping state between virtual objects, the polygon model, and the drawing method for the shadow and lighting of the texture are changed. In the description of the present embodiment, the drawing method of each virtual object is changed as follows including FIG. 1 and later. As for the size of the virtual object, an example is shown in which virtual object 103 becomes half the size when it was being displayed, and virtual object 104 becomes the original size (twice) of virtual object 104. As for the display method for the overlapping state between virtual objects, an example is shown in which virtual object 105, virtual object 104, and virtual object 103 are in this order from the front. As for the polygon model, an example is shown in which virtual object 103 is of low quality, virtual object 104 is of high quality, and virtual object 105 is not changed. As for the drawing method for the texture shadow and lighting, an example is shown in which virtual objects 103 and 104 are omitted and applied to virtual object 105.

[0013] Also, the image 106 is an example of an image when the user is going down the stairs. In the present embodiment, when the user is moving forward and downward, such as going down the stairs, for the purpose of lowering the user's line of sight downward to avoid danger, the virtual object is relocated to the lower part of the screen. The change in the size of the virtual object and the display method for the overlapping state are the same as those in the image 102 when the user is walking forward. In the present embodiment, the frame rate displayed in the image changes when the user's situation is stopped, when walking forward, and when going down the stairs. It is set to high quality when stopped and low quality in other operating situations. For example, it is 120 fps (frames per second) when stopped and 30 fps in other operating situations.

[0014] The above settings are an example in the present embodiment, and different settings / initial values may be given. The coordinates for relocating the virtual object are not limited to those in the example shown in FIG. 1 and may be different from FIG. 1. Also, although the image 102 when the user is walking forward is shown, the user's action for relocating the virtual object may be triggered by other actions. For example, it is possible to change the coordinates to be relocated according to the type of movement in directions such as forward, backward, left, right, up, and down in the image.

[0015] FIG. 2 is a block diagram showing an example of the hardware configuration of the image processing apparatus 200 that outputs the image of FIG. 1. CPU 201 represents a processor, ROM 202 represents a read-only memory, RAM 203 represents a random access memory, and I / F 204 represents an external interface. The image processing apparatus 200 of this embodiment includes a CPU 201, a ROM 202, a RAM 203, and an I / F 204, which are connected by a bus 205. The operation of the image processing apparatus 200 is controlled by the CPU 201, and the processing of the flowchart described later is realized by the program loaded in the ROM 202 and the RAM 203. The RAM 203 is further used as a work memory for storing temporary data of the processing performed by the CPU 201, and also functions as an image buffer for temporarily holding the image data to be displayed. The I / F 204 is an interface for communicating with the outside, and inputs the image data of the real world and the data for judging the situation of the user, and outputs the image data to be displayed.

[0016] Although only one CPU is illustrated in FIG. 2, the image processing apparatus may be realized by a plurality of processors. Further, supplementary configurations such as a graphics processing unit (GPU) may be included. Furthermore, although only the RAM 203 is shown as a configuration for holding a temporary work memory, secondary and tertiary storage areas may be provided on the same or different media. Examples of other media include a hard disk drive (HDD) and a solid state drive (SSD). The configuration of the bus 205 is not limited to this, and the components may be connected in multiple stages. In order to realize this embodiment, the configuration of FIG. 2 is provided in the HMD. However, it is not limited to this, and a device connected by wire or wirelessly separately from the HMD may be provided with some or all of the components shown in FIG. 2.

[0017] FIG. 3 is a block diagram showing a functional configuration example of the image processing apparatus 200. In FIG. 3, the information input unit 301 receives situation data for determining the operation status of the user input from the I / F 204, and updates the operation status data 302. Here, the situation data is data obtained from various sensors and the like provided in the HMD, and includes information such as the current position, moving speed, and direction of the user. The rearrangement determination unit 303 determines whether or not to rearrange the virtual object using the operation status data 302, and when it is further determined that rearrangement is necessary, holds the information for rearrangement as rearrangement data 305. The image processing unit 304 creates an image for each frame using the rearrangement data 305 and outputs it to the display unit 306. When generating an image for each frame, the image processing unit 304 acquires real-world image data from the camera mounted on the HMD via the I / F 204 and superimposes a virtual object based on the rearrangement data 305. Thereby, the user can visually see the image example shown in FIG. 1.

[0018] FIG. 4 is a diagram showing an example of the operation status data 302. In the present embodiment, as described above, the information input unit 301 receives the situation data of the user via the I / F 204, updates and holds the operation status data 302 based on the situation data. As shown in FIG. 4, the operation status data 302 includes information on the operation status, speed, traveling direction, and head angle of the user.

[0019] The "operation status" indicates any one of information on being stopped, walking, and descending. Note that a part or all of the information such as in front of the screen, behind, left, right, ascending, and descending may be used. Alternatively, for the descending state, information may be obtained from the "traveling direction" described later, and the operation status may not hold any information. The "speed" indicates the moving speed of the user. The "traveling direction" indicates the vector of the traveling direction of the user. The "head angle" indicates the vector of the direction of the user's head. Also, in the example shown in FIG. 4, the operation status data 302 holds the information on "speed", but it may be held in another information form. For example, acceleration may be held as an item, or a concept of length may be added to the vector of the traveling direction to hold information such as the amount of movement per unit time.

[0020] FIG. 5 is a diagram showing an example of the rearrangement data 305. The rearrangement determination unit 303 holds the rearrangement data 305 for each of the virtual objects 103, 104, and 105.

[0021] In FIG. 5, the "coordinates before rearrangement" indicates the coordinates that the application requests as output in providing the composite world. For example, it indicates the coordinates of the virtual objects 103, 104, and 105 in the image 101 when the user is stopped in FIG. 1.

[0022] The "size at rearrangement" indicates the size of the virtual object at the time of rearrangement. For example, in the case of the virtual object 103 in FIG. 1, it is defined as 0.5 times. Note that it may be defined so that the aspect ratios in the vertical and horizontal directions are different when changing the size. The "polygon model at rearrangement" indicates the information of the polygon model of the virtual object to be drawn at the time of rearrangement. In this embodiment, two types of low quality and high quality are prepared in advance for each virtual object, and it is defined whether to change from before at the time of rearrangement and which polygon model to display if changing. Note that a polygon model may be added to this column and defined to include its value.

[0023] The "shadow at rearrangement" indicates the shadow information applied to the texture of the virtual object to be drawn at the time of rearrangement. In this embodiment, it is defined in the form of yes / no whether to perform the drawing of the texture using the shadow information. The "lighting at rearrangement" indicates the lighting information applied to the texture of the virtual object to be drawn at the time of rearrangement. In this embodiment, it is defined in the form of yes / no whether to perform the drawing of the texture using the lighting information. Note that the shadow at rearrangement and the lighting at rearrangement may be defined so as to define the image processing techniques applied to the texture, respectively.

[0024] "Superimposition priority" indicates information on the display method for the degree of overlap of virtual objects after rearrangement. In this embodiment, it is expressed in three levels: highest, high, and low, and the higher the level, the more forward it is displayed. Also, an integer may be used as the superimposition priority, or the priority may be changed. "Walking placement coordinates" indicate the rearrangement coordinates when walking forward, and are defined by the coordinates shown in, for example, image 102 when the user is walking forward in FIG. 1. "Descending placement coordinates" indicate the rearrangement coordinates when descending the stairs, and are defined by the coordinates shown in, for example, image 106 when the user is descending the stairs in FIG. 1.

[0025] "Rearrangement coordinates" indicate the rearrangement coordinates according to the current operation status of the user, and either the pre-rearrangement coordinates, the walking placement coordinates, or the descending placement coordinates are held. "Current coordinates" indicate the display coordinates in the frame during rearrangement. "Frame rate" indicates the quality of the frame rate for each operation status, and is defined as either high quality or low quality. Note that the frame rate may be defined by specific values (fps) for each operation status.

[0026] The types of values shown for the rearrangement size, rearrangement polygon model, rearrangement shadow, rearrangement lighting, superimposition priority, and frame rate in this embodiment are not limited to these. As the walking placement coordinates shown in this embodiment, one or more coordinate information may be held according to the speed threshold during walking. For example, it may be divided in the form of walking and running, or, for example, rearranged to different coordinates every 2 m / s.

[0027] FIG. 6 is a flowchart showing an example of the processing procedure performed by the information input unit 301. The processing in FIG. 6 starts when the user wears the HMD and starts experiencing the composite world, but the processing in FIG. 6 may also start at the timing of starting the screen display of the HMD. In step S601, the information input unit 301 determines whether to continue the experience in the composite world. As a result of this determination, if continuing the experience in the composite world, the process proceeds to step S602; if not, the process ends. In step S602, the information input unit 301 interprets the user's situation data obtained via I / F 204, and updates the operation status, speed, traveling direction, and head angle data of the operation status data 302.

[0028] In this embodiment, a method of always acquiring the user's situation data and updating the operation status data 302 is disclosed, but the timing of the update is not particularly limited. For example, it may be updated for each frame, or it may be updated when the user's situation data obtained via I / F 204 changes.

[0029] FIG. 7 is a flowchart showing an example of a processing procedure for updating the rearrangement data 305 by the rearrangement determination unit 303. This process starts when the user wears the HMD and starts the experience in the composite world, or at the timing of starting the screen display of the HMD. The process may be started for each frame output so that it can be used for the image processing of that frame. In step S701, the rearrangement determination unit 303 determines whether the information in the operation status data 302 has been updated. As a result of this determination, if it has been updated, the process proceeds to step S702; if not, the process proceeds to step S703.

[0030] In step S702, the rearrangement determination unit 303 updates the rearrangement coordinates of each virtual object in the rearrangement data 305 based on the update information of the operation status data 302. The specific processing will be described later with reference to FIG. 8. In step S703, the rearrangement determination unit 303 determines whether to continue the experience in the composite world. As a result of this determination, if continuing the experience in the composite world, the process returns to step S701; if not, the process ends.

[0031] FIG. 8 is a flowchart showing an example of a detailed processing procedure for updating the rearrangement coordinates by the rearrangement determination unit 303 in step S702. This flowchart shows the processing performed in step S702 of FIG. 7. In step S801, the rearrangement determination unit 303 determines whether the current operation status in the operation status data 302 is in a stopped state. As a result of this determination, if it is stopped, the process proceeds to step S802, and if not, the process proceeds to step S803.

[0032] In step S802, the rearrangement determination unit 303 updates the rearrangement coordinates in the rearrangement data 305 with the coordinates before rearrangement, and ends the process. In step S803, the rearrangement determination unit 303 determines whether the operation status in the operation status data 302 is decreasing. As a result of this determination, if it is decreasing, the process proceeds to step S804, and if not, the process proceeds to step S805.

[0033] In step S804, the rearrangement determination unit 303 updates the rearrangement coordinates in the rearrangement data 305 with the coordinates at the time of descent, and ends the process. In step S805, since it is shown that the operation status in the operation status data 302 is in the walking state, the rearrangement determination unit 303 updates the rearrangement coordinates in the rearrangement data 305 with the coordinates at the time of walking, and ends the process. Note that the processing order according to the types of operation status shown in this embodiment is not limited to this, and the processing order may be changed. Alternatively, the speed in the operation status data 302 may be referred to during walking, and the processing may be further switched according to the speed.

[0034] FIG. 9 is a flowchart showing an example of a processing procedure performed by the image processing unit 304. This processing is started when an image of the real world is input. In step S901, the image processing unit 304 determines whether the "current coordinates" in the rearrangement data 305 are the same as the rearrangement coordinates. As a result of this determination, if the coordinates are the same, the process proceeds to step S905, and if not, the process proceeds to step S902.

[0035] In step S902, the image processing unit 304 calculates the coordinates to be displayed for each virtual object in the next frame and updates the current coordinates in the rearrangement data 305. The new current coordinates are calculated by calculating the coordinates when moving from the previous current coordinates to the target rearrangement coordinates at a certain speed.

[0036] In step S903, the image processing unit 304 superimposes the virtual object on the real-world image based on the information in the rearrangement data 305, creates the image data to be displayed on the display unit 306, and stores it in the image buffer. Here, as the image processing information, the polygon model, shadow, and lighting information at the time of rearrangement in the rearrangement data 305 are referred to. Also, when "unchanged" is not specified for the size at the time of rearrangement in the rearrangement data 305, the magnification is calculated when changing from the previous size to the size at the target rearrangement coordinates at a certain speed, and the virtual object is enlarged or reduced and adjusted. In step S904, the image processing unit 304 waits for the image data stored in the image buffer in step S903 to be displayed on the display unit 306, and returns to step S901.

[0037] On the other hand, in step S905, the image processing unit 304 superimposes the virtual object on the real-world image based on the information in the rearrangement data 305, creates the image data to be displayed on the display unit 306, and stores it in the image buffer. In this case, the position where the virtual object is displayed is the position of the rearrangement coordinates, and the image data is generated according to the size, polygon model, etc. defined in the rearrangement data 305.

[0038] In step S906, the image processing unit 304 waits for the image data stored in the image buffer in step S905 to be displayed on the display unit 306. In step S907, the image processing unit 304 determines whether there is a frame to be displayed next. As a result of this determination, if there is a frame to be displayed next, it returns to step S901, and if not, the process ends.

[0039] According to the present embodiment as described above, by relocating virtual objects while accompanying a visual effect, it becomes possible to provide a technology that prevents the loss of opportunity for crisis detection in the real world and does not impair the user experience.

[0040] (Second Embodiment) In this embodiment, an example will be described in which, when relocating virtual objects, instead of holding coordinate information for each virtual object, information on the image area where the virtual objects are relocated is held. Since the internal configuration of the image processing apparatus according to this embodiment is the same as that in FIGS. 2 and 3, the description thereof will be omitted. Hereinafter, differences from the first embodiment will be described.

[0041] FIG. 10 is a diagram showing an example of an image displayed in this embodiment. Image 1001 is an example of an image when the user is stopped. On the other hand, image 1002 is an example of an image when the user is walking forward. Similar to the first embodiment, when the user starts walking, virtual objects 103, 104, and 105 are relocated to the indicated coordinates respectively. However, in this embodiment, when the user starts walking, each virtual object is mapped to the area 1003 in the upper 1 / 3 of the screen. Also, image 1004 is an example of an image when the user is running forward. In this embodiment, when the user starts running, each virtual object is mapped to the area 1005 in the upper 1 / 3 and the right 1 / 3 of the screen.

[0042] Also, although not shown in FIG. 10, when the user is moving forward and downward, such as going down stairs, the image areas for relocating the virtual objects are made different depending on whether the user is walking or running. Specifically, when the user starts descending at a walking speed, each virtual object is mapped to the area in the lower 1 / 3 of the screen, and when the user starts descending at a brisk pace, each virtual object is mapped to the area in the lower 1 / 3 and the right 1 / 3 of the screen.

[0043] FIG. 11 is a diagram showing an example of the rearrangement data 305 in the present embodiment. In the present embodiment, compared with the example of FIG. 5, an arrangement area during walking and running is added, and the arrangement coordinates during walking and the arrangement coordinates during descent are deleted. The "arrangement area during walking" and the "arrangement area during descent" hold information on the mapping area for calculating the rearrangement coordinates of the virtual object during walking and running. Note that m represents the coordinate size of the horizontal width with the left end being 0, and n represents the coordinate size of the vertical height with the upper end being 0. Further, the "rearrangement coordinates" hold the mapping coordinates calculated by the processing of the rearrangement determination unit 303 described later.

[0044] FIG. 12 is a flowchart showing an example of a processing procedure for calculating the rearrangement coordinates of a virtual object in the present embodiment. The processing shown in FIG. 12 is performed instead of the processes in steps S804 and S805 of FIG. 8.

[0045] In step S1201, the rearrangement determination unit 303 determines the image area of the mapping destination from the speed of the operation status data 302, and further acquires information on the walking arrangement area or the descent arrangement area from the rearrangement data 305. In this process, first, it is determined whether the user is walking or running from the "speed" in the operation status data 302, and if the speed is equal to or higher than the threshold, it is determined that the user is running. Then, information on the "walking arrangement area" or the "descent arrangement area" during walking or running is acquired. And the rearrangement coordinates to be mapped are calculated within the corresponding arrangement area. Note that there is no particular limitation on which position in the walking arrangement area or the descent arrangement area is used as the rearrangement coordinates, and the rearrangement coordinates may be arbitrarily determined within the corresponding area.

[0046] As described above, according to the present embodiment, it is possible to provide a technique for holding image area information related to the range for rearranging a virtual object without previously holding coordinate information for each virtual object when rearranging the virtual object.

[0047] (Other Embodiments) The present invention can also be realized by supplying a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be realized by a circuit (for example, ASIC) that implements one or more functions.

Description of Reference Numerals

[0048] 301 Information input unit, 303 Rearrangement determination unit, 304 Image processing unit

Claims

Claim 1 An image processing apparatus for generating an image of a composite world by superimposing a virtual object on an image of the real world, comprising: acquisition means for acquiring information related to an operation state indicating a traveling direction of a user of the image processing apparatus in the real world; determination means for determining a target position for repositioning the virtual object in the real-world image from a position required by an application that generated the virtual object based on the acquired operation state to a target position that directs the user's line of sight in the traveling direction based on the acquired operation state; image processing means for generating an image of a composite world in which the virtual object is arranged at the determined target position; An image processing apparatus, characterized by comprising the above. Claim 2. The image processing apparatus according to claim 1, wherein the image processing means generates an image of a composite world in which the virtual object is arranged at the determined target position such that the entire virtual object is visible and is displayed at a quality level corresponding to the operation state. Claim 3 The image processing apparatus according to claim 1, wherein the image processing means arranges the virtual object so that the virtual object moves to the target position over a plurality of frames. Claim 4 The image processing apparatus according to claim 3, wherein when the operation state of the user is forward movement, the determination means determines a target position for repositioning the virtual object such that the target position of the virtual object is at the upper part of the real-world image. Claim 5 The image processing apparatus according to claim 3, wherein when the operation state of the user is downward movement on a staircase, the determination means determines a target position for repositioning the virtual object such that the target position of the virtual object is at the lower part of the real-world image. Claim 6 The image processing apparatus according to claim 4 or 5, wherein the determination means further changes the range of the target position of the virtual object according to the moving speed of the user. Claim 7 The image processing apparatus according to any one of claims 1 to 6, wherein the image processing means changes the frame rate according to the operation state of the user. Claim 8 The image processing apparatus according to any one of claims 1 to 7, wherein the image processing means adjusts the size of the virtual object over a plurality of frames. Claim 9 The image processing means arranges the plurality of virtual objects based on a predetermined overlay priority when there are a plurality of the virtual objects, according to any one of claims 1 to 8, wherein the image processing apparatus is characterized.

10. The operating state includes information on whether or not the device is moving, speed, and direction of travel, according to any one of claims 1 to 9, wherein the image processing apparatus is characterized.

11. The image processing means performs image processing on the virtual object according to the operating state and arranges it, according to any one of claims 1 to 10, wherein the image processing apparatus is characterized.

12. The image processing is a process of applying a shadow to the virtual object, according to claim 11, wherein the image processing apparatus is characterized.

13. The image processing is a process of applying lighting to the virtual object, according to claim 11 or 12, wherein the image processing apparatus is characterized.

14. The image processing is a process of changing the quality of rendering the virtual object, according to any one of claims 11 to 13, wherein the image processing apparatus is characterized.

15. The target position for repositioning the virtual object corresponds to a predetermined area of the screen and is based on the operating state, according to any one of claims 1 to 14, wherein the image processing apparatus is characterized.

16. The quality level at which the virtual object is displayed when the user is moving in the direction of travel is lower than the quality level in the operating state where the user is not moving, according to any one of claims 1 to 15, wherein the image processing apparatus is characterized.

17. The image processing apparatus further includes a calculating means for calculating repositioning coordinates used to draw the virtual object from a position in the image of the composite world required by the application that generated the virtual object to the target position, according to any one of claims 1 to 16, wherein the image processing apparatus is characterized.

18. An image processing method executed by an image processing apparatus that generates an image of a composite world by superimposing a virtual object on an image of the real world, An acquisition step of acquiring information related to an operating state indicating the direction of travel of the user of the image processing apparatus in the real world; Based on the acquired operating state, from the position required by the application that generated the virtual object, a determination step of determining a target position for repositioning the virtual object on the image of the real world to a target position that directs the user's line of sight in the direction of travel based on the acquired operating state; An image processing step of generating an image of a composite world in which the virtual object is arranged at the determined target position; An image processing method characterized by comprising the above.

19. A program for causing a computer to execute each step of the image processing method according to Claim 18.

Images