Information processing device, eyeglass-type device, and program

Abstract

An information processing device is provided that is mounted on a glasses-type device having a display unit. [Solution] An information processing device is provided that includes an information storage unit that stores captured images, an identification unit that identifies a predetermined visual target of the user of the eyeglass-type device that is included in the captured image stored in the information storage unit, a determination unit that determines the life-size size of the visual target identified by the identification unit, and an adjustment unit that adjusts the display of the captured image on the display unit based on the life-size size of the visual target determined by the determination unit so that the display size of the visual target included in the captured image more closely matches the visual size of the visual target when viewed from a predetermined distance away.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an eyeglass-type device, and a program. [Background technology]

[0002] Patent Document 1 describes a head-mounted display system that can prevent a decrease in convenience for the operator even when the size of an image is changed. [Prior art document] [Patent documents] [Patent Document 1] JP 2022-154409 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided an information processing device mounted on an eyeglass-type device having a display unit. The information processing device may include an information storage unit that stores captured images. The information processing device may include an identification unit that identifies a predetermined visual target of a user of the eyeglass-type device that is included in the captured images stored in the information storage unit. The information processing device may include a determination unit that determines a life-size size of the visual target identified by the identification unit. The information processing device may include an adjustment unit that adjusts the display of the captured image on the display unit based on the life-size size of the visual target determined by the determination unit so that the display size of the visual target included in the captured image more closely matches the visual size of the visual target when viewed from a predetermined distance away.

[0004] In the information processing device, when the adjustment unit adjusts the display of the captured image on the display unit so that the captured image is enlarged at a magnification higher than a predetermined magnification threshold, the adjustment unit may perform a high-resolution process on the display portion of the captured image that is displayed on the display unit.

[0005] Any of the information processing devices may further include a determination unit that determines whether the captured image includes the entire visual target, and when the determination unit determines that the captured image includes the entire visual target, the adjustment unit may adjust the display of the captured image on the display unit so that the display size of the visual target included in the captured image more closely matches the visual size of the visual target when viewed from a distance of the separation distance, within a range in which the entire visual target is displayed on the display unit.

[0006] Any of the information processing devices may further include an image generation unit that, when the determination unit determines that the captured image does not include the entirety of the visual target, generates a complementary image based on the captured image, in which missing portions of the visual target that are not included in the captured image are filled in, and the adjustment unit may adjust the display of the complementary image on the display unit so that, within a range in which the entirety of the visual target is displayed on the display unit, the display size of the visual target included in the complementary image more closely matches the visual size of the visual target when viewed from a distance of the separation distance.

[0007] In any of the information processing devices, the image generation unit may generate the complementary image from the captured image stored in the information storage unit using a generation model that generates a complementary image in which missing parts of the visual target are complemented from an captured image that includes a part of the visual target.

[0008] Any of the information processing devices may further include a learning data storage unit that stores learning data including captured images in which the entire visual target is captured; a model generation unit that uses the plurality of learning data stored in the learning data storage unit as teacher data to generate a generative model through machine learning that generates a complemented image in which missing portions of the visual target are complemented from captured images that include a portion of the visual target; and an image generation unit that uses the generative model to generate the complemented image from the captured image stored in the information storage unit.

[0009] In any of the information processing devices, the adjustment unit v =(S r × f) / D to estimate the perceived size of the viewed object when viewed from the separation distance, where S v may be the visual size of the visually recognized object when the visually recognized object is viewed from a distance of the separation distance, and S r may be the life size of the viewed object, f may be the average focal length of the human eye, and D may be the separation distance.

[0010] Any of the information processing devices may further include a setting unit that sets the range of the separation distance that the user can set based on the life-size size of the visual target so that the visual size of the visual target is within a predetermined range.

[0011] In any of the information processing devices, the determination unit may determine the life-size size of the visually recognized target by estimating the life-size size of the visually recognized target based on the captured image.

[0012] In any of the information processing devices, the determination unit may determine the life-size size of the visual target by estimating the life-size size of the visual target based on scan data obtained by scanning the visual target.

[0013] In any of the information processing devices, the information storage unit may further store visual target identification information that identifies the visual target and life-size information that indicates the life-size size of the visual target in association with each other, and the determination unit may determine the life-size size of the visual target by acquiring the life-size size information of the visual target that is stored in the information storage unit in association with the visual target identification information of the visual target identified by the identification unit.

[0014] According to one embodiment of the present invention, there is provided a glasses-type device equipped with any one of the above information processing devices.

[0015] According to one embodiment of the present invention, there is provided a program that, when executed by a computer, causes the computer to function as any one of the information processing devices.

[0016] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0017] [Figure 1] An example of a system 10 is shown schematically. [Figure 2] 10 is an explanatory diagram for explaining an example of the relationship between the distance between a user 102 and an object 55 watched by the user 102 and the visual size of the object 55 watched by the user 102. FIG. [Figure 3] 1 is an explanatory diagram for explaining an example of displaying an image on the glasses-type device 100. FIG. [Figure 4] 1 shows an example of the configuration of a glasses-type device 100. [Figure 5] 10 is an explanatory diagram for explaining an example of estimating the visual size of a visually recognized object 55. FIG. [Figure 6] 10 is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. FIG. [Figure 7] 10 is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. FIG. [Figure 8] 10 is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. FIG. [Figure 9] 1 shows an example of a functional configuration of an information processing device 150. [Figure 10] An example of information stored in the information processing device 150 is shown below. [Figure 11] 2 is an explanatory diagram illustrating an example of a processing flow of the glasses-type device 100. FIG. [Figure 12]An example of the hardware configuration of a computer 1200 that functions as an information processing device 150 is shown in schematic form. DETAILED DESCRIPTION OF THE INVENTION

[0018] When captured images are displayed on AR (Augmented Reality) glasses or the like, all captured images are displayed at the same display size unless the user of the AR glasses manually changes the display size of the captured images. Therefore, in order for subjects such as people and buildings in the captured images to be displayed on the AR glasses or the like as if they were viewed directly by the user's eyes, the user must manually set the display size of each captured image displayed on the AR glasses or the like. The system according to this embodiment employs a mechanism that uses, for example, AI (Artificial Intelligence) technology to estimate the life-size size of the main subject in the captured image and then displays the captured image on the AR glasses or the like that is automatically adjusted to more closely match the perceived size of the subject when viewed directly by the user's eyes.

[0019] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. In the drawings, the same reference numerals are used to designate the same or similar parts, and redundant explanations may be omitted.

[0020] 1 schematically illustrates an example of a system 10. The system 10 may include a glasses-type device 100. The system 10 may include an image database 200.

[0021] The glasses-type device 100 may be any device that has a display function for displaying content on a display unit mounted on the device itself. The glasses-type device 100 is, for example, AR glasses.

[0022] The content may be in any data format. For example, the content may be a still image. For example, the content may be a video. For example, the content may be text. For example, the content may be an illustration. For example, the content may be 3D CG (Computer Graphics) data. The content may include a combination of these.

[0023] The eyeglasses-type device 100 may be a device that can give the user 102 of the eyeglasses-type device 100 the sensation that the content is located in real space by displaying the content on transparent or semi-transparent glasses. That is, the eyeglasses-type device 100 may be a so-called transparent (see-through) device. The eyeglasses-type device 100 may also be a video pass-through device. In this case, the eyeglasses-type device 100 may display, on a non-transparent display, a captured image capturing a range corresponding to the field of view of the user 102 of the eyeglasses-type device 100, and superimpose the content.

[0024] The glasses-type device 100 has, for example, an access function for accessing the network 20. The glasses-type device 100 is, for example, smart glasses.

[0025] The network 20 may include a cloud. The network 20 may include a mobile communication network. The mobile communication network may conform to any of the following communication methods: 3G (3rd Generation), LTE (Long Term Evolution), 5G (5th Generation), 6G (6th Generation) and later. The network 20 may conform to a short-range wireless communication method such as Wi-Fi (registered trademark), Bluetooth (registered trademark), or Zigbee (registered trademark). The network 20 may include a wired network.

[0026] The glasses-type device 100 communicates with, for example, an image database 200 that stores captured images. The glasses-type device 100 communicates with the image database 200 via, for example, a network 20.

[0027] The captured image may be, for example, a still image, a moving image, or an example of content.

[0028] The glasses-type device 100 receives captured images from, for example, the image database 200. The captured images are, for example, captured images taken by the user 102 of the glasses-type device 100 using a camera. The captured images may also be, for example, captured images taken by a user other than the user 102 of the glasses-type device 100 using a camera. The glasses-type device 100 may display the captured images received from the image database 200 on its own display unit. Details of the glasses-type device 100 displaying captured images on its own display unit will be described later.

[0029] 2 is an explanatory diagram illustrating an example of the relationship between the distance between the user 102 and the visual target 55 of the user 102 and the visual size of the visual target 55 by the user 102. In FIG. 2, an example is shown in which the user 102 visually recognizes the visual target 55 while wearing the eyeglass-type device 100.

[0030] The visually recognized object 55 may be any object. The object may be, for example, a human being. The object may be, for example, a non-human animal. The object may be, for example, a building such as a house or a building. The object may also be a natural object such as a tree or a rock. FIG. 2 shows an example in which the visually recognized object 55 is a human being.

[0031] 2, as the distance between the user 102 and the visual target 55 increases from 1 m to 5 m to 10 m, the visual size of the visual target 55 by the user 102 decreases. Note that the visual size may mean the size of the visual target 55 that appears in the eyes of the user 102 when the user 102 visually recognizes the visual target 55.

[0032] 3 is an explanatory diagram for explaining an example of displaying an image on the glasses-type device 100. Here, an example in which the glasses-type device 100 displays a captured image 50 on its own display unit will be mainly described.

[0033] For example, the glasses-type device 100 identifies the visual target 55 of the user 102 included in the captured image 50. Next, the glasses-type device 100 determines the life-size of the visual target 55. Note that the life-size of the visual target 55 may mean the actual size of the visual target 55. For example, if the height of the visual target 55 is 170 cm, the life-size of the visual target 55 is 170 cm.

[0034] Thereafter, the glasses-type device 100 adjusts the display of the captured image 50 on the display unit based on the life-size size of the viewed target 55. The glasses-type device 100 adjusts the display of the captured image 50 on the display unit, for example, so that the display size of the viewed target 55 included in the captured image 50 more closely matches the visual size of the viewed target 55 when viewed from a predetermined distance away. For example, if the distance is 1 meter, the glasses-type device 100 adjusts the display of the captured image 50 on the display unit so that the size of the viewed target 55 as seen by the user 102's eyes when the user 102 views the viewed target 55 included in the captured image 50 displayed on the display unit more closely matches the size of the viewed target 55 as seen by the user 102's eyes when the user 102 views the viewed target 55 from a position 1 meter away. Note that the display size may refer to the size displayed on the display unit of the glasses-type device 100.

[0035] The upper diagram in Fig. 3 is the captured image 50 displayed on the display unit before the display of the captured image 50 on the display unit is adjusted. The lower diagram in Fig. 3 is the captured image 50 displayed on the display unit after the display of the captured image 50 on the display unit is adjusted. As shown in Fig. 3, the glasses-type device 100 adjusts the display of the captured image 50 on the display unit by enlarging the display size of the viewed object 55 so that the display size of the viewed object 55 more closely matches the viewed size of the viewed object 55 when viewed from a distance of the distance.

[0036] Since the emergence of eyeglass-type devices such as AR glasses and smart glasses on the market, demand for eyeglass-type devices has been on the rise, and this trend is expected to continue. One example of a use of eyeglass-type devices is viewing captured images. When viewing captured images using eyeglass-type devices, there is a need to view the captured images while focusing on a specific subject that may be included in the captured images. For example, a parent of a child viewing multiple captured images in which the child may be a subject may want to view the multiple captured images while focusing on the child that may be included in the multiple captured images. However, when viewing captured images using a conventional eyeglass-type device, unless a user of the eyeglass-type device manually adjusts the display size of the captured images, the eyeglass-type device displays all of the multiple captured images at the same display size on its display unit. Therefore, unless a user of the eyeglass-type device manually adjusts the display size of the captured images, the display size of a specific subject that may be included in the captured images displayed on the display unit of the eyeglass-type device changes depending on the imaging distance from the camera that captured the captured images to the specific subject. Therefore, when a specific subject included in one captured image is displayed on the display unit of the eyeglass-type device, the specific subject may be displayed at a large size, while when a specific subject included in another captured image is displayed on the display unit of the eyeglass-type device, the specific subject may be displayed at a small size. This means that it is difficult to view the multiple captured images while focusing on a specific subject that may be included in the multiple captured images. Therefore, when viewing multiple captured images using a conventional eyeglass-type device while focusing on a specific subject that may be included in the multiple captured images, the user of the eyeglass-type device must manually adjust the display size of the captured image displayed on the display unit of the eyeglass-type device to make it easier to focus on the specific subject, which places a heavy burden on the user of the eyeglass-type device. For these reasons, it is desirable to reduce the burden on the user of the eyeglass-type device when viewing multiple captured images using the eyeglass-type device while focusing on a specific subject that may be included in the multiple captured images.

[0037] In contrast, according to the system 10 of this embodiment, when the glasses-type device 100 displays a captured image 50, the glasses-type device 100 identifies a visual target 55 that is included in the captured image 50 and that is visually recognized by the user 102 of the glasses-type device 100, and determines the actual size of the identified visual target 55. Then, based on the determined actual size, the glasses-type device 100 adjusts the display of the captured image 50 on the display unit so that the display size of the visual target 55 included in the captured image 50 more closely matches the visual size of the visual target 55 when viewed from a predetermined distance away. Thus, the glasses-type device 100 can automatically adjust the display of the captured image 50 on the display unit so that the user 102 can easily focus on a specific visual target 55 that may be included in multiple captured images 50, without the user 102 having to manually adjust the display of the captured image 50 on the display unit. As a result, the system 10 according to the present embodiment can reduce the burden on a user of an eyeglass-type device when viewing a plurality of captured images while focusing on a specific subject that may be included in the captured images. Furthermore, according to the system 10 according to the present embodiment, a user 102 viewing a plurality of captured images 50 of a visual target 55 captured at different times using the eyeglass-type device 100 can recognize the transition of the visual target 55 over time not only from changes in the appearance of the visual target 55 but also from changes in the display size of the visual target 55. For example, a parent viewing a plurality of captured images 50 of a child captured at different times using the eyeglass-type device 100 can recognize the child's growth not only from changes in the child's facial expression and clothing but also from changes in the child's display size. As a result, the system 10 according to the present embodiment can more easily enable a user viewing a plurality of captured images of a visual target captured at different times using the eyeglass-type device to intuitively recognize the transition of the visual target over time. As a result, the system 10 of this embodiment can contribute to the spread of eyeglass-type devices by satisfying the need for eyeglass-type devices that allow users to view multiple captured images while focusing on specific subjects that may be contained in the images.

[0038] 4 schematically shows an example of the configuration of the eyeglass-type device 100. The eyeglass-type device 100 may include a frame 110 having rims 112, a bridge 114, and temples 118.

[0039] The glasses-type device 100 includes, for example, a lens 116. The lens 116 may be an example of a display unit of the glasses-type device 100.

[0040] The lens 116 may be a transparent or semi-transparent display. In this case, the eyeglass-type device 100 may be a transparent or semi-transparent display-type device. The eyeglass-type device 100 may also have a projection function for projecting a display onto the lens 116. In this case, the eyeglass-type device 100 may be a projection-type device. The projection function of the eyeglass-type device 100 may be an example of the display function of the eyeglass-type device 100.

[0041] The glasses-type device 100 includes, for example, a scanner 180. The scanner 180 may scan an object. The scanner 180 has, for example, an imaging function. In this case, the scanner 180 includes a camera. The scanner 180 has, for example, a ranging function. In this case, the scanner 180 includes a ranging sensor such as LiDAR (Light Detection And Ranging). The scanner 180 may include both a camera and a ranging sensor.

[0042] The glasses-type device 100 includes, for example, a camera 190. Fig. 4 shows an example in which the glasses-type device 100 includes one camera 190. The glasses-type device 100 may include multiple cameras 190. When the scanner 180 has an imaging function, the scanner 180 and the camera 190 may be an integrated device or separate devices.

[0043] The imaging range of camera 190 is, for example, a range corresponding to the field of view of user 102. The imaging range of camera 190 may be wider than the field of view of user 102, or may be substantially the same as the field of view of user 102.

[0044] 4 is an example. The scanner 180 and the camera 190 may be placed at any other position on the frame 110.

[0045] The glasses-type device 100 includes, for example, an operation unit. The glasses-type device 100 receives various inputs via the operation unit. The operation unit may be disposed at any position on the glasses-type device 100.

[0046] The glasses-type device 100 includes an input unit such as a button. The glasses-type device 100 receives various inputs via the input unit. The input unit may be disposed at any position on the frame 110.

[0047] The glasses-type device 100 includes, for example, an information processing device 150. The information processing device 150 may perform various types of information processing.

[0048] The information processing device 150, for example, executes a communication process to communicate with an external device. The information processing device 150, for example, executes a display process to display content on a display unit of the glasses-type device 100. The information processing device 150, for example, executes an identification process to identify a visual target 55 of the user 102 included in a captured image 50. The information processing device 150, for example, executes a determination process to determine the life-size size of the visual target 55. The information processing device 150, for example, executes a setting process to set a separation distance. The information processing device 150, for example, executes an adjustment process to adjust the display of the captured image 50 on a display unit of the glasses-type device 100.

[0049] The arrangement of the information processing device 150 in FIG. 4 is one example. The information processing device 150 may be arranged at another position on the frame 110. For example, the information processing device 150 is arranged at an arbitrary position inside the frame 110. For example, the information processing device 150 is arranged inside the temple 118. As a specific example, the information processing device 150 may be arranged at the tip of the temple 118, i.e., at the end of the temple. The information processing device 150 may also be arranged at an arbitrary position outside the frame 110.

[0050] The glasses-type device 100 includes, for example, a battery. The information processing device 150 may perform various information processing operations using power stored in the battery.

[0051] The glasses-type device 100 may communicate with an external device. For example, the glasses-type device 100 performs wireless communication with the external device. For example, the glasses-type device 100 performs short-range wireless communication with the external device. For example, the glasses-type device 100 performs mobile communication with the external device. The glasses-type device 100 may also perform wired communication with the external device.

[0052] The glasses-type device 100 communicates with, for example, an image database 200. The glasses-type device 100 may communicate with a communication terminal such as a mobile phone such as a smartphone, a tablet terminal, a wearable terminal, or a PC (Personal Computer). The glasses-type device 100 may accept various inputs by receiving input information from the communication terminal.

[0053] Fig. 5 is an explanatory diagram for explaining an example of estimating the visual size of visually recognized object 55. Here, various parameters shown in Fig. 5 will be explained first, and then an example of estimating the visual size of visually recognized object 55 will be explained.

[0054] S r is the life size of the viewed object 55. f is the average focal length of the human eye 105. D is the separation distance at which the viewed object 55 is viewed. S vis the visual size of the visual target 55 when the visual target 55 is visually recognized from a distance D.

[0055] The eyeglass-type device 100 is, for example, r Based on ,f, and D, S v The glasses-type device 100 estimates, for example, S v =(S r × f) / D, S v Estimate.

[0056] 6 is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. Here, differences from the above-described example of displaying an image on the glasses-type device 100 will be mainly described.

[0057] The upper diagram in Fig. 6 is the captured image 50 displayed on the display unit of the eyeglass-type device 100 before the display of the captured image 50 on the display unit is adjusted. The lower diagram in Fig. 6 is the captured image 50 displayed on the display unit after the display of the captured image 50 on the display unit is adjusted. As shown in Fig. 6, when the display of the captured image 50 on the display unit is adjusted to enlarge and display the visual target 55 on the display unit at a high magnification, the eyeglass-type device 100 may display only a portion of the captured image 50 that includes the visual target 55 on the display unit.

[0058] 7 is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. Here, differences from the above-described example of displaying an image on the glasses-type device 100 will be mainly described.

[0059] The upper diagram of Fig. 7 schematically shows an example of a captured image 50. As shown in the upper diagram of Fig. 7, the captured image 50 does not include the entire visual target 55. Note that a portion of the visual target 55 that is not included in the captured image 50 may be referred to as a missing portion of the visual target 55.

[0060] 7 is an explanatory diagram for explaining an example of complementing a missing portion 57 of a visually recognized object 55. Here, an example in which the glasses-type device 100 complements a missing portion 57 of a visually recognized object 55 will be mainly explained.

[0061] The glasses-type device 100 generates, for example, a complemented image 60 in which a missing portion 57 of a visual target 55 is complemented. The glasses-type device 100 generates the complemented image 60 from a captured image 50 using, for example, a generation model that generates, from a captured image including a part of the visual target, a complemented image in which a missing portion of the captured image is complemented.

[0062] Thereafter, the glasses-type device 100 may adjust the display of the generated complementary image 60 on the display unit of the glasses-type device 100. The glasses-type device 100 may adjust the display of the complementary image 60 on the display unit in the same manner as when adjusting the display of the captured image 50 on the display unit.

[0063] The lower diagram of Fig. 7 shows the complementary image 60 displayed on the display unit after the display of the complementary image 60 on the display unit has been adjusted. As shown in the lower diagram of Fig. 7, the glasses-type device 100 displays the complementary image 60 on its own display unit so that the display size of the visually recognized target 55 more closely matches the visually recognized size of the visually recognized target 55 when the visually recognized target 55 is viewed from a predetermined distance away.

[0064] According to the system 10 shown in Fig. 7, when a missing portion 57 of the visual target 55 exists, the glasses-type device 100 generates a complemented image 60 in which the missing portion 57 of the visual target 55 is complemented. The glasses-type device 100 then adjusts the display of the generated complemented image 60 on the display unit of the glasses-type device 100. In this way, the system 10 shown in Fig. 7 enables the user of the glasses-type device to view a captured image in which a missing portion of the visual target exists with the same feeling as when viewing a captured image in which the entire visual target is captured.

[0065] Fig. 8 is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. Fig. 8 shows an example in which the visual target 55 of the user 102 is a building. Here, differences from the above-described example of displaying an image on the glasses-type device 100 will be mainly described.

[0066] When the visual target 55 is a large, life-size object such as a building shown in FIG. 8 , the glasses-type device 100 may limit the range of the separation distance that the user 102 can set. For example, the glasses-type device 100 limits the separation distance that the user 102 can set by setting a lower limit value for the separation distance that the user 102 can set. The glasses-type device 100 may further limit the separation distance that the user 102 can set by further setting an upper limit value for the separation distance that the user 102 can set. The glasses-type device 100 may limit the separation distance that the user 102 can set by setting only the upper limit value for the separation distance.

[0067] The upper diagram in Fig. 8 shows the captured image 50 displayed on the display unit of the glasses-type device 100 before the display of the captured image 50 on the display unit is adjusted. The lower diagram in Fig. 8 shows the captured image 50 displayed on the display unit after the display of the captured image 50 on the display unit is adjusted. As shown in Fig. 8, the glasses-type device 100 limits the range of separation distances that can be set by the user 102, and then adjusts the display of the captured image 50 on the display unit so that the display size of the visual target 55 more closely matches the visual size of the visual target 55 when viewed from a distance of that separation distance.

[0068] When a visual target whose display size is too large is displayed on the display unit of the eyeglass-type device, the user may find it difficult to visually recognize the visual target, or the image quality of the visual target may be reduced. On the other hand, when a visual target whose display size is too small is displayed on the display unit of the eyeglass-type device, the user 102 may find it difficult to visually recognize the visual target. Therefore, when a visual target whose display size is too large or too small is displayed on the display unit of the eyeglass-type device, this may be one of the causes of a deterioration in the user experience of the eyeglass-type device.

[0069] In contrast to this, according to the system 10 shown in Fig. 8, the glasses-type device 100 limits the range of separation distances that can be set by the user 102 based on the life-size size of the visual target 55. As a result, the system 10 shown in Fig. 8 can appropriately limit the range of separation distances that can be set by the user while taking into consideration the life-size size of the visual target that the user is viewing, and therefore prevents a visual target that is too large or too small from being displayed on the display unit of the glasses-type device, thereby improving the user experience of the glasses-type device.

[0070] 9 schematically illustrates an example of the functional configuration of the information processing device 150. The information processing device 150 includes an information storage unit 152, an acquisition unit 154, a setting unit 155, an identification unit 156, a determination unit 158, an adjustment unit 160, a determination unit 162, an image generation unit 164, a training data storage unit 166, a model generation unit 168, and a model storage unit 170. Note that it is not essential that the information processing device 150 includes all of these components.

[0071] The information storage unit 152 stores various types of information. The information storage unit 152 stores, for example, the captured image 50. The information storage unit 152 stores, for example, metadata of the captured image 50. The metadata of the captured image 50 includes, for example, image capture date and time data indicating the date and time when the captured image 50 was captured. The captured image 50 includes, for example, resolution data indicating the resolution of the captured image 50. The metadata of the captured image 50 may include any other data related to the captured image 50. The information storage unit 152 stores, for example, the captured image 50 and the metadata of the captured image 50 in association with each other.

[0072] The information storage unit 152 stores, for example, visually recognized target identification information that identifies the visually recognized target 55 of the user 102 of the glasses-type device 100. The visually recognized target identification information is, for example, an identifier that identifies the visually recognized target 55. The information storage unit 152 stores, for example, the visually recognized target identification information and various information related to the visually recognized target 55 in association with each other.

[0073] The information storage unit 152 stores, for example, attribute information indicating the attributes of the visually recognized target 55. The attribute information of the visually recognized target 55 includes, for example, classification information indicating the classification of the object. Examples of object classification include humans, animals, buildings, natural objects, etc. The attribute information of the visually recognized target 55 includes, for example, gender information indicating the gender of the object. The attribute information of the visually recognized target 55 includes, for example, age information indicating the age of the object. The information storage unit 152 stores, for example, visually recognized target identification information and attribute information indicating the attributes of the visually recognized target 55 in association with each other.

[0074] The information storage unit 152 stores, for example, life-size information indicating the life-size size of the visually identified target 55. The information storage unit 152 stores, for example, the visually identified target identification information and the life-size information of the visually identified target 55 in association with each other.

[0075] For example, if the viewed target 55 is a human or an animal, the life-size information of the viewed target 55 may indicate the life-size of the viewed target 55 for each age group. Examples of age intervals include 6 months, 1 year, 3 years, 5 years, and 10 years. For example, if the viewed target 55 is a 2-year-old child and the age interval is 1 year, the life-size information of the child may indicate the life-size of the child when he / she was 0 years old, the life-size of the child when he / she was 1 year old, and the life-size of the child when he / she was 2 years old.

[0076] The information storage unit 152 stores, for example, image data of the visually identified target 55. The image data of the visually identified target 55 is, for example, image data including the entire visually identified target 55. The image data of the visually identified target 55 may be image data including only a characteristic part of the visually identified target 55. The information storage unit 152 stores, for example, visually identified target identification information and image data of the visually identified target 55 in association with each other.

[0077] For example, if the viewed target 55 is a human or an animal, the image data of the viewed target 55 may include image data for each age of the viewed target 55. For example, if the viewed target 55 is a two-year-old child and the age interval is one year, the image data of the child includes image data when the child was zero years old, image data when the child was one year old, and image data when the child was two years old.

[0078] The information storage unit 152 stores, for example, average focal length information indicating the average focal length of the human eye, and user focal length information indicating the focal length of the eye of the user 102.

[0079] The acquisition unit 154 acquires various types of information. For example, the acquisition unit 154 acquires the various types of information by receiving the various types of information via the network 20. The acquisition unit 154 acquires the various types of information by an operation unit or an input unit included in the glasses-type device 100 accepting various inputs. The acquisition unit 154 may store the acquired various types of information in the information storage unit 152.

[0080] The acquisition unit 154 acquires, for example, a captured image. The acquisition unit 154 acquires, for example, a captured image by receiving the captured image from the image database 200. The acquisition unit 154 may acquire the captured image by capturing the captured image using a camera 190 mounted on the glasses-type device 100.

[0081] The acquisition unit 154 acquires, for example, scan data. The acquisition unit 154 acquires, for example, scan data obtained by scanning an object. The acquisition unit 154 acquires, for example, scan data obtained by scanning the visually identified target 55.

[0082] The acquiring unit 154 acquires, for example, scan data obtained by a scanner 180 installed in the glasses-type device 100. The acquiring unit 154 may also acquire scan data obtained by a scanner not installed in the glasses-type device 100. In this case, the acquiring unit 154 may acquire the scan data by receiving the scan data from the scanner via the network 20.

[0083] The acquiring unit 154 may acquire, for example, various models. The acquiring unit 154 may store the acquired various models in the model storage unit 170.

[0084] The acquisition unit 154 acquires, for example, an identification model. The identification model may be a model that identifies a visually recognized target included in a captured image from the captured image. The identification model may be an AI model.

[0085] The acquisition unit 154 acquires, for example, an estimation model. The estimation model may be a model that estimates the life-size size of a visually recognized object included in a captured image from the captured image. The estimation model may be an AI model.

[0086] The acquisition unit 154 acquires, for example, a high resolution processing model. The high resolution processing model is a model that, when a captured image, resolution data of the captured image, magnification data indicating a magnification rate of the captured image, and display portion data indicating a display portion of the captured image to be displayed on a display unit are input as input data, performs high resolution processing on the display portion. The high resolution processing model may be an AI model.

[0087] The acquisition unit 154 acquires, for example, a generative model. The generative model is a model that generates a complemented image in which a missing part of a visually recognized object is complemented from a captured image including the part of the visually recognized object. The generative model may be an AI model.

[0088] The setting unit 155 executes various setting processes. The setting unit 155 executes various setting processes based on, for example, various information stored in the information storage unit 152. The setting unit 155 executes various setting processes based on, for example, input information acquired by an operation unit or an input unit included in the glasses-type device 100 receiving various inputs. The setting unit 155 may execute various setting processes based on input information received from a communication terminal via the network 20.

[0089] The setting unit 155 sets, for example, the visual target 55 of the user 102 of the glasses-type device 100. The setting unit 155 sets, for example, the attributes of the visual target 55. The setting unit 155 sets, for example, the life-size size of the visual target 55. The setting unit 155 sets, for example, a separation distance used to adjust the display of the captured image 50 including the visual target 55 on the display unit of the glasses-type device 100.

[0090] The setting unit 155 sets, for example, a range of the separation distance that can be set by the user 102. The information storage unit 152 sets the range of the separation distance that can be set by the user 102 based on, for example, the life-size size of the visually recognized target 55.

[0091] The setting unit 155 sets a range of separation distances that can be set by the user 102, for example, so that the visible size of the visually recognized target 55 is within a predetermined range. The setting unit 155 sets a range of separation distances that can be set by the user 102, for example, by setting a lower limit value of the separation distances that can be set by the user 102. For example, when the life-size size of the visually recognized target 55 indicated by the life-size size information is larger than a predetermined first size threshold, the setting unit 155 sets a lower limit value of the separation distance. For example, the setting unit 155 sets a range of separation distances that can be set by the user 102, for example, by setting an upper limit value of the separation distances that can be set by the user 102. For example, the setting unit 155 sets a range of separation distances that can be set by the user 102, for example, by setting both a lower limit value of the separation distances that can be set by the user 102 and an upper limit value of the separation distances that can be set by the user 102.

[0092] The setting unit 155 sets the lower limit of the separation distance that can be set by the user 102 to, for example, the life-size of the visually recognized target 55×α. α may be a constant. For example, α is 0.5. For example, if the life-size of the visually recognized target 55 is 10 m and α is 0.5, the lower limit of the separation distance that can be set by the user 102 is 10 m×0.5=5 m.

[0093] The setting unit 155 sets the upper limit of the separation distance that can be set by the user 102 to, for example, the life-size of the visually recognized target 55×β. β may be a constant. For example, β is 5. For example, if the life-size of the visually recognized target 55 is 10 m and β is 5, the upper limit of the separation distance that can be set by the user 102 is 10 m×5=50 m.

[0094] The identification unit 156 identifies a predetermined visual target 55 of the user 102 of the glasses-type device 100 included in the captured image 50 stored in the information storage unit 152. The visual target 55 may be set by the setting unit 155.

[0095] The identification unit 156 identifies the visual target 55 included in the captured image 50, for example, by performing image analysis on the captured image 50. The identification unit 156 identifies the visual target included in the captured image 50 from the captured image 50, for example, by using an identification model stored in the model storage unit 170.

[0096] For example, when the captured image 50 includes a plurality of objects classified into the same category as the visually identified target 55, the identifying unit 156 identifies the visually identified target 55 from among the plurality of objects based on image data of the visually identified target 55 that is associated with the visually identified target identification information and stored in the information storage unit 152. The identifying unit 156 identifies the visually identified target 55 from among the plurality of objects, for example, by comparing the captured image 50 with the image data of the visually identified target 55.

[0097] When the visually identified target 55 is a human being or an animal, and the image data of the visually identified target 55 stored in the information storage unit 152 includes image data for each era of the visually identified target 55, the identification unit 156 may identify the visually identified target 55 from the plurality of objects further based on the metadata of the captured image 50 stored in the information storage unit 152. For example, the identification unit 156 identifies the era of the visually identified target 55 when the captured image 50 was captured from the imaging date and time of the captured image 50 indicated by imaging date and time data included in the metadata of the captured image 50, and identifies the visually identified target 55 from the plurality of objects by comparing the captured image 50 with the image data of the visually identified target 55 for the identified era.

[0098] When the plurality of objects are included in the captured image 50, the identification unit 156 may identify the visually identified target 55 from among the plurality of objects based on information for identifying the visually identified target 55 from among the plurality of objects. The information for identifying the visually identified target 55 from among the plurality of objects may be acquired by an operation unit or an input unit included in the glasses-type device 100 accepting various inputs.

[0099] The determining unit 158 ​​determines the life-size of the visually identified target 55 identified by the identifying unit 156. The determining unit 158 ​​determines the life-size of the visually identified target 55 based on, for example, various information stored in the information storage unit 152. When the determining unit 158 ​​determines the life-size of the visually identified target 55, the determining unit 158 ​​may update the life-size information of the visually identified target 55 stored in the information storage unit 152.

[0100] The determination unit 158 ​​determines the life-size of the visually identified target 55, for example, by estimating the life-size of the visually identified target 55 based on the captured image 50. The determination unit 158 ​​estimates the life-size of the visually identified target 55, for example, by performing image analysis on the captured image 50. The determination unit 158 ​​estimates the life-size of the visually identified target 55 included in the captured image 50, from the captured image 50, using an estimation model stored in the model storage unit 170, for example.

[0101] The determination unit 158 ​​may determine the life-size of the visually identified target 55 by estimating the life-size of the visually identified target 55 based on scan data obtained by scanning the visually identified target 55 and stored in the information storage unit 152. The determination unit 158 ​​may determine the life-size of the visually identified target 55 by acquiring life-size information of the visually identified target 55 that is associated with the visually identified target identification information of the visually identified target 55 and stored in the information storage unit 152.

[0102] The adjustment section 160 adjusts the display of the image on the display section of the glasses-type device 100. The glasses-type device 100 may display the image on the display section in accordance with the adjustment made by the adjustment section 160.

[0103] The image is, for example, a captured image 50. The image may also be a complement image 60.

[0104] The adjustment unit 160 adjusts the display of the captured image 50 on the display unit, for example. The adjustment unit 160 adjusts the display of the captured image 50 on the display unit, for example, based on the life-size size of the viewed target 55 determined by the determination unit 158. When the viewed target 55 is a human or an animal and the life-size information of the viewed target 55 stored in the information storage unit 152 indicates the life-size size of the viewed target 55 for each era, the adjustment unit 160 may adjust the display of the captured image 50 on the display unit further based on the metadata of the captured image 50 stored in the information storage unit 152. For example, the adjustment unit 160 identifies the era of the viewed target 55 when the captured image 50 was captured from the imaging date and time of the captured image 50 indicated by the imaging date and time data included in the metadata of the captured image 50, and adjusts the display of the captured image 50 on the display unit based on the life-size size of the viewed target 55 for the identified era.

[0105] The adjustment unit 160 adjusts the display of the captured image 50 on the display unit, for example, so that the display size of the visually recognized target 55 included in the captured image 50 more closely matches the visually recognized size of the visually recognized target 55 when the visually recognized target 55 is viewed from a distance of a predetermined distance. The distance may be set by the setting unit 155. If the distance is not set by the setting unit 155, the distance may be a default value registered in the information processing device 150 when the information processing device 150 is manufactured.

[0106] The adjustment unit 160 determines, for example, the visual size S of the visual target 55 when the visual target 55 is visually recognized at a distance D. v The adjustment unit 160 adjusts the display of the captured image 50 on the display unit by estimating the average focal length f of the human eye indicated by the average focal length information stored in the information storage unit 152 and the life-size size S of the visually recognized object 55. r Based on and v The adjustment unit 160 estimates, for example, S v =(S r × f) / D, S v The adjustment unit 160 estimates the focal length f of the eye of the user 102 indicated by the user focal length information stored in the information storage unit 152. u and a life-size S of the visual target 55 r Based on and v The adjustment unit 160 may estimate, for example, S v =(S r ×f u ) / D, S v Estimate.

[0107] The adjustment unit 160 adjusts the display of the captured image 50 on the display unit, for example, by adjusting the display position of the visual target 55 included in the captured image 50. The adjustment unit 160 adjusts the display position of the visual target 55 on the display unit so that the visual target 55 is displayed in the center of the display unit, for example.

[0108] The adjustment unit 160 adjusts the display of the captured image 50 on the display unit, for example, by performing a high resolution process on the captured image 50. The adjustment unit 160 performs a high resolution process on, for example, a display portion of the captured image 50 that is displayed on the display unit. For example, when adjusting the display of the captured image 50 on the display unit so that the captured image 50 is enlarged and displayed at a magnification rate higher than a predetermined magnification rate threshold, the adjustment unit 160 performs a high resolution process on the display portion of the captured image 50 that is displayed on the display unit.

[0109] The adjustment unit 160, for example, uses a high resolution processing model to perform high resolution processing on a display portion of the captured image 50 that is displayed on the display unit. The adjustment unit 160 performs high resolution processing on a display portion of the captured image 50 that is displayed on the display unit by inputting, as input data, the captured image 50, resolution data of the captured image 50 that is included in the metadata of the captured image 50 stored in the information storage unit 152, magnification rate data that indicates the magnification rate of the captured image 50, and display portion data that indicates the display portion of the captured image 50 that is displayed on the display unit, into the high resolution processing model.

[0110] The adjustment unit 160 may adjust the display of the captured image 50 on the display unit of the eyeglasses-type device 100 based on information for adjusting the display of the captured image 50 on the display unit of the eyeglasses-type device 100. The information for adjusting the display of the captured image 50 on the display unit of the eyeglasses-type device 100 may be acquired by an operation unit or an input unit included in the eyeglasses-type device 100 accepting various inputs.

[0111] The determination unit 162 executes various determination processes based on various information stored in the information storage unit 152, for example.

[0112] The determination unit 162 executes various determination processes, for example, when the identification unit 156 identifies the visual target 55 of the user 102 of the glasses-type device 100 included in the captured image 50. The determination unit 162 executes various determination processes, for example, when the determination unit 158 ​​determines the life-size size of the visual target 55.

[0113] The determination unit 162 determines, for example, whether or not the captured image 50 includes the entire visual target 55. The determination unit 162 determines, for example, whether or not the captured image 50 includes the entire visual target 55 by performing image analysis on the captured image 50.

[0114] For example, when the determination unit 162 determines that the captured image 50 includes the entire visual target 55, the adjustment unit 160 adjusts the display of the captured image 50 on the display unit. In this case, the adjustment unit 160 may adjust the display of the captured image 50 on the display unit so that the display size of the visual target included in the captured image 50 more closely matches the visual size of the visual target 55 when viewed from a predetermined distance away, within a range in which the entire visual target 55 is displayed on the display unit.

[0115] The image generation unit 164 generates a complemented image 60 in which a missing portion 57 of the visual target 55 that is not included in the captured image 50 is complemented. The image generation unit 164 generates the complemented image 60, for example, when the determination unit 162 determines that the captured image 50 does not include the entire visual target 55.

[0116] The image generation unit 164 generates the complemented image 60 based on, for example, the captured image 50. The image generation unit 164 generates the complemented image 60 from the captured image 50 using, for example, a generative model stored in the model storage unit 170.

[0117] The adjustment unit 160 adjusts, for example, the display of the complementary image 60 on the display unit. For example, when the image generation unit 164 generates the complementary image 60, the adjustment unit 160 adjusts the display of the complementary image 60 on the display unit. For example, the adjustment unit 160 adjusts the display of the complementary image 60 on the display unit so that the display size of the visually recognized target 55 included in the complementary image 60 more closely matches the visually recognized size of the visually recognized target 55 when viewed from a predetermined distance away, within a range in which the entire visually recognized target 55 is displayed on the display unit.

[0118] The adjustment section 160 may adjust the display of the complementary image 60 on the display section based on the same information as the information used when adjusting the display of the captured image 50 on the display section. The adjustment section 160 may adjust the display of the complementary image 60 on the display section in the same manner as when adjusting the display of the captured image 50 on the display section.

[0119] The determination unit 162 may determine whether the life-size of the viewed target 55 determined by the determination unit 158 ​​is larger than a predetermined second size threshold. For example, if the determination unit 162 determines that the life-size of the viewed target 55 is larger than the second size threshold, the adjustment unit 160 adjusts the display of the image on the display unit so that the display size of the viewed target 55 more closely matches the visual size of the viewed target 55 when viewed from a predetermined distance away. On the other hand, if the determination unit 162 determines that the life-size of the viewed target 55 is smaller than the second size threshold, the adjustment unit 160 does not adjust the display of the image on the display unit so that the display size of the viewed target 55 more closely matches the visual size of the viewed target 55 when viewed from a predetermined distance away. In this case, the adjustment unit 160 may adjust the display of the image on the display unit by performing at least one of an adjustment to adjust the display position at which the visual target 55 included in the image is displayed on the display unit and an adjustment to perform high-resolution processing on the captured image 50.

[0120] The acquisition unit 154 acquires, for example, learning data including a captured image in which the entire target 55 viewed by the user 102 of the glasses-type device 100 is captured. The acquisition unit 154 may store the acquired learning data in the learning data storage unit 166.

[0121] The model generation unit 168 generates a generative model. The model generation unit 168 generates the generative model by machine learning, for example, using a plurality of pieces of training data stored in the training data storage unit 166 as training data. The model generation unit 168 may store the generated generative model in the model storage unit 170.

[0122] 10 shows an example of information stored in the information processing device 150. Here, the example of information stored in the information processing device 150 in association with an identifier that identifies the visual target 55 of the user 102 of the glasses-type device 100 will be mainly described.

[0123] 10 , attribute information of the visually recognized target 55, life-size information of the visually recognized target 55, and image data of the visually recognized target 55 are stored in the information processing device 150 in association with the identifier of the visually recognized target 55. Also, as shown in FIG. 10 , the attribute information of the visually recognized target 55 may include classification information of the visually recognized target 55, gender information of the visually recognized target 55, and age information of the visually recognized target 55.

[0124] The attribute information, life-size information, and image data of visually identified object 55 having the identifier "0001" shown in Fig. 10 are as follows: Note that visually identified object 55 having the identifier "0001" may be referred to as visually identified object 1.

[0125] The attribute information of the viewed target 1 includes classification information of the viewed target 1, gender information of the viewed target 1, and age information of the viewed target 1. The classification of the viewed target 1 indicated by the classification information of the viewed target 1 is "human". The gender of the viewed target 1 indicated by the gender information of the viewed target 1 is "male". The age of the viewed target 1 indicated by the age information of the viewed target 1 is "2 years old". The life-size information of the viewed target 1 indicates the life-size of the viewed target 1 for each age group. When the viewed target 1 is "0 years old", the life-size of the viewed target 1 is "70 cm". When the viewed target 1 is "1 year old", the life-size of the viewed target 1 is "80 cm". When the viewed target 1 is "2 years old", the life-size of the viewed target 1 is "90 cm". The image data of the viewed target 1 includes image data for each age group of the viewed target 1. The image data of the visually identified object 1 includes image data of the visually identified object 1 when the visually identified object 1 is "0 years old", image data of the visually identified object 1 when the visually identified object 1 is "1 year old", and image data of the visually identified object 1 when the visually identified object 1 is "2 years old".

[0126] The attribute information, life-size information, and image data of visually identified target 55 with the identifier "0002" shown in Fig. 10 are as follows: Note that visually identified target 55 with the identifier "0002" may be referred to as visually identified target 2.

[0127] The attribute information of the viewed target 2 includes classification information of the viewed target 2, gender information of the viewed target 2, and age information of the viewed target 2. The classification of the viewed target 2 indicated by the classification information of the viewed target 2 is "animal." The gender of the viewed target 2 indicated by the gender information of the viewed target 2 is "female." The age of the viewed target 2 indicated by the age information of the viewed target 2 is "3 years old." The life-size size of the viewed target 2 indicated by the life-size size information of the viewed target 1 is "50 cm." The image data of the viewed target 2 is image data of a "dog."

[0128] The attribute information, life-size information, and image data of visually identified target 55 with the identifier "0003" shown in Fig. 10 are as follows: Note that visually identified target 55 with the identifier "0003" may be referred to as visually identified target 3.

[0129] The attribute information of the visually identified target 3 includes classification information of the visually identified target 3. The classification of the visually identified target 3 indicated by the classification information of the visually identified target 3 is "animal." The life-size information of the visually identified target 3 indicates that the life-size of the visually identified target 3 is "3.0 mm." The image data of the visually identified target 3 is image data of an "ant."

[0130] The attribute information, life-size information, and image data of visually identified target 55 with the identifier "0004" shown in Fig. 10 are as follows: Note that visually identified target 55 with the identifier "0004" may be referred to as visually identified target 4.

[0131] The attribute information of the visually identified target 4 includes classification information of the visually identified target 4. The classification of the visually identified target 4 indicated by the classification information of the visually identified target 4 is a "building." The life-size information of the visually identified target 4 indicates that the life-size of the visually identified target 4 is "5m." The image data of the visually identified target 4 is image data of a "detached house."

[0132] 11 is an explanatory diagram illustrating an example of the processing flow of the glasses-type device 100. Here, the starting state is a state in which the separation distance used to adjust the display of the captured image 50 including the visual target 55 of the user 102 of the glasses-type device 100 on the display unit of the glasses-type device 100 has not been set.

[0133] In step (step may be abbreviated as S) 102, the setting unit 155 sets the separation distance. In S104, the identification unit 156 identifies the visual target 55 included in the captured image 50 stored in the information storage unit 152. In S106, the determination unit 158 ​​determines the life-size size of the visual target 55 identified by the identification unit 156 in S104.

[0134] In S108, the determination unit 162 determines whether or not the captured image 50 includes the entire visual target 55 identified by the identification unit 156 in S104. If the determination unit 162 determines that the captured image 50 includes the entire visual target 55, the process proceeds to S110. If the determination unit 162 determines that the captured image 50 does not include the entire visual target 55, the process proceeds to S114.

[0135] In S110, the adjustment unit 160 adjusts the display of the captured image 50 on the display unit based on the life-size size of the visual target 55 determined by the determination unit 158 ​​in S106 so that the display size of the visual target 55 included in the captured image 50 more closely matches the visual size of the visual target 55 when the visual target 55 is visually recognized from a distance set by the setting unit 155 in S102. In S112, the glasses-type device 100 displays the captured image 50 on the display unit in accordance with the adjustment by the adjustment unit 160 in S110. Thereafter, the process of the glasses-type device 100 adjusting the display of the image on the display unit ends.

[0136] In S114, the image generation unit 164 generates a complemented image 60 by complementing a missing portion 57 of the visual target 55 that is not included in the captured image 50, based on the captured image 50. In S116, the adjustment unit 160 adjusts the display of the complemented image 60 on the display unit based on the life-size size of the visual target 55 determined by the determination unit 158 ​​in S106, so that the display size of the visual target 55 included in the complemented image 60 generated by the image generation unit 164 in S114 more closely matches the visual size of the visual target 55 when viewed from a distance set by the setting unit 155 in S102. In S118, the glasses-type device 100 displays the complemented image 60 on the display unit in accordance with the adjustment by the adjustment unit 160 in S116. Thereafter, the process of the glasses-type device 100 adjusting the display of the image on the display unit ends.

[0137] 12 schematically illustrates an example of the hardware configuration of a computer 1200 functioning as the information processing device 150. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "units" of an apparatus according to the present embodiment, or can cause the computer 1200 to execute operations associated with the apparatus according to the present embodiment or one or more "units," and / or can cause the computer 1200 to execute a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0138] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0139] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0140] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from a DVD-ROM 1227 or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0141] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0142] The programs are provided by a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0143] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, the DVD-ROM 1227, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0144] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive 1226 (DVD-ROM 1227), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0145] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0146] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0147] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0148] A computer-readable medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that the computer-readable medium having instructions stored thereon comprises an article of manufacture containing instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable media may include electronic, magnetic, optical, electromagnetic, and semiconductor storage media. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, and the like.

[0149] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0150] The computer-readable instructions may be provided to a processor or programmable circuit of a programmable data processing device, such as a computer, locally or via a wide area network (WAN) such as a local area network (LAN) or the Internet, and the computer-readable instructions may be executed to create means for performing the operations specified in the flowcharts or block diagrams. Here, the computer may be a personal computer (PC), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, a special-purpose computer, or the like, or may be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system, and is a broad definition of computer. In a distributed computing system, the multiple computers collectively execute a program by each executing a portion of the program and passing data between the computers as needed during program execution.

[0151] Examples of processors include a computer processor, a central processing unit (CPU), a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, etc. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute the program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0152] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0153] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0154] 10 system, 20 network, 50 captured image, 55 visually recognized object, 57 missing part, 60 complemented image, 100 eyeglass-type device, 102 user, 105 eye, 110 frame, 112 rim, 114 bridge, 116 lens, 118 temple, 150 information processing device, 152 information storage unit, 154 acquisition unit, 155 setting unit, 156 identification unit, 158 determination unit, 160 adjustment unit, 162 judgment unit, 164 image generation unit, 166 learning data storage unit, 168 model generation unit, 170 model storage unit, 180 scanner, 190 camera, 200 image database, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphics controller, 1218 display device, 1220 input / output controller, 1222 Communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. An information processing device mounted on a glasses-type device having a display unit, an information storage unit for storing captured images; an identification unit that identifies a predetermined visual target of a user of the eyeglass-type device included in the captured image stored in the information storage unit; a determination unit that determines a life-size size of the visually recognized object identified by the identification unit; an adjustment unit that adjusts the display of the captured image on the display unit based on the life-size size of the visually recognized target determined by the determination unit so that the display size of the visually recognized target included in the captured image more closely matches the visually recognized size of the visually recognized target when the visually recognized target is visually recognized at a predetermined distance away; a determination unit that determines whether the captured image includes the entire visual target; Equipped with When the determination unit determines that the captured image includes the entire visual target, the adjustment unit adjusts the display of the captured image on the display unit so that the display size of the visual target included in the captured image more closely matches the visual size of the visual target when the visual target is visually recognized at a distance of the separation distance, within a range in which the entire visual target is displayed on the display unit. Information processing device.

2. 2. The information processing device according to claim 1, wherein when the adjustment unit adjusts the display of the captured image on the display unit so that the captured image is enlarged at a magnification higher than a predetermined magnification threshold, the adjustment unit performs a high-resolution process on the display portion of the captured image that is displayed on the display unit.

3. an image generating unit that generates, based on the captured image, a complemented image in which a missing portion of the visual target that is not included in the captured image is complemented when the determining unit determines that the captured image does not include the entire visual target; Furthermore, the adjustment unit adjusts the display of the complementary image on the display unit so that the display size of the visually recognized target included in the complementary image more closely matches the visually recognized size of the visually recognized target when the visually recognized target is visually recognized at a distance of the separation distance, within a range in which the entire visually recognized target is displayed on the display unit.

3. The information processing device according to claim 1 or 2.

4. 4. The information processing device according to claim 3, wherein the image generation unit generates the complemented image from the captured image stored in the information storage unit using a generative model that generates a complemented image in which a missing portion of the visual target is complemented from a captured image including a portion of the visual target.

5. The adjustment unit estimates the visual size of the visually recognized target when the visually recognized target is visually recognized at a distance of the separation distance using the following formula: S ｖ =(S ｒ ×f) / D Here, S ｖ is the visual size of the visually recognized object when the visually recognized object is viewed from a distance of the separation distance, and S ｒ is the life-size of the viewed object, f is the average focal length of the human eye, and D is the separation distance.

3. The information processing device according to claim 1 or 2.

6. a setting unit that sets a range of the separation distance that can be set by the user based on the life-size size of the visually recognized target so that the visually recognized size of the visually recognized target falls within a predetermined range of sizes; The information processing device according to claim 1 , further comprising:

7. The information processing device according to claim 1 , wherein the determination unit determines the life-size of the visually recognized target by estimating the life-size of the visually recognized target based on the captured image.

8. The information processing device according to claim 1 , wherein the determination unit determines the life-size of the visually identified target by estimating the life-size of the visually identified target based on scan data obtained by scanning the visually identified target.

9. the information storage unit further stores visually identified information for identifying the visually identified object and actual size information indicating the actual size of the visually identified object in association with each other; The determination unit determines the life-size size of the visually identified target by acquiring the life-size size information of the visually identified target stored in the information storage unit in association with the visually identified target identification information of the visually identified target identified by the identification unit.

3. The information processing device according to claim 1 or 2.

10. An information processing device mounted on an eyeglass-type device having a display unit, an information storage unit for storing captured images; an identification unit that identifies a predetermined visual target of a user of the eyeglass-type device included in the captured image stored in the information storage unit; a determination unit that determines a life-size size of the visually recognized object identified by the identification unit; an adjustment unit that adjusts the display of the captured image on the display unit based on the life-size size of the visually recognized target determined by the determination unit so that a display size of the visually recognized target included in the captured image matches a visually recognized size of the visually recognized target when the visually recognized target is viewed from a predetermined distance away; An information processing device comprising:

11. An eyeglass-type device equipped with the information processing device according to claim 1 .

12. A program that, when executed by a computer, causes the computer to function as the information processing device according to any one of claims 1, 2, and 10.

Images