Information processing device, eyeglass-type device, and program

Abstract

An information processing device is provided that is mounted on a spectacle-type device having a display unit. [Solution] An information processing device is provided, comprising: an information storage unit for storing an image; an identification unit for identifying a predetermined object to be viewed by the user of the glasses-type device, which is included in the image stored in the information storage unit; a determination unit for determining the actual size of the object to be viewed, which is identified by the identification unit; and an adjustment unit for adjusting the display of the image on the display unit, based on the actual size of the object to be viewed, which is determined by the determination unit, so that the display size of the object to be viewed included in the image more closely matches the viewing size of the object to be viewed when the object is viewed from a predetermined distance apart.

Description

[Technical Field]

[0001] This invention relates to an information processing device, a spectacle-type device, and a program. [Background technology]

[0002] Patent Document 1 describes a head-mounted display system that can suppress a decrease in user convenience even when changing the size of an image. [Prior art document] [Patent] [Patent Document 1] Japanese Unexamined Patent Publication No. 2022-154409 [Overview of the project] [Means for solving the problem]

[0003] According to one embodiment of the present invention, an information processing device is provided that is mounted on a spectacle-type device having a display unit. The information processing device may include an information storage unit for storing an image. The information processing device may include a specification unit for identifying a predetermined object to be viewed by the user of the spectacle-type device, which is included in the image stored in the information storage unit. The information processing device may include a determination unit for determining the actual size of the object to be viewed, which is identified by the specification unit. The information processing device may include an adjustment unit for adjusting the display of the image on the display unit, based on the actual size of the object to be viewed, which is determined by the determination unit, so that the display size of the object to be viewed included in the image better matches the viewing size of the object when the object is viewed from a predetermined distance apart.

[0004] In the aforementioned information processing device, if the adjustment unit adjusts the display of the captured image on the display unit to enlarge the captured image at a magnification higher than a predetermined magnification threshold, it may perform a high-resolution processing on the display portion of the captured image that is displayed on the display unit.

[0005] Any of the above-mentioned information processing devices may further include a determination unit for determining whether the captured image includes the entire object to be viewed, and if the determination unit determines that the captured image includes the entire object to be viewed, the adjustment unit may adjust the display of the captured image on the display unit so that, within the range in which the entire object to be viewed is displayed on the display unit, the display size of the object to be viewed included in the captured image better matches the view size of the object to be viewed when the object to be viewed from a distance equal to the separation distance.

[0006] Any of the above-mentioned information processing devices may further include an image generation unit that, if the determination unit determines that the captured image does not include the entire object to be viewed, generates a supplemental image based on the captured image in which the missing portion of the object to be viewed that is not included in the captured image is supplemented, and the adjustment unit may adjust the display of the supplemental image on the display unit so that, within the range in which the entire object to be viewed is displayed on the display unit, the display size of the object to be viewed included in the supplemental image more closely matches the viewing size of the object to be viewed when the object to be viewed from the distance of the separation.

[0007] In any of the above-mentioned information processing devices, the image generation unit may generate the complementary image from the captured image stored in the information storage unit using a generation model that generates a complementary image in which the missing portion of the object being viewed is filled in from an captured image that includes a part of the object being viewed.

[0008] Any of the above-mentioned information processing devices may further include: a learning data storage unit that stores learning data including an image of the entire object being viewed; a model generation unit that uses a plurality of the learning data stored in the learning data storage unit as training data to generate a generation model by machine learning that generates a complementary image in which missing parts of the object being viewed are filled in from an image of the object being viewed that includes a part of the object being viewed; and an image generation unit that uses the generation model to generate the complementary image from the image of the object being viewed that is stored in the information storage unit.

[0009] In any of the above-mentioned reporting devices, the adjustment unit is S v =( S r The visible size of the object when it is visible from a distance equal to the separation distance can be estimated using ×f) / D, where S v This may be the visible size of the object when the object is visible from a distance equal to the aforementioned separation distance, S r f may be the actual size of the object to be viewed, f may be the average focal length of the human eye, and D may be the separation distance.

[0010] Any of the above-mentioned information processing devices may further include a setting unit that sets a user-configurable range of separation distances, based on the actual size of the object to be viewed, so that the viewing size of the object to be viewed becomes within a predetermined range.

[0011] In any of the above-mentioned information processing devices, the determination unit may determine the actual size of the object to be viewed by estimating the actual size of the object to be viewed based on the captured image.

[0012] In any of the above-mentioned information processing devices, the determination unit may determine the actual size of the object to be viewed by estimating the actual size of the object to be viewed based on scan data obtained by scanning the object to be viewed.

[0013] In any of the above-mentioned information processing devices, the information storage unit may further store, in association with, object identification information that identifies the object to be viewed and actual-size information indicating the actual-size of the object to be viewed, and the determination unit may determine the actual-size of the object to be viewed by obtaining the actual-size information of the object to be viewed, which is stored in the information storage unit in association with the object identification information of the object to be viewed, which has been identified by the identification unit.

[0014] According to one embodiment of the present invention, a glasses-type device equipped with any of the above-mentioned information processing devices is provided.

[0015] According to one embodiment of the present invention, when executed by a computer, a program is provided for causing the computer to function as any of the information processing apparatuses.

[0016] Note that the above summary of the invention does not list all of the necessary features of the present invention. Also, sub-combinations of these feature groups can also be inventions.

Brief Description of the Drawings

[0017] [Figure 1] An example of the system 10 is schematically shown. [Figure 2] It is an explanatory diagram for explaining an example of the relationship between the separation distance between the user 102 and the viewing target 55 of the user 102 and the viewing size of the viewing target 55 by the user 102. [Figure 3] It is an explanatory diagram for explaining an example of displaying an image on the glasses-type device 100. [Figure 4] An example of the configuration of the glasses-type device 100 is schematically shown. [Figure 5] It is an explanatory diagram for explaining an example of estimating the viewing size of the viewing target 55. [Figure 6] It is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. [Figure 7] It is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. [Figure 8] It is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. [Figure 9] An example of the functional configuration of the information processing apparatus 150 is schematically shown. [Figure 10] An example of the information stored in the information processing apparatus 150 is shown. [Figure 11] It is an explanatory diagram for explaining an example of the processing flow of the glasses-type device 100. [Figure 12]A schematic example of the hardware configuration of a computer 1200 that functions as an information processing device 150 is shown. [Modes for carrying out the invention]

[0018] When captured images are displayed in AR (Augmented Reality) glasses, all captured images will be displayed at the same size unless the user of the AR glasses manually changes the display size of the captured images. Therefore, in order for subjects such as people and buildings in the captured images to be displayed in the AR glasses as if they were directly seen by the user, the user must manually set the display size of each captured image to be displayed in the AR glasses. The system according to this embodiment employs a mechanism that, for example, uses AI (Artificial Intelligence) technology to estimate the actual size of the main subject in the captured image and displays the captured image on the AR glasses, which has been automatically adjusted to more closely match the perceived size of the subject when it is directly seen by the user.

[0019] The present invention will be described below through embodiments, but these embodiments are not intended to limit the invention as defined in the claims. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention. In addition, in the drawings, identical or similar parts may be given the same reference numeral to omit redundant descriptions.

[0020] Figure 1 schematically shows an example of system 10. System 10 may include a glasses-type device 100. System 10 may include an image database 200.

[0021] The glasses-type device 100 can be any device that has a display function to display content on a display unit mounted on the device. The glasses-type device 100 is, for example, AR glasses.

[0022] The content may be in any data format. For example, the content may be a still image. For example, the content may be a video. For example, the content may be text. For example, the content may be an illustration. For example, the content may be 3D computer graphics (CG) data. The content may also include a combination of these.

[0023] The glasses-type device 100 may be a device capable of giving the user 102 of the glasses-type device 100 the sensation that the content is placed in real space by displaying the content on transparent or semi-transparent glasses. In other words, the glasses-type device 100 may be a so-called see-through device. The glasses-type device 100 may also be a video pass-through device. In this case, the glasses-type device 100 may display captured images that capture the range corresponding to the field of view of the user 102 of the glasses-type device 100 on an opaque display and superimpose the content.

[0024] The glasses-type device 100 has, for example, an access function to access the network 20. The glasses-type device 100 is, for example, smart glasses.

[0025] Network 20 may include a cloud. Network 20 may include a mobile communication network. The mobile communication network may comply with any of the following communication methods: 3G (3rd Generation), LTE (Long Term Evolution), 5G (5th Generation), 6G (6th Generation), or later. Network 20 may comply with short-range wireless communication methods such as Wi-Fi®, Bluetooth®, and Zigbee®. Network 20 may also include wired connections.

[0026] The glasses-type device 100 communicates, for example, with an image database 200 that stores captured images. The glasses-type device 100 communicates with the image database 200, for example, via a network 20.

[0027] The captured image may be, for example, a still image. The captured image may also be, for example, a moving image. The captured image may be an example of content.

[0028] The glasses-type device 100 receives captured images from, for example, the image database 200. The captured images are, for example, images captured by the user 102 of the glasses-type device 100 using a camera. The captured images may also be images captured by a user other than the user 102 of the glasses-type device 100 using a camera. The glasses-type device 100 may display the captured images received from the image database 200 on its own display unit. Details on how the glasses-type device 100 displays captured images on its own display unit will be described later.

[0029] Figure 2 is an explanatory diagram illustrating an example of the relationship between the distance between user 102 and the object 55 viewed by user 102 and the size of the object 55 as viewed by user 102. Figure 2 shows an example in which user 102 views the object 55 while wearing the glasses-type device 100.

[0030] The object of observation 55 can be any object. For example, the object could be a human being. It could be an animal other than a human being. It could be a building such as a house or office building. It could also be a natural object such as a tree or a rock. Figure 2 shows an example where the object of observation 55 is a human being.

[0031] As shown in Figure 2, as the distance between user 102 and the object 55 increases from 1m, 5m, to 10m, the perceived size of the object 55 by user 102 decreases. The perceived size may refer to the size of the object 55 as it appears to the user 102's eye when the user 102 views it.

[0032] Figure 3 is an explanatory diagram illustrating an example of displaying an image on the glasses-type device 100. Here, we will mainly explain an example in which the glasses-type device 100 displays the captured image 50 on its own display unit.

[0033] For example, the glasses-type device 100 identifies the object 55 that user 102 is viewing, which is included in the captured image 50. Next, the glasses-type device 100 determines the actual size of the object 55. The actual size of the object 55 may refer to the actual size of the object 55. For example, if the height of the object 55 is 170 cm, the actual size of the object 55 is 170 cm.

[0034] Subsequently, the glasses-type device 100 adjusts the display of the captured image 50 on its display unit based on the actual size of the object to be viewed 55. For example, the glasses-type device 100 adjusts the display of the captured image 50 on its display unit so that the displayed size of the object to be viewed 55 included in the captured image 50 more closely matches the displayed size of the object to be viewed 55 when the object to be viewed 55 is viewed from a predetermined distance apart. For example, if the distance apart is 1m, the glasses-type device 100 adjusts the display of the captured image 50 on its display unit so that the size of the object to be viewed 55 as seen by the user 102 when the user 102 views the object to be viewed 55 included in the captured image 50 displayed on its display unit more closely matches the size of the object to be viewed 55 as seen by the user 102 when the user 102 views the object to be viewed 55 from a distance of 1m. Note that the display size may refer to the size displayed on the display unit of the glasses-type device 100.

[0035] The upper part of Figure 3 shows the captured image 50 displayed on the display unit before the display of the captured image 50 is adjusted. The lower part of Figure 3 shows the captured image 50 displayed on the display unit after the display of the captured image 50 is adjusted. As shown in Figure 3, the glasses-type device 100 adjusts the display of the captured image 50 on the display unit by enlarging the display size of the object to be viewed 55 so that the display size of the object to be viewed 55 better matches the viewable size of the object to be viewed 55 when the object to be viewed 55 is viewed from the distance of that separation.

[0036] Since the introduction of glasses-type devices such as AR glasses and smart glasses to the market, the demand for glasses-type devices has been increasing, and this trend is expected to continue. One example of the use of glasses-type devices is viewing captured images. When viewing captured images using glasses-type devices, there is a need to view multiple captured images while focusing on a specific subject that may be included in those images. For example, a parent viewing multiple captured images in which the child may be the subject may want to view those images while focusing on the child that may be included in those images. However, when viewing captured images using conventional glasses-type devices, unless the user manually adjusts the display size of the captured images, the glasses-type device displays all captured images at the same size on its display. Therefore, if the user does not manually adjust the display size of the captured images, the display size of a specific subject that may be included in the captured images displayed on the glasses-type device's display changes depending on the imaging distance from the camera that captured the image to the specific subject. Therefore, when a specific subject included in one captured image is displayed on the display unit of the glasses-type device, that subject may be displayed at a large size, while when a specific subject included in another captured image is displayed on the display unit of the glasses-type device, that subject may be displayed at a small size. This means that it is difficult to view multiple captured images while focusing on a specific subject that may be included in multiple captured images. Consequently, when viewing multiple captured images while focusing on a specific subject that may be included in multiple captured images using a conventional glasses-type device, the user of the glasses-type device must manually adjust the display size of the captured images displayed on the display unit of the glasses-type device in order to easily focus on the specific subject, which places a heavy burden on the user of the glasses-type device. For these reasons, it is desirable to reduce the burden on the user of the glasses-type device when viewing multiple captured images while focusing on a specific subject that may be included in multiple captured images using the glasses-type device.

[0037] In contrast, according to the system 10 of this embodiment, when the glasses-type device 100 displays the captured image 50, the glasses-type device 100 identifies the object 55 that the user 102 of the glasses-type device 100 will view, and determines the actual size of the identified object 55. Then, based on the determined actual size, the glasses-type device 100 adjusts the display on the display unit of the captured image 50 so that the display size of the object 55 included in the captured image 50 more closely matches the viewing size of the object 55 when viewed from a predetermined distance apart. Therefore, the glasses-type device 100 can automatically adjust the display on the display unit of the captured image 50 so that the user 102 can easily focus on a specific object 55 that may be included in multiple captured images 50, without the user 102 having to manually adjust the display on the display unit of the captured image 50. As a result, the system 10 according to this embodiment can reduce the burden on the user of the glasses-type device when viewing multiple captured images while focusing on a specific subject that may be included in the multiple captured images. Furthermore, according to the system 10 according to this embodiment, a user 102 who views multiple captured images 50 of a viewing object 55 taken at different times using the glasses-type device 100 can recognize the temporal changes of the viewing object 55 not only by changes in the appearance of the viewing object 55, but also by changes in the display size of the viewing object 55. For example, a parent of a child who views multiple captured images 50 of the child taken at different times using the glasses-type device 100 can recognize the child's growth not only by changes in the child's facial expressions and clothing, but also by changes in the display size of the child. As a result, the system 10 according to this embodiment makes it easier for a user who views multiple captured images of a viewing object taken at different times using the glasses-type device to intuitively recognize the temporal changes of the viewing object. As a result, the system 10 according to this embodiment can contribute to the widespread adoption of eyeglasses by satisfying the need for eyeglasses to view multiple captured images while focusing on a specific subject that may be included in those images.

[0038] Figure 4 schematically shows an example of the configuration of the eyeglass-type device 100. The eyeglass-type device 100 may include a frame 110 having a rim 112, a bridge 114, and temples 118.

[0039] The eyeglass-type device 100 includes, for example, a lens 116. The lens 116 may be an example of a display unit of the eyeglass-type device 100.

[0040] The lens 116 may be a transparent or translucent display. In this case, the eyeglass-type device 100 may be a transparent or translucent display-type device. The eyeglass-type device 100 may also have a projection function that projects a display onto the lens 116. In this case, the eyeglass-type device 100 may be a projection-type device. The projection function of the eyeglass-type device 100 may be an example of the display function of the eyeglass-type device 100.

[0041] The glasses-type device 100 includes, for example, a scanner 180. The scanner 180 may scan objects. The scanner 180 may have, for example, an imaging function. In this case, the scanner 180 includes a camera. The scanner 180 may have, for example, a distance measuring function. In this case, the scanner 180 includes a distance measuring sensor such as LiDAR (Light Detection and Ranging). The scanner 180 may include both a camera and a distance measuring sensor.

[0042] The glasses-type device 100 includes, for example, a camera 190. Figure 4 shows an example where the glasses-type device 100 has one camera 190. The glasses-type device 100 may have multiple cameras 190. If the scanner 180 has an imaging function, the scanner 180 and the camera 190 may be an integrated device or separate devices.

[0043] The imaging range of camera 190 is, for example, the range corresponding to the user 102's field of view. The imaging range of camera 190 may be wider than the user 102's field of view, or it may be substantially the same as the user 102's field of view.

[0044] The arrangement of the scanner 180 and camera 190 in Figure 4 is an example. The scanner 180 and camera 190 may be placed at any other location in the frame 110.

[0045] The glasses-type device 100 includes, for example, an operating section. The glasses-type device 100 receives various inputs via the operating section. The operating section may be located at any position on the glasses-type device 100.

[0046] The glasses-type device 100 includes, for example, an input section such as a button. The glasses-type device 100 receives various inputs via the input section. The input section may be placed at any position on the frame 110.

[0047] The glasses-type device 100 includes, for example, an information processing device 150. The information processing device 150 may perform various information processing operations.

[0048] The information processing device 150 performs, for example, communication processing to communicate with an external device. The information processing device 150 performs, for example, display processing to display content on the display unit of the glasses-type device 100. The information processing device 150 performs, for example, identification processing to identify the object 55 viewed by the user 102 included in the captured image 50. The information processing device 150 performs, for example, determination processing to determine the actual size of the object 55 viewed. The information processing device 150 performs, for example, setting processing to set a separation distance. The information processing device 150 performs, for example, adjustment processing to adjust the display of the captured image 50 on the display unit of the glasses-type device 100.

[0049] The arrangement of the information processing device 150 in Figure 4 is an example. The information processing device 150 may be placed in other locations on the frame 110. The information processing device 150 may be placed in any location inside the frame 110, for example. For example, the information processing device 150 may be placed inside the temple 118. Specifically, the information processing device 150 may be placed at the tip of the temple 118, i.e., the temple. The information processing device 150 may also be placed in any location outside the frame 110.

[0050] The glasses-type device 100 includes, for example, a battery. The information processing device 150 may perform various information processing using the power stored in the battery.

[0051] The glasses-type device 100 may communicate with an external device. For example, the glasses-type device 100 may communicate wirelessly with an external device. For example, the glasses-type device 100 may communicate wirelessly with an external device. For example, the glasses-type device 100 may communicate with an external device via mobile communication. The glasses-type device 100 may also communicate with an external device via wired communication.

[0052] The glasses-type device 100 communicates with, for example, an image database 200. The glasses-type device 100 may also communicate with a communication terminal such as a smartphone or other mobile phone, a tablet device, a wearable device, or a PC (Personal Computer). The glasses-type device 100 may accept various inputs by receiving input information from the communication terminal.

[0053] Figure 5 is an explanatory diagram illustrating an example of estimating the viewing size of the viewing object 55. Here, we will first explain the various parameters shown in Figure 5, and then describe an example of estimating the viewing size of the viewing object 55.

[0054] S r is the actual size of the object 55 being viewed. f is the average focal length of the human eye 105. D is the distance at which the object 55 is viewed. S vis the visual size of the visual target 55 when the visual target 55 is visually recognized only at a distance D.

[0055] The glasses-type device 100, for example, S r , f, and D, estimates S v The glasses-type device 100, for example, S v = (S r × f) / D to estimate S v .

[0056] FIG. 6 is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. Here, the points different from the above example of displaying an image on the glasses-type device 100 will be mainly explained.

[0057] The upper diagram of FIG. 6 is the captured image 50 displayed on the display unit of the glasses-type device 100 before the display of the captured image 50 on the display unit is adjusted. The lower diagram of FIG. 6 is the captured image 50 displayed on the display unit after the display of the captured image 50 on the display unit is adjusted. As shown in FIG. 6, when the visual target 55 is enlarged and displayed on the display unit at a high magnification by adjusting the display of the captured image 50 on the display unit, the glasses-type device 100 may display only a part of the captured image 50 that includes the visual target 55 on the display unit.

[0058] FIG. 7 is an explanatory diagram for explaining another example of displaying an image on the glasses-type device 100. Here, the points different from the above example of displaying an image on the glasses-type device 100 will be mainly explained.

[0059] The upper diagram of FIG. 7 schematically shows an example of the captured image 50. As shown in the upper diagram of FIG. 7, the captured image 50 does not include the entire visual target 55. In some cases, the part of the visual target 55 not included in the captured image 50 may be described as the missing part of the visual target 55.

[0060] The middle diagram of FIG. 7 is an explanatory diagram for explaining an example of complementing the missing part 57 of the visual target 55. Here, an example of the case where the glasses-type device 100 complements the missing part 57 of the visual target 55 will be mainly explained.

[0061] The glasses-type device 100 generates a complementary image 60 in which, for example, the missing portion 57 of the object to be viewed 55 is filled in. The glasses-type device 100 generates the complementary image 60 from the captured image 50 using, for example, a generation model that generates a complementary image in which the missing portion of the captured image is filled in from an captured image that includes a part of the object to be viewed.

[0062] Subsequently, the glasses-type device 100 may adjust the display of the generated interpolated image 60 on the display unit of the glasses-type device 100. The glasses-type device 100 may adjust the display of the interpolated image 60 on the display unit in the same manner as when adjusting the display of the captured image 50 on the display unit.

[0063] The lower part of Figure 7 shows the complementary image 60 displayed on the display unit after the display of the complementary image 60 has been adjusted. As shown in the lower part of Figure 7, the glasses-type device 100 displays the complementary image 60 on its display unit so that the display size of the object to be viewed 55 more closely matches the viewing size of the object to be viewed 55 when the object to be viewed 55 is viewed from a predetermined distance apart.

[0064] According to the system 10 shown in Figure 7, the glasses-type device 100 generates a complementary image 60 in which the missing portion 57 of the object to be viewed 55 is filled in, if such a portion exists. Subsequently, the glasses-type device 100 adjusts the display of the generated complementary image 60 on the display unit of the glasses-type device 100. As a result, the system 10 shown in Figure 7 enables the user of the glasses-type device to view an image with missing portions of the object to be viewed with the same sensation as when viewing an image in which the entire object to be viewed is captured.

[0065] Figure 8 is an explanatory diagram illustrating another example of displaying an image on the glasses-type device 100. Figure 8 shows an example where the object 55 viewed by the user 102 is a building. Here, we will mainly explain the differences from the aforementioned example of displaying an image on the glasses-type device 100.

[0066] The glasses-type device 100 may limit the range of separation distances that the user 102 can set if the object to be viewed 55 is a large, life-size object such as a building as shown in Figure 8. The glasses-type device 100 may limit the separation distances that the user 102 can set, for example, by setting a lower limit for the separation distances that the user 102 can set. The glasses-type device 100 may further limit the separation distances that the user 102 can set by setting an upper limit for the separation distances that the user 102 can set. The glasses-type device 100 may also limit the separation distances that the user 102 can set by setting only an upper limit for the separation distance.

[0067] The upper part of Figure 8 shows the captured image 50 displayed on the display unit of the glasses-type device 100 before the display of the captured image 50 is adjusted. The lower part of Figure 8 shows the captured image 50 displayed on the display unit after the display of the captured image 50 has been adjusted. As shown in Figure 8, the glasses-type device 100 limits the range of separation distance that can be set by the user 102, and adjusts the display of the captured image 50 on the display unit so that the display size of the object to be viewed 55 is more consistent with the viewing size of the object to be viewed 55 when the object to be viewed 55 is viewed from that separation distance.

[0068] When an object that is too large in size is displayed on the display of a glasses-type device, it can become difficult for the user to see the object, or the image quality of the object may deteriorate. On the other hand, when an object that is too small in size is displayed on the display of a glasses-type device, it can become difficult for the user 102 to see the object. Therefore, displaying an object that is too large or too small in size on the display of a glasses-type device can be one of the causes of a decrease in the user experience of the glasses-type device.

[0069] In contrast, according to the system 10 shown in Figure 8, the glasses-type device 100 limits the range of distance that the user 102 can set based on the actual size of the object to be viewed 55. As a result, the system 10 shown in Figure 8 can appropriately limit the range of distance that the user can set while taking into account the actual size of the object to be viewed by the user, thereby preventing objects that are too large or too small from being displayed on the display of the glasses-type device, and consequently improving the user experience of the glasses-type device.

[0070] Figure 9 schematically shows an example of the functional configuration of the information processing device 150. The information processing device 150 comprises an information storage unit 152, an acquisition unit 154, a setting unit 155, an identification unit 156, a determination unit 158, an adjustment unit 160, a judgment unit 162, an image generation unit 164, a learning data storage unit 166, a model generation unit 168, and a model storage unit 170. However, it is not necessarily required that the information processing device 150 comprises all of these components.

[0071] The information storage unit 152 stores various types of information. For example, the information storage unit 152 stores the captured image 50. For example, the information storage unit 152 stores the metadata of the captured image 50. The metadata of the captured image 50 includes, for example, capture date and time data indicating the date and time the captured image 50 was taken. The captured image 50 includes, for example, resolution data indicating the resolution of the captured image 50. The metadata of the captured image 50 may also include other arbitrary data related to the captured image 50. For example, the information storage unit 152 stores the captured image 50 and the metadata of the captured image 50 in association.

[0072] The information storage unit 152 stores, for example, object identification information that identifies the object 55 viewed by the user 102 of the glasses-type device 100. The object identification information is, for example, an identifier that identifies the object 55. The information storage unit 152 stores, for example, the object identification information in association with various information related to the object 55.

[0073] The information storage unit 152 stores, for example, attribute information indicating the attributes of the object being viewed 55. The attribute information of the object being viewed 55 includes, for example, classification information indicating the classification of the object. Examples of object classifications include humans, animals, buildings, natural objects, etc. The attribute information of the object being viewed 55 includes, for example, gender information indicating the gender of the object. The attribute information of the object being viewed 55 includes, for example, age information indicating the age of the object. The information storage unit 152 stores, for example, the identification information of the object being viewed and the attribute information indicating the attributes of the object being viewed 55 in association.

[0074] The information storage unit 152 stores, for example, actual-size information indicating the actual size of the object to be viewed 55. The information storage unit 152 stores, for example, the object identification information and the actual-size information of the object to be viewed 55 in association with each other.

[0075] For example, if the object to be observed 55 is a human or an animal, the actual size information for the object to be observed 55 may show the actual size of the object to be observed 55 at different ages. Examples of age intervals include 6 months, 1 year, 3 years, 5 years, 10 years, etc. For example, if the object to be observed 55 is a 2-year-old child and the age interval is 1 year, the actual size information for the child may show the actual size of the child at 0 years old, the actual size of the child at 1 year old, and the actual size of the child at 2 years old.

[0076] The information storage unit 152 stores, for example, image data of the object to be viewed 55. The image data of the object to be viewed 55 may be, for example, image data that includes the entire object to be viewed 55. The image data of the object to be viewed 55 may also be image data that includes only characteristic parts of the object to be viewed 55. The information storage unit 152 stores, for example, object identification information and image data of the object to be viewed 55 in association.

[0077] For example, if the object being observed 55 is a human or an animal, the image data of the object being observed 55 may include image data of the object being observed 55 at different ages. For example, if the object being observed 55 is a 2-year-old child and the age interval is 1 year, the image data of the child may include image data of the child at 0 years old, image data of the child at 1 year old, and image data of the child at 2 years old.

[0078] The information storage unit 152 stores, for example, average focal length information, which indicates the average focal length of the human eye. The information storage unit 152 also stores, for example, user focal length information, which indicates the focal length of the user 102's eye.

[0079] The acquisition unit 154 acquires various types of information. The acquisition unit 154 acquires various types of information, for example, by receiving various types of information via the network 20. The acquisition unit 154 acquires various types of information when the operation unit or input unit of the glasses-type device 100 accepts various types of input. The acquisition unit 154 may store the acquired information in the information storage unit 152.

[0080] The acquisition unit 154 acquires, for example, an image. The acquisition unit 154 acquires an image by, for example, receiving an image from an image database 200. The acquisition unit 154 may also acquire an image by capturing an image using a camera 190 mounted on the glasses-type device 100.

[0081] The acquisition unit 154 acquires, for example, scan data. The acquisition unit 154 acquires, for example, scan data obtained by scanning an object. The acquisition unit 154 acquires, for example, scan data obtained by scanning a visual target 55.

[0082] The acquisition unit 154 acquires scan data scanned by, for example, a scanner 180 mounted on the glasses-type device 100. The acquisition unit 154 may also acquire scan data scanned by a scanner not mounted on the glasses-type device 100. In this case, the acquisition unit 154 may acquire the scan data by receiving it from the scanner via the network 20.

[0083] The acquisition unit 154 acquires, for example, various models. The acquisition unit 154 may store the acquired models in the model storage unit 170.

[0084] The acquisition unit 154 acquires, for example, a specific model. The specific model may be a model that identifies a visually perceptible object included in the captured image. The specific model may be an AI model.

[0085] The acquisition unit 154 acquires, for example, an estimation model. The estimation model may be a model that estimates the actual size of the object visible in the captured image from the captured image. The estimation model may be an AI model.

[0086] The acquisition unit 154 acquires, for example, a high-resolution processing model. The high-resolution processing model is a model that performs high-resolution processing on a display portion when input data includes, for example, an captured image, the resolution data of the captured image, the magnification data indicating the magnification of the captured image, and the display portion data indicating the display portion of the captured image that is displayed on the display unit. The high-resolution processing model may be an AI model.

[0087] The acquisition unit 154 acquires, for example, a generation model. The generation model is a model that generates a complementary image from an image containing a part of the object being viewed, in which the missing part of the object being viewed is filled in. The generation model may be an AI model.

[0088] The setting unit 155 performs various setting processes. The setting unit 155 performs various setting processes based on, for example, various information stored in the information storage unit 152. The setting unit 155 performs various setting processes based on, for example, input information received by the operation unit or input unit of the glasses-type device 100. The setting unit 155 may also perform various setting processes based on input information received from a communication terminal via the network 20.

[0089] The setting unit 155 sets, for example, the object 55 to be viewed by the user 102 of the glasses-type device 100. The setting unit 155 sets, for example, the attributes of the object 55. The setting unit 155 sets, for example, the actual size of the object 55. The setting unit 155 sets, for example, the separation distance used to adjust the display of the captured image 50 including the object 55 on the display unit of the glasses-type device 100.

[0090] The setting unit 155 sets, for example, the range of separation distances that the user 102 can set. The information storage unit 152 sets, for example, the range of separation distances that the user 102 can set based on the actual size of the object to be viewed 55.

[0091] The setting unit 155 sets, for example, a range of separation distances that the user 102 can set, such that the visible size of the visible object 55 is within a predetermined range. The setting unit 155 sets, for example, a range of separation distances that the user 102 can set, by setting a lower limit of the separation distance that the user 102 can set. The setting unit 155 sets, for example, a lower limit of the separation distance if the actual size of the visible object 55, as indicated by the actual size information, is greater than a predetermined first size threshold. The setting unit 155 sets, for example, a range of separation distances that the user 102 can set, by setting an upper limit of the separation distance that the user 102 can set. The setting unit 155 sets, for example, a range of separation distances that the user 102 can set, by setting both a lower limit and an upper limit of the separation distance that the user 102 can set.

[0092] The setting unit 155 sets, for example, the lower limit of the separation distance that the user 102 can set to the actual size of the object to be observed 55 × α. α may be a constant. For example, α is 0.5. For example, if the actual size of the object to be observed 55 is 10m and α is 0.5, the lower limit of the separation distance that the user 102 can set is 10m × 0.5 = 5m.

[0093] The setting unit 155 sets, for example, the upper limit of the separation distance that the user 102 can set to is the actual size of the object to be observed 55 × β. β may be a constant. For example, β is 5. For example, if the actual size of the object to be observed 55 is 10m and β is 5, the upper limit of the separation distance that the user 102 can set is 10m × 5 = 50m.

[0094] The identification unit 156 identifies the predetermined objects 55 that the user 102 of the glasses-type device 100 will view, which are included in the captured image 50 stored in the information storage unit 152. The objects 55 may be set by the setting unit 155.

[0095] The identification unit 156 identifies the object to be seen 55 included in the captured image 50 by, for example, performing image analysis on the captured image 50. The identification unit 156 identifies the object to be seen included in the captured image 50 from the captured image 50 using, for example, a specific model stored in the model storage unit 170.

[0096] For example, if the captured image 50 contains multiple objects classified in the same category as the object to be seen 55, the identification unit 156 identifies the object to be seen 55 from among the multiple objects based on the image data of the object to be seen 55 stored in the information storage unit 152 in association with the object to be seen identification information. For example, the identification unit 156 identifies the object to be seen 55 from among the multiple objects by comparing the captured image 50 with the image data of the object to be seen 55.

[0097] If the object to be observed 55 is a human or an animal, and the image data of the object to be observed 55 stored in the information storage unit 152 includes image data of the object to be observed 55 for each period of time, the identification unit 156 may further identify the object to be observed 55 from among the multiple objects based on the metadata of the captured image 50 stored in the information storage unit 152. For example, the identification unit 156 identifies the period of the object to be observed 55 when the captured image 50 was taken, based on the capture date and time data included in the metadata of the captured image 50, and identifies the object to be observed 55 from among the multiple objects by comparing the captured image 50 with the image data of the object to be observed 55 for the identified period of time.

[0098] If the plurality of objects are included in the captured image 50, the identification unit 156 may identify the object to be viewed 55 from among the plurality of objects based on information for identifying the object to be viewed 55 from among the plurality of objects. The information for identifying the object to be viewed 55 from among the plurality of objects may be obtained by the operation unit or input unit of the glasses-type device 100 receiving various inputs.

[0099] The determination unit 158 ​​determines the actual size of the object to be observed 55 identified by the identification unit 156. The determination unit 158 ​​determines the actual size of the object to be observed 55 based on various information stored in the information storage unit 152, for example. If the determination unit 158 ​​has determined the actual size of the object to be observed 55, it may update the actual size information of the object to be observed 55 stored in the information storage unit 152.

[0100] The determination unit 158 ​​determines the actual size of the object to be seen 55 by, for example, estimating the actual size of the object to be seen 55 based on the captured image 50. The determination unit 158 ​​estimates the actual size of the object to be seen 55 by, for example, performing image analysis on the captured image 50. The determination unit 158 ​​estimates the actual size of the object to be seen 55 contained in the captured image 50 from the captured image 50 using, for example, an estimation model stored in the model storage unit 170.

[0101] The determination unit 158 ​​may determine the actual size of the object to be seen 55 by estimating the actual size of the object to be seen 55 based on scan data of the object to be seen 55 stored in the information storage unit 152. Alternatively, the determination unit 158 ​​may determine the actual size of the object to be seen 55 by obtaining the actual size information of the object to be seen 55, which is stored in the information storage unit 152 and associated with the object identification information of the object to be seen 55.

[0102] The adjustment unit 160 adjusts the display of the image on the display unit of the glasses-type device 100. The glasses-type device 100 may display the image on the display unit according to the adjustments made by the adjustment unit 160.

[0103] The image is, for example, the captured image 50. The image may also be a complementary image 60.

[0104] The adjustment unit 160 adjusts, for example, the display of the captured image 50 on the display unit. The adjustment unit 160 adjusts the display of the captured image 50 on the display unit based, for example, the actual size of the object to be observed 55 determined by the determination unit 158. If the object to be observed 55 is a person or an animal, and the actual size information of the object to be observed 55 stored in the information storage unit 152 indicates the actual size of the object to be observed 55 for each era, the adjustment unit 160 may further adjust the display of the captured image 50 on the display unit based on the metadata of the captured image 50 stored in the information storage unit 152. For example, the adjustment unit 160 identifies the era of the object to be observed 55 when the captured image 50 was taken, based on the capture date and time data included in the metadata of the captured image 50, and adjusts the display of the captured image 50 on the display unit based on the actual size of the object to be observed 55 for the identified era.

[0105] The adjustment unit 160 adjusts the display of the captured image 50 on the display unit so that the display size of the object to be viewed 55 included in the captured image 50 more closely matches the display size of the object to be viewed 55 when it is viewed from a predetermined separation distance. The separation distance may be set by the setting unit 155. If the separation distance is not set by the setting unit 155, the separation distance may be the default value registered in the information processing device 150 at the time of manufacture.

[0106] The adjustment unit 160 adjusts, for example, the visible size S of the visible object 55 when the visible object 55 is visible from a distance D. v The display of the captured image 50 on the display unit is adjusted by estimating the average focal length f of the human eye, indicated by the average focal length information stored in the information storage unit 152, and the actual size S of the object being viewed 55. r Based on that, S v The adjustment unit 160 estimates, for example, S v =( S r Using ×f) / D, S v The adjustment unit 160 estimates the focal length f of the user 102's eye, as indicated by the user focal length information stored in the information storage unit 152. u And, life-size S of the 55 visible objects. r Based on that, S v It may be estimated that... The adjustment unit 160 is, for example, S v =( S r ×f u Using ) / D, S v We estimate this.

[0107] The adjustment unit 160 adjusts the display of the captured image 50 on the display unit by, for example, adjusting the display position in which the object to be seen 55 included in the captured image 50 is displayed on the display unit. The adjustment unit 160 adjusts the display position in which the object to be seen 55 is displayed on the display unit so that, for example, the object to be seen 55 is displayed in the central part of the display unit.

[0108] The adjustment unit 160 adjusts the display of the captured image 50 on the display unit by, for example, performing a high-resolution processing on the captured image 50. The adjustment unit 160 performs a high-resolution processing on, for example, the display portion of the captured image 50 that is displayed on the display unit. For example, when the adjustment unit 160 adjusts the display of the captured image 50 on the display unit so that the captured image 50 is enlarged to a higher magnification threshold than predetermined, it performs a high-resolution processing on the display portion of the captured image 50 that is displayed on the display unit.

[0109] The adjustment unit 160 performs high-resolution processing on the display portion of the captured image 50 that is displayed on the display unit, for example, using a high-resolution processing model. The adjustment unit 160 performs high-resolution processing on the display portion of the captured image 50 that is displayed on the display unit by inputting the captured image 50, the resolution data of the captured image 50 contained in the metadata of the captured image 50 stored in the information storage unit 152, the magnification data indicating the magnification of the captured image 50, and the display portion data indicating the display portion of the captured image 50 that is displayed on the display unit, as input data to the high-resolution processing model.

[0110] The adjustment unit 160 may adjust the display of the captured image 50 on the display unit of the glasses-type device 100 based on information for adjusting the display of the captured image 50 on the display unit of the glasses-type device 100. Information for adjusting the display of the captured image 50 on the display unit of the glasses-type device 100 may be obtained by the operation unit or input unit of the glasses-type device 100 receiving various inputs.

[0111] The determination unit 162 performs various determination processes. For example, the determination unit 162 performs various determination processes based on the various information stored in the information storage unit 152.

[0112] The determination unit 162 performs various determination processes, for example, when the identification unit 156 identifies the object 55 that the user 102 of the glasses-type device 100 is viewing, which is included in the captured image 50. The determination unit 162 also performs various determination processes, for example, when the determination unit 158 ​​determines the actual size of the object 55.

[0113] The determination unit 162 determines, for example, whether the captured image 50 includes the entire object to be viewed 55. The determination unit 162 determines, for example, whether the captured image 50 includes the entire object to be viewed 55 by performing image analysis on the captured image 50.

[0114] For example, if the determination unit 162 determines that the captured image 50 includes the entire object to be viewed 55, the adjustment unit 160 adjusts the display of the captured image 50 on the display unit. In this case, the adjustment unit 160 may adjust the display of the captured image 50 on the display unit so that, within the range in which the entire object to be viewed 55 is displayed on the display unit, the display size of the object to be viewed included in the captured image 50 more closely matches the viewing size of the object to be viewed 55 when it is viewed from a predetermined separation distance.

[0115] The image generation unit 164 generates a supplemented image 60 in which the missing portion 57 of the object to be viewed 55 that is not included in the captured image 50 is filled in. For example, if the determination unit 162 determines that the captured image 50 does not include the entire object to be viewed 55, the image generation unit 164 generates the supplemented image 60.

[0116] The image generation unit 164 generates a complementary image 60 based on the captured image 50, for example. The image generation unit 164 generates the complementary image 60 from the captured image 50 using a generation model stored in the model storage unit 170, for example.

[0117] The adjustment unit 160 adjusts, for example, the display of the complementary image 60 on the display unit. The adjustment unit 160 adjusts, for example, the display of the complementary image 60 on the display unit when the image generation unit 164 generates the complementary image 60. The adjustment unit 160 adjusts the display of the complementary image 60 on the display unit so that, within the range in which the entire object to be seen 55 is displayed on the display unit, the display size of the object to be seen 55 included in the complementary image 60 more closely matches the viewing size of the object to be seen 55 when the object to be seen 55 is viewed from a predetermined separation distance.

[0118] The adjustment unit 160 may adjust the display of the interpolated image 60 on the display unit based on the same information used when adjusting the display of the captured image 50 on the display unit. The adjustment unit 160 may adjust the display of the interpolated image 60 on the display unit in the same manner as when adjusting the display of the captured image 50 on the display unit.

[0119] The determination unit 162 may determine whether the actual size of the object to be viewed 55 determined by the decision unit 158 ​​is greater than a predetermined second size threshold. For example, if the determination unit 162 determines that the actual size of the object to be viewed 55 is greater than the second size threshold, the adjustment unit 160 adjusts the display of the image on the display unit so that the display size of the object to be viewed 55 better matches the viewable size of the object to be viewed 55 when viewed from a predetermined distance apart. On the other hand, if the determination unit 162 determines that the actual size of the object to be viewed 55 is smaller than the second size threshold, the adjustment unit 160 does not adjust the display of the image on the display unit so that the display size of the object to be viewed 55 better matches the viewable size of the object to be viewed 55 when viewed from a predetermined distance apart. In this case, the adjustment unit 160 may adjust the display of the image on the display unit by performing at least one of the following: an adjustment to adjust the display position in which the visible object 55 included in the image is displayed on the display unit, and an adjustment to perform a high-resolution processing on the captured image 50.

[0120] The acquisition unit 154 acquires training data, for example, which includes an image of the entire object 55 viewed by the user 102 of the glasses-type device 100. The acquisition unit 154 may store the acquired training data in the training data storage unit 166.

[0121] The model generation unit 168 generates a generative model. For example, the model generation unit 168 uses multiple training data stored in the training data storage unit 166 as training data to generate a generative model using machine learning. The model generation unit 168 may store the generated generative model in the model storage unit 170.

[0122] Figure 10 shows an example of information stored in the information processing device 150. Here, we will mainly describe an example of information stored in the information processing device 150 that is associated with an identifier that identifies the object 55 viewed by the user 102 of the glasses-type device 100.

[0123] As shown in Figure 10, attribute information of the object to be viewed 55, actual size information of the object to be viewed 55, and image data of the object to be viewed 55 are stored in the information processing device 150, associated with the identifier of the object to be viewed 55. Also, as shown in Figure 10, the attribute information of the object to be viewed 55 may include classification information of the object to be viewed 55, gender information of the object to be viewed 55, and age information of the object to be viewed 55.

[0124] The attribute information, actual size information, and image data of the object 55 with identifier "0001" shown in Figure 10 are as follows. Note that object 55 with identifier "0001" may sometimes be referred to as object 1.

[0125] The attribute information for object 1 includes the classification information, gender information, and age information of object 1. The classification of object 1, as indicated by the classification information, is "human". The gender of object 1, as indicated by the gender information, is "male". The age of object 1, as indicated by the age information, is "2 years old". The actual size information for object 1 shows the actual size of object 1 for each age group. When object 1 is "0 years old", the actual size of object 1 is "70 cm". When object 1 is "1 year old", the actual size of object 1 is "80 cm". When object 1 is "2 years old", the actual size of object 1 is "90 cm". The image data for object 1 includes image data for each age group of object 1. The image data of object 1 includes image data of object 1 when object 1 is "0 years old", image data of object 1 when object 1 is "1 year old", and image data of object 1 when object 1 is "2 years old".

[0126] The attribute information, actual size information, and image data of the object 55 with identifier "0002" shown in Figure 10 are as follows. Note that object 55 with identifier "0002" may sometimes be referred to as object 2.

[0127] The attribute information for object 2 includes its classification information, gender information, and age information. The classification of object 2, as indicated by its classification information, is "animal". The gender of object 2, as indicated by its gender information, is "female". The age of object 2, as indicated by its age information, is "3 years old". The actual size of object 2, as indicated by its actual size information for object 1, is "50 cm". The image data for object 2 is image data of a "dog".

[0128] The attribute information, actual size information, and image data of the object 55 with identifier "0003" shown in Figure 10 are as follows. Note that object 55 with identifier "0003" may sometimes be referred to as object 3.

[0129] The attribute information for object 3 includes its classification information. The classification of object 3, as indicated by its classification information, is "animal". The actual size of object 3, as indicated by its actual size information, is "3.0 mm". The image data for object 3 is image data of an "ant".

[0130] The attribute information, actual size information, and image data of the object 55 with identifier "0004" shown in Figure 10 are as follows. Note that object 55 with identifier "0004" may sometimes be referred to as object 4.

[0131] The attribute information for object 4 includes its classification information. The classification of object 4, as indicated by its classification information, is "building". The actual size of object 4, as indicated by its actual size information, is "5m". The image data for object 4 is image data of a "detached house".

[0132] Figure 11 is an explanatory diagram illustrating an example of the processing flow of the glasses-type device 100. Here, the starting state is defined as a state in which the separation distance used to adjust the display of the captured image 50, which includes the object 55 viewed by the user 102 of the glasses-type device 100, on the display unit of the glasses-type device 100 has not been set.

[0133] In step 102 (step may be abbreviated as S), the setting unit 155 sets the separation distance. In S104, the identification unit 156 identifies the object to be seen 55 included in the captured image 50 stored in the information storage unit 152. In S106, the determination unit 158 ​​determines the actual size of the object to be seen 55 identified by the identification unit 156 in S104.

[0134] In S108, the determination unit 162 determines whether the captured image 50 includes the entire object to be viewed 55 that the identification unit 156 identified in S104. If the determination unit 162 determines that the captured image 50 includes the entire object to be viewed 55, the process proceeds to S110. If the determination unit 162 determines that the captured image 50 does not include the entire object to be viewed 55, the process proceeds to S114.

[0135] In S110, the adjustment unit 160 adjusts the display of the captured image 50 on the display unit, based on the actual size of the object to be viewed 55 determined by the determination unit 158 ​​in S106, so that the display size of the object to be viewed 55 included in the captured image 50 more closely matches the viewing size of the object to be viewed 55 when it is viewed from a distance set by the setting unit 155 in S102. In S112, the glasses-type device 100 displays the captured image 50 on the display unit according to the adjustment made by the adjustment unit 160 in S110. After that, the process of the glasses-type device 100 adjusting the display of the image on the display unit is completed.

[0136] In S114, the image generation unit 164 generates a complementary image 60 based on the captured image 50, filling in the missing portion 57 of the object to be viewed 55 that is not included in the captured image 50. In S116, the adjustment unit 160 adjusts the display of the complementary image 60 on the display unit so that the display size of the object to be viewed 55 included in the complementary image 60 generated by the image generation unit 164 in S114 better matches the viewing size of the object to be viewed 55 when viewed from a distance set by the setting unit 155 in S102, based on the actual size of the object to be viewed 55 determined by the determination unit 158 ​​in S106. In S118, the glasses-type device 100 displays the complementary image 60 on the display unit according to the adjustment made by the adjustment unit 160 in S116. After that, the process of the glasses-type device 100 adjusting the display of the image on the display unit is completed.

[0137] Figure 12 schematically shows an example of the hardware configuration of a computer 1200 that functions as an information processing device 150. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the apparatus according to this embodiment, or to cause the computer 1200 to execute operations associated with the apparatus according to this embodiment or such one or more "parts", and / or to cause the computer 1200 to execute a process or a stage of such process according to this embodiment. Such a program may be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0138] The computer 1200 according to this embodiment includes a CPU 1212, RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive and a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive and a solid-state drive, etc. The computer 1200 also includes legacy input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0139] The CPU 1212 operates according to the programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires the image data generated by the CPU 1212 and stores it in the frame buffer provided in RAM 1214 or within itself, so that the image data is displayed on the display device 1218.

[0140] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from a DVD-ROM 1227, etc., and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0141] The ROM 1230 stores boot programs and / or hardware-dependent programs of the computer 1200, which are executed by the computer 1200 upon activation. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

[0142] The program is provided on a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The program is read from the computer-readable storage medium and installed on a storage device 1224, RAM 1214, or ROM 1230, which are examples of computer-readable storage media, and executed by the CPU 1212. The information processing described within these programs is read by the computer 1200, resulting in coordination between the program and the various types of hardware resources described above. The apparatus or method may be configured to realize the operation or processing of information in accordance with the use of the computer 1200.

[0143] For example, when communication is performed between a computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into RAM 1214 and, based on the processing described in the communication program, instruct the communication interface 1222 to perform communication processing. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as RAM 1214, storage device 1224, DVD-ROM 1227, or IC card, transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area or the like provided on the recording medium.

[0144] Furthermore, the CPU 1212 may read all or necessary parts of files or databases stored on external recording media such as the storage device 1224, DVD drive 1226 (DVD-ROM 1227), or IC card into the RAM 1214, and perform various types of processing on the data in the RAM 1214. The CPU 1212 may then write the processed data back to the external recording media.

[0145] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 1212 may perform various types of processing on the data read from RAM 1214, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., as described throughout this disclosure and specified by the program instruction sequence, and write the results back to RAM 1214. The CPU 1212 may also retrieve information in files, databases, etc., within the recording medium. For example, if multiple entries are stored in the recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 may search among the multiple entries for an entry that matches the specified condition for the attribute value of the first attribute, read the attribute value of the second attribute stored in that entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies the predetermined condition.

[0146] The program or software module described above may be stored on or near the computer 1200 in a computer-readable storage medium. Alternatively, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0147] In this embodiment, blocks in the flowchart and block diagram may represent a stage in a process in which an operation is performed or a "part" of a device that has the role of performing an operation. A particular stage and "part" may be implemented by a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include reconfigurable hardware circuits, such as field-programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), which include logical AND, logical OR, exclusive OR, negated AND, negated OR, and other logical operations, flip-flops, registers, and memory elements.

[0148] Computer-readable media may include any tangible device capable of storing instructions to be executed by a suitable device, and as a result, computer-readable media having instructions stored therein will comprise a product containing instructions that can be executed to create means for performing operations specified in a flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital multipurpose disc (DVD), Blu-ray® disc, memory stick, integrated circuit card, etc.

[0149] Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, Java®, C++, and traditional procedural programming languages ​​such as the C programming language or similar programming languages.

[0150] Computer-readable instructions are provided locally or via a wide area network (WAN) such as a local area network (LAN) or the internet to the processor or programmable circuit of a programmable data processing device such as a computer, and may be executed to create means for performing operations specified in a flowchart or block diagram. Here, the computer may be a PC (personal computer), tablet computer, smartphone, workstation, server computer, general-purpose computer, or special-purpose computer, and may also be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system and is a computer in a broad sense. In a distributed computing system, multiple computers execute a program collectively by having each computer execute a part of the program and passing data during program execution between computers as needed.

[0151] Examples of processors include computer processors, central processing units (CPUs), processing units, microprocessors, digital signal processors, controllers, and microcontrollers. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of the program, and the processors collectively execute the program by passing program execution data between them as needed. For example, in the execution of multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at each time slice. In this case, which part of a program each processor executes changes dynamically. Which part of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0152] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0153] It should be noted that the execution order of operations, procedures, steps, and stages in the apparatus, systems, programs, and methods shown in the claims, specifications, and drawings is not explicitly stated as "before" or "prior to," and that these can be implemented in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," and "next," for convenience, this does not mean that it is essential to perform the operations in that order. [Explanation of symbols]

[0154] 10 System, 20 Network, 50 Captured Image, 55 Object of View, 57 Missing Part, 60 Completion Image, 100 Eyeglass-type Device, 102 User, 105 Eye, 110 Frame, 112 Rim, 114 Bridge, 116 Lens, 118 Temple, 150 Information Processing Device, 152 Information Storage Unit, 154 Acquisition Unit, 155 Setting Unit, 156 Identification Unit, 158 Decision Unit, 160 Adjustment Unit, 162 Judgment Unit, 164 Image Generation Unit, 166 Training Data Storage Unit, 168 Model Generation Unit, 170 Model Storage Unit, 180 Scanner, 190 Camera, 200 Image Database, 1200 Computer, 1210 Host Controller, 1212 CPU, 1214 RAM, 1216 Graphics Controller, 1218 Display Device, 1220 Input / Output Controller, 1222 Communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. An information processing device mounted on a spectacle-type device having a display unit, An information storage unit for storing captured images, A specification unit that identifies a predetermined object viewed by the user of the glasses-type device, which is included in the captured image stored in the information storage unit, A determination unit that determines the actual size of the object to be observed, which has been identified by the specified unit, Based on the actual size of the object to be viewed determined by the determination unit, an adjustment unit adjusts the display of the captured image on the display unit so that the display size of the object to be viewed included in the captured image more closely matches the viewing size of the object when the object to be viewed from a predetermined distance apart. An information processing device equipped with the following features.

2. The information processing apparatus according to claim 1, wherein the adjustment unit adjusts the display of the captured image on the display unit so as to enlarge the captured image to a magnification higher than a predetermined magnification threshold, and performs a high-resolution processing on the display portion of the captured image that is displayed on the display unit.

3. Determination unit for determining whether the captured image includes the entire object to be viewed. Furthermore, When the determination unit determines that the captured image includes the entire object to be viewed, the adjustment unit adjusts the display of the captured image on the display unit so that the display size of the object to be viewed included in the captured image better matches the display size of the object to be viewed when the object is viewed from a distance equal to the separation distance, within the range in which the entire object to be viewed is displayed on the display unit. The information processing apparatus according to claim 1 or 2.

4. If the determination unit determines that the captured image does not include the entire object to be viewed, the image generation unit generates a supplementary image based on the captured image, in which the missing portion of the object to be viewed that is not included in the captured image is filled in. Furthermore, The adjustment unit adjusts the display of the complementary image on the display unit so that, within the range in which the entire object to be viewed is displayed on the display unit, the display size of the object to be viewed included in the complementary image more closely matches the display size of the object to be viewed when the object to be viewed from a distance equal to the separation distance. The information processing apparatus according to claim 3.

5. The information processing apparatus according to claim 4, wherein the image generation unit generates the complementary image from the captured image stored in the information storage unit using a generation model that generates a complementary image in which the missing portion of the object being viewed is filled in from a captured image that includes a part of the object being viewed.

6. The adjustment unit estimates the visible size of the visible object when the visible object is visible from a distance equal to the separation distance, using the following formula: S ｖ =(S ｒ ×f) / D Here, S ｖ S is the visible size of the visible object when the visible object is visible from a distance equal to the aforementioned separation distance. ｒ f is the actual size of the object being viewed, f is the average focal length of the human eye, and D is the distance between the two objects. The information processing apparatus according to claim 1 or 2.

7. Based on the actual size of the object to be viewed, the setting unit sets a user-configurable range of separation distance so that the viewing size of the object to be viewed falls within a predetermined range. The information processing apparatus according to claim 1 or 2, further comprising:

8. The information processing apparatus according to claim 1 or 2, wherein the determination unit determines the actual size of the object to be viewed by estimating the actual size of the object to be viewed based on the captured image.

9. The information processing apparatus according to claim 1 or 2, wherein the determination unit determines the actual size of the object to be viewed by estimating the actual size of the object to be viewed based on scan data obtained by scanning the object to be viewed.

10. The information storage unit further stores, in association with, object identification information that identifies the object to be viewed and actual size information that indicates the actual size of the object to be viewed. The determination unit determines the actual size of the object by acquiring the actual size information of the object to be viewed, which is stored in the information storage unit in association with the object identification information of the object to be viewed identified by the identification unit. The information processing apparatus according to claim 1 or 2.

11. A glasses-type device equipped with the information processing device described in claim 1 or 2.

12. A program that, when executed by a computer, causes the computer to function as an information processing device according to claim 1 or 2.

Images