Visibility regulation system, visibility regulation apparatus and visibility regulation method

Abstract

To provide a technique for suitably capturing an image from a position different from the viewpoint of a user.SOLUTION: A visibility regulation system comprises: an image acquisition unit for executing image-capturing from a position different from the viewpoint of a user to acquire an image; a target specification unit for detecting an object included in the image to determine whether or not it is a prescribed target; and a notification output instruction unit for notifying the user of whether or not the prescribed target is included in the image per target.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a field of view adjustment system, a field of view adjustment device, and a field of view adjustment method. [Background technology]

[0002] Patent document 1 describes the image processing device as follows: "An image processing device that receives an image signal and generates a display image having an image range corresponding to a display area of ​​a display device from an image based on the image signal, the image processing device comprising: a position of interest calculation unit that calculates a position of interest, which is the position of an object of interest in an image based on the image signal; a frame-out probability calculation unit that calculates a frame-out probability indicating the probability that the position of interest will fall outside the image range, based on the position of interest and the image range; a modification variable determination unit that determines a processing variable of a modification processing to be applied to an image based on the image signal, based on the frame-out probability; and an image modification unit that applies the modification processing to an image based on the image signal, based on the determined processing variable, to generate a signal for the display image." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-170046 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above technology, it is possible to process the captured image in order to guide the user to prevent the object of interest from going out of the frame, but this technology is based on the premise that the imaging device is approximately the same as the user's viewpoint. Also, it is based on the premise that the user adjusts the field of view of the imaging device while checking the captured image. In contrast, it is difficult to properly capture an object without checking the image captured from a position different from the user's viewpoint.

[0005] An object of the present invention is to provide a technique for appropriately capturing an image from a position different from the user's viewpoint. [Means for solving the problem]

[0006] The present application includes a number of means for solving at least part of the above problems, examples of which are as follows: In order to solve the above problems, the field of view adjustment system according to the present invention includes an image acquisition unit that captures an image from a position different from the user's viewpoint and acquires an image, an object identification unit that detects objects included in the image and determines whether or not the object is a predetermined object, and a notification output instruction unit that notifies the user, for each object, whether or not the predetermined object is included in the image.

[0007] In addition, the above-mentioned field of view adjustment system may be provided with a guidance unit that guides the user's actions so that the object is included in the image depending on the detection state of the object that is not included in the image.

[0008] In the above-described field of view adjustment system, the object may include at least one of a human hand and a predetermined figure.

[0009] In addition, in the above-mentioned field of view adjustment system, the object may include at least a predetermined figure, and the guidance unit may guide the user's action by showing the figure when the predetermined figure is not detected in the image.

[0010] In addition, in the above-mentioned field of view adjustment system, the object may include at least a human hand and a specified figure, and the guidance unit may guide the user to move his or her arm when the specified figure has not been detected in the image but the human hand has been detected.

[0011] In addition, in the above-mentioned field of view adjustment system, the guidance unit may guide the user to move backwards when the movement of the user's arm is detected and the specified figure remains undetected in the image.

[0012] In addition, in the above-mentioned field of view adjustment system, the object may include at least a human hand and a specified figure, and the guidance unit may guide the image acquisition unit to move its position when the human hand and the figure have been detected and the figure has framed out of the image.

[0013] In addition, in the above-mentioned field of view adjustment system, the object may include at least a human hand and a specified figure, and the guidance unit may guide the user to move their arm in a specified direction when the human hand and the figure have been detected and the figure is obstructed by the human hand.

[0014] In the above-described visual field adjustment system, the direction in which the guiding unit guides the movement of the user's arm may be a central left-right direction of the user's body.

[0015] In addition, a field of view adjustment device according to another embodiment of the present application includes an image receiving unit that receives an image captured from a position different from the user's viewpoint, an object identification unit that detects objects included in the image and determines whether or not they are specified objects, and a notification output instruction unit that causes a device worn by the user to output information informing the user whether or not the specified object is included in the image for each object.

[0016] Furthermore, a field of view adjustment method according to another embodiment of the present application is a field of view adjustment method using an information processing device, in which the information processing device performs an image acquisition step of capturing an image from a position different from the user's viewpoint, an object identification step of detecting objects included in the image and determining whether or not they are specified objects, and a notification output instruction step of notifying the user, for each object, whether or not the specified object is included in the image.

[0017] According to the present invention, it is possible to provide a technique for appropriately capturing an image from a position different from the user's viewpoint. However, problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating an example of the configuration of a visual field adjustment system according to a first embodiment. [Diagram 2] 1A to 1C are diagrams illustrating an example of the appearance of a wearable device and a camera field of view. [Diagram 3] 11 is a diagram illustrating an example of a data structure stored in an object storage unit. FIG. [Figure 4] FIG. 2 is a diagram illustrating a hardware configuration of a wearable terminal. [Diagram 5] FIG. 11 is a diagram illustrating an example of a flow of a field of view adjustment process. [Figure 6] FIG. 13 is a diagram illustrating an example of an output of the guidance process. [Figure 7] FIG. 13 is a diagram illustrating an example of an output of the guidance process. [Figure 8] FIG. 13 is a diagram illustrating an example of an output of the guidance process. [Figure 9] FIG. 13 is a diagram illustrating an example of an output of the guidance process. [Figure 10] FIG. 11 is a diagram illustrating an example of the configuration of a visual field adjustment system according to a second embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a flow of a field of view adjustment process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, a visual field adjusting system 1 to which a first embodiment of the present invention is applied will be described with reference to the drawings.

[0020] FIG. 1 is a diagram showing a configuration example of a visual field adjustment system 1 according to a first embodiment of the present invention. The visual field adjustment system 1 assumed in this embodiment includes a wearable terminal 100. In this system, a user 10 performs a predetermined operation on an operation target 500 while wearing the wearable terminal 100 on the body (for example, on the face in the case of a glasses-type device, so-called smart glasses, etc.). For example, the operation target 500 performs maintenance work, installation work, use operation, etc. on the operation target 500. Various things apply to the operation target 500. It does not necessarily have to be an object to be operated, and may be, for example, ingredients in a cooking scene, building materials in an interior construction scene, interior decoration in an environment where a game device is installed in the case of a video distributor such as a game live broadcast, or an answer sheet in an educational scene.

[0021] The wearable terminal 100 is a device also called a wearable device, including any computer that the user 10 can detachably attach to a part of the body. Preferably, the wearable terminal 100 is worn by the user, can acquire image information captured in the same direction as the user's field of vision, and can obtain audio information perceived by the user. Smart glasses are one such so-called wearable device, and since they do not restrict the user's hands when worn, they can more efficiently support the user in improving operability.

[0022] The wearable terminal 100 may be a terminal owned by the user 10, or may be a terminal loaned to the user by an organization to which the user 10 belongs.

[0023] 2 is a diagram showing an example of the appearance of a wearable device and a camera field of view. The wearable device 100 is a glasses-type device that can be worn without blocking the field of view 410 of the eye 400 of the user 10 as much as possible. In addition, a camera 17 that captures an image from a viewpoint different from the field of view 410 is provided in a part of the wearable device 100 (for example, the end of the temple on the right eye side) in the direction of the field of view 410 of the eye 400 of the user 10.

[0024] That is, the viewpoint of camera 17 is offset to a position (for example, about several centimeters to the right eye side) different from the viewpoints of either the left or right eye of user 10. Therefore, there is a difference between the field of view of user 10 and the field of view of the image captured by camera 17, and it may happen that what is seen from the viewpoint of user 10 cannot be captured by camera 17, or vice versa.

[0025] Furthermore, the wearable terminal 100 alone cannot view the images captured by the camera 17 in real time. That is, the user 10 cannot check the field of view of the camera 17 or how the object is reflected in real time.

[0026] The wearable terminal 100 includes a storage unit 110, a display control unit 120, an image processing unit 150, an audio output unit 160, and a communication unit 170.

[0027] The display control unit 120 includes a notification display unit 121. The notification display unit 121 displays information to the wearer of the wearable terminal 100, i.e., the user 10, via a transmissive display. In other words, the notification display unit 121 is a display unit for the user, and displays information to the wearer by superimposing it through the lenses, for example, by projecting an image of visual display information onto the lens surface close to the wearer's side of the glasses.

[0028] The storage unit 110 includes an object storage unit 111. The object storage unit 111 stores information about an object for which object detection is to be performed.

[0029] 3 is a diagram showing the data structure of the object storage unit 111. The object storage unit 111 includes an object identifier 111a, an object name 111b, an object substitute icon 111c, and object information 111d.

[0030] The object identifier 111a is information for identifying an object for which object detection is to be performed from an image captured by the camera 17. The object name 111b is name information of the object. The object substitute icon 111c is information for specifying icon image information used when displaying a result of whether or not the object is detected. The object information 111d is information such as model information for performing object detection. For example, if the object is an AR (Augmented Reality) marker, model information for detecting the AR marker is stored in the object information 111d. Note that here, the model refers to a trained model that has undergone machine learning such as so-called deep learning, and can detect and identify whether an object is included in an image. Also, if the object is a human hand, model information for detecting the hand is stored in the object information 111d. Note that the object includes at least either a human hand or a predetermined figure (such as the above-mentioned AR marker). In addition, the model may be capable of detecting an object that has been specified in advance as object information 111d in object memory unit 111, regardless of whether the object is a human hand, a specified figure, a specified device, etc.

[0031] Returning to the explanation of Fig. 1, the image processing unit 150 includes an image acquiring unit 151, an object identifying unit 152, a notification output instruction unit 153, and a guidance unit 154. The image acquiring unit 151 acquires an external image acquired by the wearable terminal 100, that is, an image captured from a position in the user's visual field direction and different from the user's viewpoint.

[0032] The object identification unit 152 detects an object included in the image and determines whether it is a predetermined object. Here, the object identification unit 152 may detect an object included in the object storage unit 111. For detection by the object identification unit 152, a known object detection mechanism using image processing (analysis using an algorithm that determines the similarity of objects, various detection methods using a trained model (convolutional neural network) that has undergone machine learning, etc.) can be adopted.

[0033] The notification output instruction unit 153 notifies the user whether or not a predetermined object is included in the image for each object. Specifically, the notification output instruction unit 153 generates screen information to be displayed on the notification display unit 121 and passes it to the notification display unit 121.

[0034] The guiding unit 154 guides the user's movement so that the object is included in the image according to the detection state of the object not included in the image. For example, when a predetermined figure has not been detected on the image, the guiding unit 154 guides the user's movement by showing the figure. Furthermore, when a predetermined figure has not been detected on the image and a human hand has been detected, the guiding unit 154 guides the user to move his / her arm. Furthermore, when the guiding unit 154 detects the movement of the user's arm and the predetermined figure remains undetected on the image, the guiding unit 154 guides the user to move backward. Furthermore, when a human hand and figure have been detected and the figure has framed out of the image, the guiding unit 154 guides the user to move the position of the image acquisition unit. Furthermore, when a human hand and figure have been detected and the figure has been blocked by the human hand, the guiding unit 154 guides the user to move his / her arm in a predetermined direction. It is preferable that the direction in which guiding unit 154 guides the movement of the user's arm is toward the center between the left and right of the user's body as seen from the viewpoint of camera 17.

[0035] The audio output unit 160 is a so-called speaker or a DAC (Digital Analog Converter) that outputs audio information to be output by voice reading or voice playback. The communication unit 170 controls communication with other devices. Specifically, it is preferable that the communication unit 170 controls short-range radio communication using, for example, Bluetooth (registered trademark), wireless LAN (Local Area Network), or NFC (Near Field Communication). However, the communication unit 170 may also be one that participates in a mobile phone communication network such as a 5G network or a 4G network, or one that controls a predetermined wired communication such as a USB (Universal Serial Bus).

[0036] 4 is a diagram showing a hardware configuration of the wearable terminal 100. The wearable terminal 100 includes at least a processor 11, a memory 12, a storage 13, a transparent display 15, a communication device 16, a camera 17, and a bus connecting the devices. In addition, the wearable terminal 100 may include a contact input device. Note that, in the case of a glasses-type wearable device, the contact input device may be, for example, a device that accepts contact with the temples of the glasses as a click input and accepts the direction of sliding in a contact state as a scroll input, but is not limited thereto.

[0037] The processor 11 is an arithmetic device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and executes processing according to a program recorded in the memory 12 or the storage 13. In the wearable terminal 100, processing is performed by the processor 11 operating according to a program read onto the memory 12 or the storage 13. The image processing unit 150, the image acquisition unit 151, the object identification unit 152, the notification output instruction unit 153, and the guidance unit 154 each realize their respective functions by the processor 11 executing the program.

[0038] The memory 12 is a storage device such as a RAM (Random Access Memory) or a flash memory, and functions as a storage area from which programs and data are temporarily read. The storage 13 is a writable and readable storage device. The function of the storage unit 110 is realized by the memory 12 or the storage 13. Note that the function of the storage unit 110 may be realized by a storage device connected via the communication device 16.

[0039] The transmissive display 15 is a device that provides an interface for output and outputs to a display. More specifically, the transmissive display 15 is a display device such as a projection device that projects an image onto the lens surface so that it can be viewed by the wearer, or a semi-transmissive display device provided in accordance with the lens surface. However, the transmissive display 15 is not limited thereto, and may be, for example, a display device that can irradiate light that forms an image on the retina of the user 10.

[0040] The communication device 16 is an interface for connecting the wearable terminal 100 to an external device for communication. For example, the communication device 16 performs wireless communication using an antenna that can use radio waves in a predetermined frequency band (for example, 2.4 GHz (gigahertz) or 5.2 to 5.6 GHz band) participating in Bluetooth, NFC, or wireless LAN. The communication device 16 may perform wireless communication using an antenna that can use predetermined radio waves participating in a mobile phone communication network such as a 5G network or 4G network, or may perform wired communication using a wired LAN or a wired communication network using optical fiber.

[0041] The camera 17 acquires image information including the visual field of the user wearing the wearable terminal 100.

[0042] The processing of each component of the wearable terminal 100 may be executed by one piece of hardware or by multiple pieces of hardware. The processing of each component of the wearable terminal 100 may be realized by one program or by multiple programs. The image processing unit 150 of the wearable terminal 100 may be constructed by hardware (ASIC, GPU, etc.).

[0043] The display control unit 120 of the wearable terminal 100 described above is realized by the transparent display 15. The audio output unit 160 is realized by a speaker device (not shown). The communication unit 170 is realized by the communication device 16.

[0044] The hardware configuration of the wearable terminal 100 has been described above.

[0045] [Explanation of Operation] Next, the operation of the field of view adjustment system 1 in this embodiment will be described. Fig. 5 is a diagram showing an example of the flow of the field of view adjustment process in this embodiment. The field of view adjustment process starts when the wearable terminal 100 is powered on or a start instruction is received.

[0046] First, the image acquisition unit 151 of the wearable terminal 100 controls the camera 17 to acquire an image (step S101). Here, the acquired image is a still image or a video (for example, 60 fps) including continuous still images acquired at a short predetermined interval.

[0047] Then, the object identification unit 152 detects a predetermined object reflected in the image acquired in step S101 (step S102). At this time, the object identification unit 152 estimates the object by various detection methods such as a known object detection mechanism using image processing (analysis using an algorithm that determines the similarity of objects, or a trained model (convolutional neural network) that has undergone machine learning).

[0048] Then, the object identification unit 152 determines whether each object is a target object (step S103). Specifically, the object identification unit 152 identifies the objects stored in the object storage unit 111, and determines whether each object detected in step S102 is a target object.

[0049] Then, the notification output instruction unit 153 highlights the proxy icon of the reflected object (step S104). Specifically, the notification output instruction unit 153 generates screen information for the object determined to be the object in step S103 so as to attract attention (for example, display the icon in green) to inform that the color of the icon corresponding to the object has been detected, and causes the notification display unit 121 to display the screen information. For an object for which no object is detected, the notification output instruction unit 153 generates screen information for displaying the icon corresponding to the object in a standard color (for example, white), and causes the notification display unit 121 to display the screen information. In addition to displaying the icon, a predetermined sound (such as a message reading sound or a beep sound that differs for each object) may be output from the sound output unit 160.

[0050] For example, when the undetected object is a hand and is a target object, the notification output instruction unit 153 configures the screen information to display a predetermined icon graphic representing the hand in white at the top right of the screen. When the detected object is an AR marker and is a target object, the notification output instruction unit 153 configures the screen information to display a predetermined icon graphic representing the AR marker in green at the top right of the screen.

[0051] Then, the guiding unit 154 determines whether all objects are reflected in the image (step S105). Specifically, the guiding unit 154 identifies objects stored in the object storage unit 111, and determines whether all of the identified objects are determined to be reflected by the object identification unit 152. If all objects are reflected in the image ("Yes" in step S105), the guiding unit 154 ends the field of view adjustment process.

[0052] If all of the objects are not reflected in the image ("No" in step S105), the guiding unit 154 starts the guiding process (step S106). The specific contents of the guiding process will be described later.

[0053] The above is the field of view adjustment process. According to the field of view adjustment process, whether or not the target object is included in the imaging range can be displayed to the user 10 by an icon on the transparent display 15, so that the image can be appropriately captured from a position different from the user's viewpoint.

[0054] Fig. 6 is a diagram showing an example of output of the guidance process. As an example of the guidance process, Fig. 6 shows an example of guidance when the AR marker is blocked by the hand after the AR marker and the hand are detected.

[0055] The hand icon 650 is an icon displayed in green on the transparent display 15 because the hand 630 is reflected in the icon. The AR marker icon 640 is an icon displayed in green on the transparent display 15 because the AR marker 620 is reflected in the icon. The upper left diagram shows a state in which both the AR marker 620 and the hand 630 are reflected in the icon, and both icons are displayed in green.

[0056] The lower right diagram shows a state in which the AR marker 620 is blocked by the moving hand 630'. In other words, the AR marker 620 is no longer reflected. In this state, the AR marker icon 640' is an icon displayed in white on the transparent display 15. Also, on the transparent display 15, a guidance message stating "You are blocking the AR marker. Move your arm to the left" is displayed by the guidance unit 154. When presenting the direction to move the arm in the guidance message, the guidance unit 154 may present the center direction of the left and right of the body of the user 10 as seen from the viewpoint of the camera 17. This is because the movement of the hand toward the center direction of the left and right of the body is not unreasonable due to the relationship between the viewpoint of the camera 17 and the parallax of the user 10.

[0057] Fig. 7 is a diagram showing an example of output of the guidance process. As an example of the guidance process, Fig. 7 shows an example of guidance when the camera 17 shifts (to the right side of the user 10) after detecting an AR marker and a hand.

[0058] The hand icon 650 is an icon displayed in green on the transparent display 15 because the hand 630 is reflected in the icon. The AR marker icon 640 is an icon displayed in green on the transparent display 15 because the AR marker 620 is reflected in the icon. The upper left diagram shows a state in which both the AR marker 620 and the hand 630 are reflected in the icon, and both icons are displayed in green.

[0059] The lower right diagram shows a state in which the AR marker 620 has gone out of frame due to the camera being moved. In other words, the AR marker 620 is no longer reflected. In this state, the AR marker icon 640' is an icon displayed in white on the transparent display 15. Also, a guidance message saying "Please return the camera position" is displayed on the transparent display 15 by the guiding unit 154. This is realized by the guiding unit 154 detecting that the AR marker 620 has gone out of frame.

[0060] Fig. 8 is a diagram showing an example of output of the guidance process. As an example of the guidance process, Fig. 8 shows an example of guidance when an AR marker is detected from a state in which neither the AR marker nor the hand is detected.

[0061] The hand icon 650' is an icon displayed in white on the transparent display 15 because only a part of the hand 635 is reflected. The AR marker icon 640' is an icon displayed in white on the transparent display 15 because the AR marker 620 is not reflected. The upper left diagram shows a state in which neither the AR marker 620 nor the hand 630 is reflected, and both icons are displayed in white. On the transparent display 15, the guidance unit 154 displays a guidance message saying, "Please find an AR marker that looks like (image showing the shape of an AR marker)."

[0062] The lower right diagram shows a state in which the AR marker 620 is reflected. That is, this is a state in which the AR marker 620 is reflected. In this state, the AR marker icon 640 is an icon displayed in green on the transmissive display 15.

[0063] Fig. 9 is a diagram showing an example of output of the guidance process. As an example of the guidance process, Fig. 9 shows a guidance example in which guidance for detecting an AR marker is performed after a hand is recognized.

[0064] The hand icon 650 is an icon displayed in green on the transparent display 15 because the hand 630 is reflected. The AR marker icon 640' is an icon displayed in white on the transparent display 15 because the AR marker 620 is not reflected. The upper left diagram shows a state in which only the hand 630 is reflected and only the hand icon 650 is displayed in green. In this case, the guidance unit 154 displays a guidance message saying "Please move your arm to the lower left" on the transparent display 15. In other words, this corresponds to a case in which the AR marker has not been detected but a human hand has been detected, and the guidance unit 154 guides the user 10 to move his / her arm.

[0065] The upper right diagram shows a state in which the hand 630 has moved. In this state, the AR marker icon 640' is an icon displayed in white on the transparent display 15. Also, on the transparent display 15, a guidance message saying "Please take one step back" is displayed by the guiding unit 154. This is realized by the guiding unit 154 detecting the movement of the hand 630. In other words, when the movement of the user's arm is detected, this corresponds to the case in which the AR marker remains undetected, and the guiding unit 154 guides the user 10 to move backward.

[0066] The lower right diagram shows a state in which the user 10 has moved backward. That is, both the hand 630 and the AR marker 620 are reflected in the image. In this state, the hand icon 650 is an icon displayed in green on the transparent display 15 because the hand 630 is reflected in the image. In addition, the AR marker icon 640 is an icon displayed in green on the transparent display 15.

[0067] The above is an example of the guidance process. With the guidance process, it is possible to guide the user to avoid the reflection of the object without looking at the captured image. Therefore, it can be said that the image can be appropriately captured from a position different from the user's viewpoint.

[0068] The above is the visual field adjusting system 1 according to the first embodiment. According to the first embodiment, it is possible to appropriately capture an image from a position different from the user's viewpoint.

[0069] The present invention is not limited to the above-mentioned first embodiment. The above-mentioned first embodiment can be modified in various ways within the scope of the technical idea of ​​the present invention. For example, in the above-mentioned first embodiment, the wearable terminal 100 is assumed to be a glasses-type wearable terminal, but is not limited to this. For example, it may be a helmet-type wearable terminal that covers the head more widely, or a wearable terminal that is attached to only one eye.

[0070] 10 is a diagram showing a configuration example of a visual field adjustment system according to a second embodiment of such a modified example. In the second embodiment, calculations for object detection and guidance are not performed in the wearable terminal 100, but are performed in the visual field adjustment device 200 via a network, thereby making it possible to reduce the computing power required for the wearable terminal 100. This is expected to have an effect of reducing the overall hardware cost and the operating costs for software update management of the wearable terminal 100, particularly when multiple wearable terminals 100 are operated in the visual field adjustment system 1.

[0071] The visual field adjusting system 1' according to the second embodiment is basically similar to the visual field adjusting system 1 according to the first embodiment, but there are some differences. The following mainly describes the differences.

[0072] The wearable terminal 100' does not need to include the object storage unit 111, the object identification unit 152, the notification output instruction unit 153, and the guidance unit 154. However, the communication unit 170 can communicate with the field of view adjustment device 200 via various communication paths. The communication path may be any one of short-range radio wave communication such as Bluetooth, NFC, and wireless LAN, a mobile phone communication network such as a 5G network or a 4G network, or a predetermined communication line such as a USB, or an equivalent thereof.

[0073] The field of view adjustment device 200 includes a storage unit 210, an image processing unit 220, and a communication unit 230. The storage unit 210 includes an object storage unit 211. The object storage unit 211 stores information on an object for which object detection is performed. The object storage unit 211 has a data structure similar to that of the object storage unit 111 in the first embodiment.

[0074] The image processing unit 220 includes an image receiving unit 221, an object identification unit 222, a notification output instruction unit 223, and a guiding unit 224. The image receiving unit 221 receives an image transmitted from the wearable terminal 100'. The object identification unit 222, the notification output instruction unit 223, and the guiding unit 224 basically perform the same processes as the object identification unit 152, the notification output instruction unit 153, and the guiding unit 154, respectively.

[0075] The communication unit 230 controls communication with other devices such as the wearable terminal 100'. Specifically, it is preferable that the communication unit 230 controls short-range radio wave communication, such as Bluetooth, wireless LAN, or NFC. However, the communication unit 230 may participate in a mobile phone communication network such as a 5G network or a 4G network, or may control a predetermined wired communication such as a USB.

[0076] 11 is a diagram showing an example of a flow of a visual field adjustment process according to the second embodiment. Among the examples of the visual field adjustment process according to the second embodiment, processes having the same numbers as the examples of the visual field adjustment process according to the first embodiment have the same content.

[0077] First, the image acquisition unit 151 of the wearable terminal 100' controls the camera 17 to acquire an image and transmit it to the field of view adjustment device 200 (step S101').

[0078] Then, the image receiving unit 221 of the field of view adjusting device 200 receives the transmitted image (step S201). Then, the object identifying unit 222 detects a predetermined object reflected in the image acquired in step S201 (step S102). Then, the object identifying unit 222 determines whether each object is a target object or not (step S103).

[0079] Then, the notification output instruction unit 223 transmits an instruction to display the screen information on the notification display unit 121 by highlighting the proxy icon of the reflected object (step S104). In response to this, the notification display unit 121 changes the display of the proxy icon to a highlighted display (step S202).

[0080] For example, when the undetected object is a hand and is the target object, the notification output instruction unit 223 configures the screen information to display a predetermined icon graphic representing the hand in white at the top right of the screen. When the detected object is an AR marker and is the target object, the notification output instruction unit 223 configures the screen information to display a predetermined icon graphic representing the AR marker in green at the top right of the screen.

[0081] Then, the guiding unit 224 determines whether or not all objects are reflected in the image (step S105). Specifically, the guiding unit 224 identifies objects stored in the object storage unit 211, and determines whether or not all identified objects are determined to be reflected by the object identification unit 222. If all objects are reflected in the image ("Yes" in step S105), the guiding unit 224 ends the field of view adjustment process.

[0082] If all of the objects are not reflected in the image ("No" in step S105), the guiding unit 224 starts the guiding process for the wearable terminal 100' (step S106). The specific content of the guiding process is the same as that described above.

[0083] The above is the field of view adjustment process in the field of view adjustment system 1' according to the second embodiment. According to the field of view adjustment process, whether or not the target object is included in the imaging range can be displayed to the user 10 by an icon on the transparent display 15, so that an image can be appropriately captured from a position different from the user's viewpoint.

[0084] The field of view adjustment device 200 is an information processing device including, for example, a server device, a personal computer, a so-called smartphone, a tablet terminal, or a mobile phone device, and is a processing device capable of communicating with other devices.

[0085] The field of view adjustment device 200 includes at least a processor 11, a memory 12, a storage 13, a communication device 16, and a bus connecting each device.

[0086] The image processing unit 220, the image receiving unit 221, the object identification unit 222, the notification output instruction unit 223, and the guidance unit 224 realize their respective functions by the processor 11 executing a program.

[0087] The function of the storage unit 210 is realized by the memory 12 or the storage 13. Note that the function of the storage unit 210 may be realized by a storage device connected via the communication device 16. The communication unit 230 is realized by the communication device 16.

[0088] The processing of each component of the field of view adjustment device 200 may be executed by one piece of hardware or by multiple pieces of hardware. The processing of each component of the field of view adjustment device 200 may be realized by one program or by multiple programs. The image processing unit 220 of the field of view adjustment device 200 may be constructed by hardware (ASIC, GPU, etc.).

[0089] The configurations of the wearable terminal 100 and the field of view adjustment device 200 can be further divided into more components depending on the processing content. Also, one component can be divided into more processes.

[0090] The above is the field of view adjustment system 1' according to the second embodiment. The field of view adjustment system 1' according to the second embodiment not only properly captures images from a position different from the user's viewpoint, but also reduces the computing power required for the wearable terminal 100. This is expected to have an effect of reducing the overall hardware cost and the operating costs such as software update management for the wearable terminal 100', particularly when multiple wearable terminals 100' are operated.

[0091] The first and second embodiments have been described above. The technical elements of the above-described embodiments may be applied independently, or may be applied after being divided into a plurality of parts, such as a program part and a hardware part.

[0092] The present invention has been described above mainly with reference to the embodiment. [Explanation of symbols]

[0093] 1...visibility adjustment system, 10...user, 100...wearable terminal, 110...memory unit, 111...object memory unit, 120...display control unit, 121...notification display unit, 150...image processing unit, 151...image acquisition unit, 152...object identification unit, 153...notification output instruction unit, 154...guidance unit, 160...audio output unit, 170...communication unit.

Claims

1. an image acquisition unit that captures an image from a position different from the user's viewpoint; an object identification unit that detects an object included in the image and determines whether or not the object is a predetermined object; a notification output instruction unit that notifies the user whether or not a predetermined object is included in the image for each object; A field of view adjustment system comprising:

2. The field of view adjustment system according to claim 1, a guidance unit that guides the user's action so that the object is included in the image according to a detection state of the object not included in the image; A field of view adjustment system comprising:

3. The field of view adjustment system according to claim 2, The object includes at least one of a human hand and a predetermined figure. A field of view adjustment system.

4. The field of view adjustment system according to claim 2, The object includes at least a predetermined figure, the guiding unit, when a predetermined figure is not detected in the image, shows the figure to guide the user's action; A field of view adjustment system.

5. The field of view adjustment system according to claim 2, The object includes at least a human hand and a predetermined figure; the guiding unit guides the user to move his / her arm when the predetermined figure has not been detected in the image and the human hand has been detected. A field of view adjustment system.

6. The field of view adjustment system according to claim 5, the guiding unit guides the user to move backward when the movement of the arm of the user is detected and the predetermined figure remains undetected in the image. A field of view adjustment system.

7. The field of view adjustment system according to claim 2, The object includes at least a human hand and a predetermined figure; the guiding unit guides the image acquiring unit to move a position when the human hand and the figure have been detected and the figure has gone out of the frame of the image. A field of view adjustment system.

8. The field of view adjustment system according to claim 2, The object includes at least a human hand and a predetermined figure; the guiding unit, when the human hand and the graphic have been detected and the graphic is blocked by the human hand, guides the user to move the arm in a predetermined direction. A field of view adjustment system.

9. The field of view adjustment system according to claim 8, The direction in which the guiding unit guides the user to move the arm is the left-right center direction of the user's body. A field of view adjustment system.

10. an image receiving unit that receives an image captured from a position different from the user's viewpoint; an object identification unit that detects an object included in the image and determines whether or not the object is a predetermined object; a notification output instruction unit that causes a device worn by the user to output information notifying the user whether or not a predetermined object is included in the image for each of the objects; A field of view adjustment device comprising:

11. A field of view adjustment method using an information processing device, comprising: The information processing device includes: an image acquisition step of capturing an image from a position different from the user's viewpoint; an object identification step of detecting an object included in the image and determining whether or not it is a predetermined object; a notification output instruction step of informing the user of whether or not a predetermined object is included in the image for each object; A method for adjusting a field of view, comprising:

Images