Automatic Focus Adjusting Glasses, Control Method for Automatic Focus Adjusting Glasses, Program

Abstract

To provide an automatic focus adjusting eyeglasses, an automatic focus adjustment method and a program that can automatically adjust the focal distance without sacrificing practicality.SOLUTION: An automatic focus adjusting eyeglasses 10 pertaining to the present disclosure comprises: an imaging unit capable of acquiring image information; a variable focus lens 12 capable of changing focal distance; a depth information acquisition unit for acquiring depth information for each position with regard to the image information acquired by the imaging unit; a position of interest positioning unit 1421 for positioning a position of interest in the image information acquired by the imaging unit; a depth determination unit 1422 for determining depth of the position of interest on the basis of the position of interest in the image information positioned by the position of interest positioning unit 1421 and the depth information; and a focal distance control unit 1423 for setting the focal distance of the variable focus lens 12 on the basis of the depth of the position of interest.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an autofocus adjustment glasses, a control method thereof, and a program.

Background Art

[0002] Autofocus adjustment glasses equipped with a function of automatically adjusting the focal length of the lens of glasses are disclosed in, for example, Patent Document 1 and Patent Document 2. The autofocus adjustment glasses disclosed in Patent Document 1 have a problem in practicality, such as a camera for imaging the pupil must be arranged at a position that does not obstruct the field of view of the glasses lens in order to detect the user's line of sight. The autofocus adjustment glasses disclosed in Patent Document 2 cannot automatically adjust the focal length because the user touches the frame to change the focal length.

[0003] In such autofocus adjustment glasses, it would be beneficial if the focal length could be automatically adjusted without sacrificing practicality.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In view of the above problems, an object of the present invention is to provide an autofocus adjustment glasses, an autofocus adjustment method, and a program that can automatically adjust the focal length without sacrificing practicality.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, an autofocus-adjusting glasses according to the present disclosure includes an imaging unit capable of acquiring image information, a variable-focus lens capable of changing a focal length, a depth information acquisition unit that acquires depth information for each position of the image information acquired by the imaging unit, a fixation position determination unit that determines a fixation position in the image information acquired by the imaging unit, a depth specifying unit that specifies the depth of the fixation position based on the fixation position in the image information determined by the fixation position determination unit and the depth information, and a focal length control unit that sets the focal length of the variable-focus lens based on the depth of the fixation position. The attention position determination unit calculates the degree of change in luminance within a unit area of the image in the image information acquired by the imaging unit, sets a target area including a unit area where the degree of change in luminance is equal to or greater than a threshold value as the attention position. The attention position determination unit executes object detection processing on the image information to detect an object, specifies the depth of the object, which is the distance between the imaging unit and the object, from the position in the image information of the target area including the detected object, calculates the rate of change in the depth of the object per unit time, and when a target area including a unit area where the degree of change in luminance is equal to or greater than a threshold value is set as the attention position and the rate of change in the depth in the shallowing direction per unit time is equal to or greater than the threshold value, switches the position of the object to the attention position 。

[0007] In order to solve the above-described problems and achieve the object, a control method of an autofocus-adjusting glasses according to the present disclosure is a control method of an autofocus-adjusting glasses including an imaging unit capable of acquiring image information and a variable-focus lens capable of changing a focal length, the method including: a step of acquiring depth information for each position of the image information acquired by the imaging unit; a step of determining a fixation position in the image information acquired by the imaging unit; a step of specifying the depth of the fixation position based on the fixation position in the image information and the depth information; and a step of setting the focal length of the variable-focus lens based on the depth of the fixation position. In the step of determining the attention position, the degree of change in luminance within a unit area of the image in the image information acquired by the imaging unit is calculated, and a target area including a unit area where the degree of change in luminance is equal to or greater than a threshold value is set as the attention position. In the step of determining the attention position, object detection processing is executed on the image information to detect an object, the depth of the object, which is the distance between the imaging unit and the object, is specified from the position in the image information of the target area including the detected object, the rate of change in the depth of the object per unit time is calculated, and when a target area including a unit area where the degree of change in luminance is equal to or greater than a threshold value is set as the attention position and the rate of change in the depth in the shallowing direction per unit time is equal to or greater than the threshold value, the position of the object is switched to the attention position 。

[0008] In order to solve the above-described problems and achieve the object, a program according to the present disclosure is a program that causes a computer to execute a control method of an autofocus-adjusting glasses including an imaging unit capable of acquiring image information and a variable-focus lens capable of changing a focal length, the method including: a step of acquiring depth information for each position of the image information acquired by the imaging unit; a step of determining a fixation position in the image information acquired by the imaging unit; a step of specifying the depth of the fixation position based on the fixation position in the image information and the depth information; and a step of setting the focal length of the variable-focus lens based on the depth of the fixation position. In the step of determining the attention position, the degree of change in luminance within a unit area of the image in the image information acquired by the imaging unit is calculated, and a target area including a unit area where the degree of change in luminance is equal to or greater than a threshold value is set as the attention position. In the step of determining the attention position, object detection processing is executed on the image information to detect an object, the depth of the object, which is the distance between the imaging unit and the object, is specified from the position in the image information of the target area including the detected object, the rate of change in the depth of the object per unit time is calculated, and when a target area including a unit area where the degree of change in luminance is equal to or greater than a threshold value is set as the attention position and the rate of change in the depth in the shallowing direction per unit time is equal to or greater than the threshold value, the position of the object is switched to the attention position 。

Advantages of the Invention

[0009] According to the present disclosure, it is possible to provide an autofocus-adjusting spectacle that can automatically adjust the focal length without sacrificing practicality.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described in detail based on the drawings. Note that the present disclosure is not limited by the embodiments described below.

[0012] (Configuration of Autofocus-Adjusting Spectacles) FIG. 1 is a schematic diagram showing a usage mode of an autofocus-adjusting glasses according to the present disclosure. As shown in FIG. 1, the autofocus-adjusting glasses 10 according to the present disclosure include a depth camera 11, a variable-focus lens 12, a glasses frame 13, and a control device 14. As shown in FIG. 1, the user wears the glasses frame 13 on the ears and visually recognizes a near-distance object TG1 and a far-distance object TG2 through the variable-focus lens 12 from the pupil EY. Here, since the focal length of the variable-focus lens 12 is automatically adjusted under the control of the control device 14, the user can visually recognize both the near-distance object TG1 and the far-distance object TG2.

[0013] The depth camera 11 acquires depth information for each position of the image information while acquiring the image information. That is, the depth camera 11 includes an imaging unit that acquires the image information, and a depth information acquisition unit that acquires depth information for each position of the image information acquired by the imaging unit. Here, the image information is data including luminance and color information of each pixel in one frame, and may be data to which a gradation for each pixel is assigned. The depth information is information indicating the distance from the depth camera 11 of an object imaged in the image information, and the distance from the depth camera 11 is recorded for each position of the image information.

[0014] The imaging unit includes an optical element and an image sensor. The optical element is an element that constitutes an optical system such as a lens, a mirror, a prism, or a filter. The image sensor is an element that converts light incident through the optical element into an image signal that is an electrical signal. The image sensor is, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor.

[0015] The depth information acquisition unit acquires depth information for each position of the image information. The depth information acquisition unit may acquire depth information for each position of the image information using, for example, the TOF (Time Of Flight) method. In this case, the depth information acquisition unit includes a light emitting element and a light receiving element, and measures the distance to an object by measuring the time from when photons emitted from the light emitting element hit the object until the reflected light is received by the light receiving element. As the light emitting element, for example, an LED (Light Emitting Diode), an LD (Laser Diode), a VCSEL (Vertical Cabity Surface Emitting LASER), etc. can be used. Also, as the light receiving element, for example, a CCD sensor, a CMOS sensor, etc. can be used.

[0016] Further, the depth information acquisition unit may input the image information to a learning model that has performed unsupervised learning using, as learning data, data in which the RGB image acquired using a monocular RGB camera, the depth information, the motion vector of the object included in the image information, and the motion vector of the camera are associated with each other. Thereby, when a moving object is imaged in the image information, the depth information of the moving object can be accurately acquired. Also, even if the depth camera 11 that acquires the image information moves, the depth information of the imaged object can be accurately acquired.

[0017] Note that in the present embodiment, the image information acquisition unit and the depth information acquisition unit are configured as integrated hardware as the depth camera 11, but it is not limited thereto, and the image information acquisition unit and the depth information acquisition unit may be separate hardware. Also, the configurations of the image information acquisition unit and the depth information acquisition unit are not limited to the above examples and may be arbitrary configurations.

[0018] The variable focal length lens 12 is a lens whose focal length can be adjusted. Examples of the mode of the variable focal length lens 12 will be described below, but the variable focal length lens 12 is not limited to the following modes.

[0019] (First mode of variable focal length lens) Figure 2 is a schematic diagram of a first aspect of a variable focus lens of an autofocus glasses according to the present disclosure. The variable focus lens 12 includes, for example, a transparent liquid 121, a transparent container 122, and an actuator 123.

[0020] The transparent liquid 121 is a transparent liquid capable of transmitting light. As the transparent liquid 121, for example, pure water or silicone such as polydimethylcyclosilane may be used.

[0021] The transparent container 122 is a flexible transparent container. The transparent container 122 may be formed by bonding two transparent plate-like bodies formed in a cylindrically symmetric shape, a paraboloid, or a cubic surface. By making the transparent container 122 have a cylindrically symmetric shape, the calculation for controlling deformation becomes easy. Note that the transparent liquid 121 is enclosed inside the transparent container 122.

[0022] The actuator 123 is a mechanical element capable of performing telescopic motion. For the actuator 123, a mechanism may be used in which a fluid is injected into a tube made of an elastic body such as rubber to change the pressure of the fluid and thereby expand and contract the tube. Further, as the actuator 123, a piezo actuator using a piezoelectric material that deforms when a voltage is applied may be used. As the piezoelectric material, piezoelectric ceramics or a polymer piezoelectric material (for example, PVDF (polyvinylidene fluoride) or a VDF / TrFE (vinylidene fluoride / trifluoroethylene) copolymer) can be used.

[0023] The actuator 123 is provided at the upper and lower end positions of the transparent container 122. When the actuator 123 expands and contracts, the interval of the transparent container 122 changes, the pressure of the transparent liquid 121 enclosed inside the transparent container 122 changes, and the transparent container 122 elastically deforms. Thereby, the focal length of the variable focus lens 12 can be changed.

[0024] (Second aspect of the variable focus lens) FIG. 3 is a schematic diagram of a second aspect of the variable focus lens of the autofocus glasses according to the present disclosure. As shown in FIG. 3, the variable focus lens may be a liquid crystal variable focus lens 20 including a transparent substrate 21, two outer transparent substrates 22, two liquid crystal lenses 23 provided so as to sandwich the transparent substrate 21, an ultrasonic vibrator 24 disposed on the transparent substrate 21, and a power supply unit 25.

[0025] The transparent substrate 21 may be a glass plate. Note that the material of the transparent substrate 21 is not limited to glass, and for example, it may be a transparent resin material. The thickness of the transparent substrate 21 is, for example, 100 to 900 μm.

[0026] The outer transparent substrate 22 may be a glass plate. One outer transparent substrate 22 is disposed on each of both surfaces of the transparent substrate 21, and a liquid crystal lens 23 described later is disposed between the transparent substrate 21 and the outer transparent substrate 22.

[0027] The liquid crystal lens 23 is composed of an alignment film 231, a liquid crystal layer 232, and a sealing material 233. The alignment film 231 is a vertical alignment film, and aligns liquid crystal molecules so that the long axes of the liquid crystal molecules are perpendicular to the main surface of the alignment film 231 in a state where no ultrasonic wave is generated.

[0028] The liquid crystal layer 232 contains liquid crystal molecules. Examples of the liquid crystal molecules include nematic liquid crystals having a negative dielectric anisotropy. The thickness of the liquid crystal layer 232 is, for example, 30 to 80 μm. By setting the thickness of the liquid crystal layer 232 to 30 μm or more, the variable range of the focal length of the liquid crystal lens 23 can be made sufficiently wide. On the other hand, by setting the thickness of the liquid crystal layer 232 to 80 μm or less, the overall thickness of the liquid crystal lens 23 can be made sufficiently thin.

[0029] The sealing material 233 seals the liquid crystal layer 232 provided between the transparent substrate 21 and the outer transparent substrate 22. The sealing material 233 is provided along the outer periphery of the transparent substrate 21 and the outer transparent substrate 22, and seals the liquid crystal layer 232 therebetween. Examples of the sealing material 233 include epoxy resin.

[0030] The ultrasonic vibrator 24 is disposed on the transparent substrate 21. The ultrasonic vibrator 24 is composed of a vibration part 241 and a pair of electrodes 242 provided so as to sandwich the vibration part 241. The electrode 242 is connected to the power supply unit 25. The ultrasonic vibrator 24 generates ultrasonic waves having a frequency that matches the resonance frequencies of the transparent substrate 21 and the liquid crystal lens 23. By the ultrasonic waves having a frequency that matches the resonance frequency, a flexural vibration in the primary mode is generated. The flexural vibration in the primary mode is a vibration in which the vibration intensity continuously decreases from the center side to the peripheral side of the liquid crystal layer 232. Due to this flexural vibration in the primary mode, an acoustic stationary wave is generated in the liquid crystal layer 232, and further an acoustic radiation force is generated, causing a change in the thickness and orientation of the liquid crystal layer 232. Due to these changes, the focal length of the liquid crystal layer 232 changes.

[0031] The frequency of the ultrasonic waves generated by the ultrasonic vibrator 24 depends on the frequency of the voltage applied from the power supply unit 25. The intensity of the ultrasonic waves of the ultrasonic vibrator 24 depends on the amplitude value of the voltage applied from the power supply unit 25. By setting the frequency of the voltage to a constant value and controlling the amplitude value of the voltage, the degree of change in the refractive index of the liquid crystal layer 232 can be adjusted, and the focal length of the liquid crystal lens 23 can be adjusted. That is, by increasing the amplitude value of the voltage applied to the ultrasonic vibrator 24, the focal length of the liquid crystal lens 23 can be shortened, and on the other hand, by decreasing the amplitude value of the voltage applied, the focal length of the liquid crystal lens 23 can be lengthened.

[0032] The ultrasonic vibrator 24 may be, for example, a piezoelectric ultrasonic vibrator. Examples of the material of the vibration part 241 of the ultrasonic vibrator 24 include ceramics such as lead zirconate titanate (PZT), barium titanate, and lead titanate, and polymers such as polyvinylidene fluoride-trifluoroethylene copolymer (P(VDF-TrFE)).

[0033] The material of the electrode 242 is, for example, silver or nickel. The electrode 242 is connected to a power supply unit 25 capable of applying a voltage. The electrode 242 may be constituted by, for example, a silver electrode formed by high-temperature baking or a nickel electrode formed by chemical plating. The electrode 242 is electrically connected to the power supply unit 25 by wiring.

[0034] For the ultrasonic vibrator 24, the frequency of the voltage applied from the power supply unit 25 can be calculated by performing a simulation using the physical property values of the transparent substrate 21 and the liquid crystal lens 23 (for example, Young's modulus, Poisson's ratio, density, etc. of the transparent substrate 21) and the physical property values of the ultrasonic vibrator 24 (for example, elastic constant matrix, density, and piezoelectric constant matrix of the ultrasonic vibrator 24).

[0035] The glasses frame 13 is a member to which the depth camera 11, the variable-focus lens 12, and a control device 14 described later are attached, and which has temples and enables the glasses to be worn on the user's ears. Note that the control device 14 is not limited to being attached to the glasses frame 13, and may be a control device 14 provided outside the glasses frame 13 and connected via a wireless or electrical communication line.

[0036] (Regarding the control device) FIG. 4 is a block diagram showing a configuration example of the control device of the autofocus glasses according to the present disclosure. The control device 14 is a computer in the present embodiment, and includes a storage unit 141 and a control unit 142. The storage unit 141 is a memory that stores various information such as the calculation content and programs of the control unit 142, and includes at least one of, for example, a main storage device such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive). The program for the control unit 142 stored in the storage unit 141 may be stored in a recording medium readable by the control unit 142.

[0037] The control unit 142 is an arithmetic unit and includes an arithmetic circuit such as a CPU (Central Processing Unit). The control unit 142 includes a target position determination unit 1421, a depth identification unit 1422, and a focal length control unit 1423. As shown in FIG. 4, the depth camera 11 and the control device 14 are connected, and image information and depth information are transmitted from the depth camera 11 to the control device 14. Also, as shown in FIG. 4, a variable focal length lens 12 is connected to the control device 14, and a control signal from the control device 14 is transmitted to the variable focal length lens 12. The control unit 142 reads and executes a program (software) from the storage unit 141 to realize the target position determination unit 1421, the depth identification unit 1422, and the focal length control unit 1423, and execute their processes. Note that the control unit 142 may execute these processes by one CPU, or may include a plurality of CPUs and execute the processes with these plurality of CPUs. Also, at least one of the target position determination unit 1421, the depth identification unit 1422, and the focal length control unit 1423 may be realized by a hardware circuit.

[0038] The target position determination unit 1421 determines the target position in the image information acquired by the depth camera 11. The target position is the position in the image information estimated to be the one at which the user of the autofocus glasses 10 looks in the image information acquired by the depth camera 11. For example, in the image information acquired by the depth camera 11, when an object with a large degree of change in luminance per unit area in the image information such as characters is captured, it may be estimated that the user looks at this, and the position in the image information of this may be set as the target position. This is because a person has a property of gazing at a specific object such as characters, and the specific object tends to have a large degree of change in luminance. Note that the target position determination unit 1421 is not limited to setting the position in the image information of an object with a large degree of change in luminance such as characters as the target position, and any other position may be set as the target position.

[0039] (First mode of processing of target position determination unit) FIG. 5 is a flowchart showing a first aspect of the processing of the attention position determination unit. The attention position determination unit 1421 calculates the degree of change in luminance within a unit area of the image in the image information acquired by the depth camera 11, and sets, as the attention position, a target area including a unit area in which the degree of change in luminance is equal to or greater than a threshold value. Hereinafter, the first aspect of the processing of the attention position determination unit 1421 will be described with reference to FIG. 5.

[0040] The attention position determination unit 1421 executes object detection processing on the image information and sets a target area that means an area including the detected object (step S100). Here, the target area is an area occupying a certain range in the image information obtained by executing object detection processing on the image information and predicting a rectangle called a bounding box including the specified object. That is, the target area is an area including an object in the image information. Note that object detection is performed by acquiring windows of a fixed size from the image information at all possible positions and inputting these areas into an image classifier. Specifically, for object detection, for example, a method such as R-CNN (Region-based Convolutional Neural Network) can be used. R-CNN performs object detection from the image information in the following procedure. First, selective-search, which is a method for finding object-likeness, is used to search for region candidates from the image information. All the region images of the region candidates are resized to a certain size and input into a CNN (Convolutional Neural Network) to extract feature amounts. Using the extracted feature amounts, a learning model is generated using a plurality of SVMs (Support Vector Machine), and by inputting the image information into the learning model, the position of the bounding box including the object in the image information is estimated. Thereby, object detection can be performed from the image information while reducing the computational load of the CNN.

[0041] Once the target area is extracted from the image information, the attention position determination unit 1421 calculates the degree of change in luminance within the unit area of the image in the image information (step S110). The unit area is an area that occupies a certain range in an arbitrarily set image. For example, assume that the image is a 4K image with 3840 pixels vertically and 2160 pixels horizontally. In this case, the attention position determination unit 1421 may, for example, divide the image vertically into 384 parts and horizontally into 216 parts, and use an area of 10 pixels vertically and 10 pixels horizontally as the unit area. Once the image information is divided into unit areas, the attention position determination unit 1421 calculates the degree of change in luminance within each unit area of the image in the image information. The attention position determination unit 1421 may, for example, use the amount of variation in the luminance distribution within the unit area of the image as the degree of change in luminance. The amount of variation is a value indicating the spread of the luminance distribution. For example, the degree of change in luminance within the unit area may be obtained using statistical summary quantities for the luminance distribution such as standard deviation, variance, maximum value - minimum value, interquartile range, percentile value, etc.

[0042] Once the degree of change in luminance within the unit area of the image in the image information is calculated, the attention position determination unit 1421 determines whether the degree of change in luminance within each unit area is equal to or greater than the first threshold value, and selects the unit areas whose degree of change in luminance is equal to or greater than the first threshold value (step S120). Here, the first threshold value may be set arbitrarily.

[0043] Once the unit areas whose degree of change in luminance within the unit area of the image in the image information is equal to or greater than the first threshold value are selected, the attention position determination unit 1421 sets the target area within the target area that contains the unit areas whose degree of change in luminance is equal to or greater than the first threshold value and has the highest ratio of such unit areas as the attention position (step S130). Here, since the target area is an area that occupies a certain range in the image information, the attention position may be any position within the target area. For example, the position at the center of the target area may be set as the attention position. Note that when there is only one target area that contains the unit areas whose degree of change in luminance is equal to or greater than the first threshold value, the process of selecting the one with the maximum ratio of unit areas as in step S130 becomes unnecessary.

[0044] (Second Aspect of the Processing of the Attention Position Determination Unit) FIG. 6 is a flowchart showing a second aspect of the processing of the attention position determination unit. When a target area including a unit area with a luminance change degree equal to or greater than a threshold value is located in the central area of the image, the attention position determination unit 1421 sets the target area as the attention position. That is, in the first aspect, when there are a plurality of target areas including unit areas with a luminance change degree equal to or greater than the first threshold value, the target area with the largest ratio of unit areas with a luminance change degree equal to or greater than the first threshold value is set as the attention position. However, in the second aspect, it is different from the first aspect in that the target area located in the central area is set as the attention position. Hereinafter, the second aspect of the processing of the attention position determination unit 1421 will be described with reference to FIG. 6.

[0045] The description of the second aspect of the processing of the attention position determination unit 1421 is omitted from step S200 to step S220 because it is the same as the processing from step S100 to step S120 of the first aspect. In step S220, when a unit area with a luminance change degree equal to or greater than the first threshold value is selected, the attention position determination unit 1421 identifies a target area in which the ratio of unit areas with a luminance change degree equal to or greater than the first threshold value is equal to or greater than the second threshold value (step S230). Here, the second threshold value may be set to any value. For example, it may be set to 60%.

[0046] When the attention position determination unit 1421 identifies a target area in which the ratio of unit areas with a luminance change degree equal to or greater than the first threshold value is equal to or greater than the second threshold value, and if the target area in which the ratio of unit areas with a luminance change degree equal to or greater than the first threshold value is equal to or greater than the second threshold value is located in the central area, the attention position determination unit 1421 sets the central area as the attention position (step S240). Here, the central area is an area occupying a certain range including the central position in the image information. For example, when the image information is a 4K image, a range of 50 pixels to the left and right and 50 pixels up and down from the central position may be used as the central area. Also, when the size of the target area is larger than the central area, the central area may be set as the attention position when a certain range of the target area, for example, 60%, is included in the central area.

[0047] Accordingly, when an object with a large degree of change in luminance such as characters is imaged in the central region of an image that is considered to be easily noticeable to the user, the central region can be set as the attention position. That is, when an object that the user pays attention to, such as characters, is imaged in the central region of the image, the focal length can be adjusted to the depth of the object located in the central region. Thus, in the second aspect, when there are a plurality of target regions including unit regions where the degree of change in luminance is equal to or greater than the first threshold, if the ratio occupied by the unit regions is equal to or greater than the second threshold and the target region is located in the central region, it is set as the attention position. By providing the second threshold in this way, for example, a target region located in the central region where the ratio occupied by unit regions with a degree of change in luminance equal to or greater than the first threshold is low can be excluded from the attention position, and another target region where the ratio occupied by unit regions with a degree of change in luminance equal to or greater than the first threshold is high can be set as the attention position, which is preferable. However, in the second aspect, it is not limited to setting the second threshold. For example, among the target regions including unit regions where the degree of change in luminance is equal to or greater than the first threshold, the target region located in the central region may be set as the attention position. Also, the second aspect is applicable even when there is one target region including a unit region where the degree of change in luminance is equal to or greater than the first threshold. In this case, for example, if the single target region is in the central region, it is set as the attention position, and if the single target region is not in the central region, it may not be necessary to set the attention position.

[0048] (Third aspect of the processing of the attention position determination unit) FIG. 7 is a flowchart showing the third aspect of the processing of the attention position determination unit. The attention position determination unit 1421 executes object detection processing on the image information to detect an object, specifies the depth of the object from the position in the image information of the target region including the detected object, calculates the rate of change in the depth of the object per unit time, and when the rate of change in depth per unit time is equal to or greater than the threshold, sets the position of the object as the attention position. Hereinafter, the third aspect of the processing of the attention position determination unit 1421 will be described with reference to FIG. 7.

[0049] The attention position determination unit 1421 executes object detection processing on the image information to set a target region that means a region including the detected object (step S300). Since the object detection processing is the same as step S100 in the first mode of the processing of the attention position determination unit 1421, the description thereof is omitted.

[0050] When the target region is set, the attention position determination unit 1421 specifies the depth of the target region from the position of the target region in the image (step S310). The specification of the depth of the target region is executed by a depth specification unit 1422 described later based on the position in the image information of the target region. The attention position determination unit 1421 inputs the position in the image information of the target region to the depth specification unit 1422 and acquires the depth output from the depth specification unit 1422. The processing of the depth specification unit 1422 will be described later.

[0051] When the depth of the target region is specified, the attention position determination unit 1421 calculates the change rate per unit time of the depth of the target region (step S320). The attention position determination unit 1421 calculates the change rate of the depth per unit time from the depth acquired for each frame of the target region. That is, when the frame rate of the image information is 30 fps, the change rate of the depth per 1 / 30 second is calculated.

[0052] When the change rate of the depth per unit time of the target region is calculated, the attention position determination unit 1421 sets, as the attention position, a target region in which the change rate of the depth per unit time is equal to or greater than a third threshold value (step S330). Here, any value may be set as the third threshold value. For example, it may be set to 60 km / hr. Note that the change rate of the depth here may indicate that the depth becomes shallower, in other words, the object approaches.

[0053] Accordingly, when an object approaching the user at high speed is imaged in the image information, the position of the object can be set as the attention position. Then, based on the position of the object set as the attention position, the depth of the object is specified by a depth specifying unit 1422 described later, and by adjusting the focal length of the variable focal lens 12, the focal length can be adjusted to the object approaching at high speed. Note that the attention position determination unit 1421 may implement at least one of the identification of the attention position according to the first to third aspects described above. Further, the attention position determination unit 1421 may combine the identification of the attention position according to the first to third aspects. That is, for example, the attention position determination unit 1421 sets the central region as the attention position according to the second aspect and adjusts the focal length of the variable focal lens 12. Then, when the change rate of the depth of the object in another target region becomes equal to or greater than a third threshold value, that is, when the object is approaching, the attention position may be switched to the target region.

[0054] (Other aspects of the processing of the attention position determination unit) Further, the attention position determination unit 1421 is not limited to setting the attention position according to the first to third aspects. The attention position determination unit 1421 sets a target region in which a preset object exists as the attention position in the image information. More specifically, for example, the preset object is a character or the like, but is not limited thereto. The attention position determination unit 1421 reads information about the preset object from the storage unit 141. The attention position determination unit 1421 performs pattern matching on the image information acquired by the depth camera 11 based on the information about the preset object to identify the character. The attention position determination unit 1421 sets the target region in which the identified character exists as the attention position in the image information. Furthermore, the attention position determination unit 1421 may detect a target region in which a preset object exists by the object detection process described in the first aspect, and set the target region as the attention position.

[0055] In addition, when the attention position determination unit 1421 performs pattern matching or object detection processing on the image information acquired by the depth camera 11 and there is text in a target area that is separated by a predetermined number of pixels or more, it separately identifies each target area where the text exists. When two or more target areas where text exists are identified, the attention position determination unit 1421 may set the target area located closer to the center area of the image as the attention position. As a result, since a person tends to center an object that they want to focus on in the center of their field of view, an object that is considered to be an object that a person wants to focus on can be selected.

[0056] The depth identification unit 1422 identifies the depth of the attention position based on the attention position in the image information determined by the attention position determination unit 1421 and the depth information recorded for each position of the image information. As described above, the depth information acquired by the depth camera 11 records the depth of the object imaged in the image information for each position of the image information.

[0057] Here, prior to the explanation of the processing of the depth identification unit 1422, the image information and the depth information will be described with reference to FIGS. 8 and 9. FIG. 8 is a schematic diagram showing the image information acquired by the imaging unit of the autofocus adjustment glasses according to the present disclosure. FIG. 8 shows, as an example, an image in which a near-distance object TG1 and a far-distance object TG2 are imaged. FIG. 9 is a schematic diagram visually showing the depth information acquired by the depth information acquisition unit of the autofocus adjustment glasses according to the present disclosure. In FIG. 9, an object that is far from the depth camera 11 is displayed in a light color, and an object that is close to the depth camera 11 is displayed in a dark color. As shown in FIG. 9, the depth information records the depth of the object imaged in the image information for each position of the image information.

[0058] Therefore, if the position in the image information of the object imaged in the image information is known, its depth can be identified. The depth identification unit 1422 identifies the depth of the attention position from the depth information acquired by the depth camera 11 based on the attention position in the image information determined by the attention position determination unit 1421.

[0059] The focal length control unit 1423 sets the focal length of the variable focal lens 12 based on the depth of the target position. That is, the focal length control unit 1423 calculates the focal length of the lens system of the autofocus glasses 10 based on the depth of the target position in the image information, generates a control signal for adjusting the focal length of the variable focal lens 12 based on this, and transmits the control signal to the variable focal lens 12. The variable focal lens 12 is controlled by the control signal generated by the focal length control unit 1423, and the depth of the target position is set as the focal length of the variable focal lens 12.

[0060] (Configuration and Effect of Autofocus Glasses) The autofocus glasses 10 according to the present disclosure include an imaging unit capable of acquiring image information, a variable focal lens 12 whose focal length can be changed, a depth information acquisition unit that acquires depth information for each position of the image information acquired by the imaging unit, a target position determination unit 1421 that determines the target position in the image information acquired by the imaging unit, a depth specification unit 1422 that specifies the depth of the target position based on the target position in the image information determined by the target position determination unit 1421 and the depth information, and a focal length control unit 1423 that sets the focal length of the variable focal lens 12 based on the depth of the target position.

[0061] According to this configuration, the target position in the image information can be determined, and the focal length of the variable focal lens 12 can be set based on the depth of the target position. Therefore, it is possible to provide the autofocus glasses 10 capable of automatically adjusting the focal length without sacrificing practicality.

[0062] The target position determination unit 1421 calculates the degree of change in luminance within a unit area of the image in the image information acquired by the imaging unit, and sets a target area including a unit area where the degree of change in luminance is equal to or greater than a threshold value as the target position.

[0063] According to this configuration, among the target areas in which an object is detected in the image information, a target area including a unit area where the degree of change in luminance is equal to or greater than a threshold value is set as the target position, and the focal length of the variable focal lens 12 can be set to match the depth of the target position.

[0064] When the target area including the unit area where the degree of change in luminance is equal to or greater than the threshold is located in the central area of the image, the attention position determination unit 1421 sets the target area as the attention position.

[0065] According to this configuration, among the target areas in which object detection has been performed in the image information, when the target area including the unit area where the degree of change in luminance is equal to or greater than the threshold is located in the central area of the image, the target area is set as the attention position, and the focal length of the variable focal length lens 12 can be set according to the depth of the attention position.

[0066] The attention position determination unit 1421 performs object detection processing on the image information to detect an object, specifies the depth of the object from the position in the image information of the target area including the detected object, calculates the change rate of the depth of the object per unit time, and sets the position of the object as the attention position when the change rate of the depth per unit time is equal to or greater than the threshold.

[0067] According to this configuration, when a target area including an object with a depth change rate equal to or greater than the threshold is imaged in the image information, the target area is set as the attention position, and the focal length of the variable focal length lens 12 can be set according to the depth of the attention position. Therefore, for example, when there is an object approaching the user, the focal length of the variable focal length lens 12 can be adjusted to match the approaching object.

[0068] A control method for the autofocus adjustment glasses 10 according to the present disclosure is a control method for the autofocus adjustment glasses 10 including an imaging unit capable of acquiring image information and a variable focal length lens 12 capable of changing the focal length, the method including: acquiring depth information for each position of the image information acquired by the imaging unit; determining an attention position in the image information acquired by the imaging unit; specifying the depth of the attention position based on the attention position and the depth information in the image information; and setting the focal length of the variable focal length lens 12 based on the depth of the attention position.

[0069] According to this configuration, it is possible to provide a control method for an autofocus adjustment glasses 10 that can automatically adjust the focal length without sacrificing practicality.

[0070] The program according to the present disclosure is a program that causes a computer to execute a control method for an autofocus adjustment glasses 10 including an imaging unit capable of acquiring image information and a variable focus lens 12 capable of changing the focal length, the method including: obtaining depth information for each position of the image information acquired by the imaging unit; determining a target position in the image information acquired by the imaging unit; specifying the depth of the target position based on the target position in the image information and the depth information; and setting the focal length of the variable focus lens based on the depth of the target position, and causing the computer to execute these steps.

[0071] According to this configuration, it is possible to provide an autofocus adjustment program that can automatically adjust the focal length without sacrificing practicality.

[0072] As described above, embodiments of the present invention have been described, but the embodiments are not limited by the content of this embodiment. Further, the above-described components include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within the so-called equivalent range. Furthermore, the above-described components can be combined as appropriate. Furthermore, various omissions, substitutions, or changes of the components can be made without departing from the gist of the above-described embodiments.

Description of Reference Numerals

[0073] 10 Autofocus adjustment glasses 11 Depth camera 12 Variable focus lens 13 Glasses frame 14 Control device 141 Storage unit 142 Control unit 1421 Target position determination unit 1422 Depth specification unit 1423 Focal length control unit

Claims

1. An imaging unit capable of acquiring image information, A variable focal length lens capable of changing the focal length, A depth information acquisition unit that acquires depth information for each position of the image information acquired by the imaging unit, A target position determination unit that determines a target position in the image information acquired by the imaging unit, A depth specification unit that specifies the depth of the target position based on the target position in the image information determined by the target position determination unit and the depth information, A focal length control unit that sets the focal length of the variable focal length lens based on the depth of the target position, Comprising, The target position determination unit calculates the degree of change in luminance within a unit area of the image in the image information acquired by the imaging unit, and sets a target area including a unit area where the degree of change in luminance is equal to or greater than a first threshold value as the target position, Performs object detection processing on the image information to detect an object, specifies the depth of the object, which is the distance between the imaging unit and the object, from the position in the image information of the target area including the detected object, calculates the rate of change in the depth of the object per unit time, and when setting a target area including a unit area where the degree of change in luminance is equal to or greater than a first threshold value as the target position and the rate of change in depth in the shallowing direction per unit time is equal to or greater than a third threshold value, switches the position of the object to the target position, When a target area where the ratio of the unit area where the degree of change in luminance is equal to or greater than a first threshold value is equal to or greater than a second threshold value is located in the central area of the image, sets the target area as the target position, An auto-focus adjustment glasses.

2. A control method for an auto-focus adjustment glasses comprising an imaging unit capable of acquiring image information and a variable focal length lens capable of changing the focal length, A step of acquiring depth information for each position of the image information acquired by the imaging unit, A step of determining a target position in the image information acquired by the imaging unit, A step of specifying the depth of the target position based on the target position in the image information and the depth information, A step of setting the focal length of the variable focal length lens based on the depth of the target position, Including, In the step of determining the target position, the degree of change in luminance within a unit area of the image in the image information acquired by the imaging unit is calculated, and a target area including a unit area where the degree of change in luminance is equal to or greater than a first threshold value is set as the target position, Performing object detection processing on the image information to detect an object, specifying the depth of the object, which is the distance from the imaging unit to the object, from the position in the image information of the target region including the detected object, calculating the change rate of the depth of the object per unit time, and when setting a target region including a unit region where the degree of change in luminance is equal to or greater than a first threshold as the attention position, and when the change rate of the depth in the shallower direction per unit time is equal to or greater than a third threshold, switching the position of the object to the attention position, When a target region where the ratio of unit regions where the degree of change in luminance is equal to or greater than the first threshold is equal to or greater than the second threshold is located in the central region of the image, setting that target region as the attention position. A method for controlling an autofocus glasses.

3. A program for causing a computer to execute a method for controlling an autofocus glasses including an imaging unit capable of acquiring image information and a variable-focus lens capable of changing the focal length, a step of acquiring depth information for each position of the image information acquired by the imaging unit, a step of determining an attention position in the image information acquired by the imaging unit, a step of specifying the depth of the attention position based on the attention position in the image information and the depth information, and a step of setting the focal length of the variable-focus lens based on the depth of the attention position, In the step of determining the attention position, calculating the degree of change in luminance within a unit region of the image in the image information acquired by the imaging unit, and setting a target region including a unit region where the degree of change in luminance is equal to or greater than a first threshold as the attention position, Performing object detection processing on the image information to detect an object, specifying the depth of the object, which is the distance from the imaging unit to the object, from the position in the image information of the target region including the detected object, calculating the change rate of the depth of the object per unit time, and when setting a target region including a unit region where the degree of change in luminance is equal to or greater than a first threshold as the attention position, and when the change rate of the depth in the shallower direction per unit time is equal to or greater than a third threshold, switching the position of the object to the attention position, When a target region where the ratio of unit regions where the degree of change in luminance is equal to or greater than the first threshold is equal to or greater than the second threshold is located in the central region of the image, setting that target region as the attention position. A program for causing a computer to execute.

Images