Method and apparatus for generating strabismus angle information

Abstract

The present specification discloses a method and an apparatus for generating strabismus angle information. The method for generating strabismus angle information according to the present specification may involve: receiving image data, captured of an optotype, from an RGB camera as input, and calculating coordinates for the optotype; storing a plurality of measurement values, for the gaze direction of an eyeball having an open field of view, from an eyeball tracking sensor; storing the timing at which one eyeball having an open field of view gazes at the optotype; and calculating the strabismus angle of a subject for each eyeball from the depth sensor by using information about the distance between the subject and the optotype, the timing information, and the gaze tracking information.

Description

Method and device for generating squint information

[0001] The present invention relates to a method and device for generating angle-of-view information, and more particularly, to a method and device for generating angle-of-view information using augmented reality equipment.

[0002] This application claims priority to Korean Patent Application No. 10-2023-0147216, filed in the Republic of Korea on October 30, 2023, the entire disclosure of which is incorporated herein by reference.

[0003] The material described in this section merely provides background information on the embodiments described herein and does not necessarily constitute prior art.

[0004] Traditionally, the prism cover test was used to examine strabismus. The prism cover test involves having the subject look at a target and placing a prism over one eye. The prism's power is then increased while alternately covering both eyes until the eye where the prism is placed stops moving. The prism power when the eye stops moving indicates the degree of strabismus. Prisms used in the prism cover test are available in glass and plastic. The method for positioning the prism for the prism cover test varies depending on the material, and the accuracy of prism positioning varies depending on the examiner's skill. Consequently, errors may occur in measuring the strabismus angle using the prism cover test.

[0005] Recently, along with virtual environment technology, technologies for testing strabismus using virtual reality (VR) equipment are being developed. Subjects wear VR equipment and view a virtual test chart displayed on the VR equipment's display device. However, this display device is a virtual representation of the test distance, which differs from the actual test distance, making it difficult to accurately measure strabismus based on the test distance.

[0006] The purpose of this specification is to provide a method and device for generating angle information.

[0007] This specification is not limited to the above-mentioned tasks, and other tasks not mentioned will be clearly understood by those skilled in the art from the description below.

[0008] In order to solve the above-described problem, a method for generating strabismus information according to the present specification is a method for generating strabismus information using a device including an RGB camera, a depth sensor, and an eye tracking sensor in a housing of a predetermined shape mounted on a head of a subject, the method comprising: (a) a step in which a processor calculates coordinates for a target from image data of a target of a predetermined shape input from the RGB camera; (b) a step in which the processor receives from the eye tracking sensor a plurality of measurement values ​​for the gaze direction of one eye of the subject whose field of vision is open and stores them; (c) a step in which the processor stores the timing (hereinafter, “timing information”) in which one eye of the subject whose field of vision is open looks at the target; and (d) a step in which the processor calculates a strabismus angle of the subject for each eye by using distance information between the subject and the target, the timing information, and the gaze tracking information, which are input from the depth sensor.

[0009] According to one embodiment of the present specification, the housing further includes a field of view blocking device for individually blocking the field of view of both eyes of the subject; and the step (b) may be characterized in that the processor outputs a blocking signal to the field of view blocking device to block the field of view of one eye of the subject for a predetermined period of time.

[0010] According to one embodiment of the present specification, the housing further includes an inertial measurement device that measures an angle of the head of the subject; and the step (c) may be a step in which the processor receives at least one of angle information of the head of the subject from the inertial measurement device, information of the target image from an RGB camera, and field of view blocking information from the field of view blocking device, and stores the timing information.

[0011] According to one embodiment of the present specification, the step (d) may be a step in which the processor calculates an angle between the subject's gaze direction and the up / down and left / right directions formed by the target.

[0012] The method for generating angle information according to the present specification may further include, after step (d), a step (e) in which a processor selects an area output with the highest frequency among a plurality of preset angle areas as a mode and converts it into a prism diopter.

[0013] The method for generating angle information according to the present specification can be implemented in the form of a computer program written to perform each step on a computer and recorded on a computer-readable recording medium.

[0014] A device for generating information on a strabismus according to the present specification may include: a housing having a predetermined shape, which is mounted on the head of a subject and includes an RGB camera, a depth sensor, and an eye tracking sensor; a coordinate calculation unit for calculating coordinates for a target from image data of a target of a predetermined shape input from the RGB camera; a gaze information storage unit for receiving, from the eye tracking sensor, a plurality of measurement values ​​for a gaze direction of one eye with an open field of vision among the subject's two eyes; a timing storage unit for storing a timing (hereinafter, "timing information") at which one eye with an open field of vision among the subject's two eyes gazes at a target; and a strabismus calculation unit for calculating a strabismus of the subject for each eye by using distance information between the subject and the target, which is input from the depth sensor, the timing information, and the gaze tracking information.

[0015] According to one embodiment of the present specification, the housing further includes a field of view blocking device for individually blocking the field of view of both eyes of the subject; and the gaze information storage unit may be characterized in that it outputs a blocking signal to the field of view blocking device to block the field of view of one eye of the subject for a predetermined period of time.

[0016] According to one embodiment of the present specification, the housing further includes an inertial measurement device that measures an angle of a head of a subject; and the timing storage unit can receive at least one of angle information of the head of the subject from the inertial measurement device, information of the target image from an RGB camera, and field of view blocking information from the field of view blocking device, and store the timing information.

[0017] According to one embodiment of the present specification, the angle calculation unit can calculate the angle between the subject's gaze direction and the up / down and left / right directions of the target.

[0018] The device for generating angle information according to the present specification may further include a diopter conversion unit that selects an area output with the highest frequency among a plurality of preset angle areas as a mode value and converts it into a prism diopter.

[0019] A device for generating angle information according to the present specification may be a component of a system for generating angle information, including: a plurality of indicators positioned at predetermined locations; a computing device for outputting an operation signal of the device for generating angle information and receiving angle information; and a communication device for transmitting and receiving signals or information between the device for generating angle information and the computing device.

[0020] Other specific details of the present invention are included in the detailed description and drawings.

[0021] According to one aspect of the present specification, an error occurring when a prism is placed in front of one eye of a subject in a conventional prism occlusion test can be eliminated.

[0022] According to another aspect of the present specification, the problem of narrowing of the field of view occurring in strabismus examination using conventional VR equipment can be solved.

[0023] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0024] FIG. 1 is a block diagram of an eye movement ability information generation device according to one embodiment of the present specification.

[0025] Figure 2 is an example image of conducting an eye movement ability test.

[0026] FIG. 3 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0027] FIG. 4 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0028] FIG. 5 is an example image of items to be quantified to evaluate eye movement ability according to one embodiment of the present specification.

[0029] FIG. 6 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0030] Figure 7 is an example image of a graph according to the gaze positions of the subject's left and right eyes.

[0031] FIG. 8 is a schematic image of an eye movement ability information generation system according to one embodiment of the present specification.

[0032] Figure 9 is an example image showing the linkage between an eye movement ability information generation device and a computing device.

[0033] Fig. 10 is a block diagram of a perspective information generating device according to one embodiment of the present specification.

[0034] Figure 11 is an example image of performing a strabismus measurement test.

[0035] Fig. 12 is a block diagram of a perspective information generating device according to another embodiment of the present specification.

[0036] Fig. 13 is a block diagram of a perspective information generating device according to another embodiment of the present specification.

[0037] Fig. 14 is a block diagram of a perspective information generating device according to another embodiment of the present specification.

[0038] FIG. 15 is a schematic image of a perspective information generation system according to one embodiment of the present specification.

[0039] Figure 16 is an example image showing the linkage between a visual information generation device and a computing device.

[0040] Fig. 17 is a block diagram of a device for generating ocular movement ability information and strabismus information according to one embodiment of the present specification.

[0041] FIG. 18 is a schematic structural diagram of software linked to an ocular movement ability information and strabismus information generation device according to one embodiment of the present specification.

[0042] Fig. 19 is a flowchart of a method for generating eye movement ability information according to one embodiment of the present specification.

[0043] FIG. 20 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0044] FIG. 21 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0045] Fig. 22 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0046] Fig. 23 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0047] Figure 24 is a flowchart of a method for generating angle information according to one embodiment of the present specification.

[0048] Fig. 25 is a flowchart of a method for generating angle information according to another embodiment of the present specification.

[0049] FIG. 26 is a flowchart of a method for generating angle information according to another embodiment of the present specification.

[0050] Figure 27 is a flowchart of a method for generating angle information according to another embodiment of the present specification.

[0051] The advantages and features of the invention disclosed in this specification, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, this specification is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of this specification is complete and to fully inform those of ordinary skill in the art (hereinafter referred to as "skilled workers") of the scope of this specification, and the scope of rights of this specification is defined only by the scope of the claims.

[0052] The terminology used herein is for the purpose of describing embodiments and is not intended to limit the scope of the present disclosure. In this specification, singular forms also include plural forms, unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0053] Throughout the specification, the same reference numerals refer to the same elements, and the term "and / or" includes each and every combination of the elements mentioned. Although terms such as "first," "second," etc. are used to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another. Therefore, it should be understood that a first element mentioned below may also be a second element within the technical scope of the present invention.

[0054] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those skilled in the art to which this specification pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0055] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0056] FIG. 1 is a block diagram of an eye movement ability information generation device according to one embodiment of the present specification.

[0057] Referring to FIG. 1, an eye movement ability information generating device (1) according to one embodiment of the present specification may include a housing (10), a coordinate calculation unit (20), a gaze information storage unit (30), a timing storage unit (40), and a movement ability calculation unit (50).

[0058] The housing (10) may be mounted on the subject's head. The housing (10) may be a head-mounted display (HMD) of an augmented reality device. The housing (10) may be any one of smart glasses, smart goggles, or a helmet-mounted display, which are merely examples and are not limited by a specific shape.

[0059] The housing (10) may include an RGB camera (11), a depth sensor (12), and an eye tracking sensor (13). The RGB camera (11) may capture an ocular movement ability test target and store it as image data. The depth sensor (12) may output distance information from a subject wearing the housing (10) to the target. The eye tracking sensor (13) may track the direction of the subject's eyes when the subject looks at a target presented on the target. The eye tracking sensor (13) may be an infrared camera, which is merely an example and is not limited by specific equipment.

[0060] Figure 2 is an example image of conducting an eye movement ability test.

[0061] Referring to Fig. 2, a subject wearing the housing (10) is looking at the target (2). The target (2) may be any one of a Hess screen chart, a Harms screen chart, and / or a Lancaster screen chart, which is an example and is not limited to a specific target.

[0062] The RGB camera (11) can capture the target (2) and store it as image data. The coordinate calculation unit (20) can receive image data of the target (2) from the RGB camera (11) and calculate coordinates for a plurality of grid points in the image of the target (2).

[0063] To calculate the above coordinates, line detection, grid search, edge detection algorithms, and / or deep learning models may be used, which are only examples and are not limited to a specific method.

[0064] To test eye movement ability, the examiner may present a target at any one of the grid points for the examinee to fixate on.

[0065] According to one embodiment of the present specification, the examiner may present the target in a predetermined order at the grid points of the target (2). The target may be presented first at the center point of the target (2) (hereinafter referred to as the "first grid point"). Next, the target may be presented at the grid point to the right of the center point (hereinafter referred to as the "second grid point"). Next, the target may be presented at the grid point at the upper right of the center point (hereinafter referred to as the "third grid point"). Next, the target may be presented at the grid point at the lower right of the center point (hereinafter referred to as the "fourth grid point"). Next, the target may be presented at the grid point below the center point (hereinafter referred to as the "fifth grid point"). Next, the target may be presented at the grid point above the center point (hereinafter referred to as the "sixth grid point"). Next, the target may be presented at the grid point at the upper left of the center point (hereinafter referred to as the "seventh grid point"). Next, the target may be presented at the grid point to the left of the central point (hereinafter referred to as the "eighth grid point"). Next, the target may be presented at the grid point to the lower left of the central point (hereinafter referred to as the "ninth grid point"). This is merely an example and is not limited by the above order.

[0066] The above target can be presented by the examiner using a laser pointer on the target (2). Alternatively, the examiner can verbally command the examinee to look at a specific grid point. Alternatively, the target (2) can include a light-emitting device at the grid point, and light can be emitted at each grid point in a predetermined order. Alternatively, a target generating device can be used to generate a target at a specific grid point. The target generating device can be an optical device that projects a specific marker onto each grid point. This is an example and is not limited to a specific method.

[0067] The subject can sequentially gaze at targets presented at the grid points while blocking the field of vision of one of the two eyes. The subject can gaze at the targets for a predetermined period of time. At this time, the eye tracking sensor (13) can track the direction of the subject's gaze and calculate a measurement value for the gaze direction. The gaze information storage unit (30) can store the measurement value for each of the grid points. At this time, the gaze information storage unit (30) can store a measurement value for the gaze direction of the eye whose field of vision is blocked among the two eyes of the subject. When the subject completes the examination for one eye, the measurement value for the other eye can be stored.

[0068] When the subject looks at the target, the timing storage unit (40) can store the timing (hereinafter, 'timing information') at which the subject looks at the target with an eye whose field of vision is not blocked.

[0069] The above motor ability calculation unit (50) can receive the timing information and gaze information measurement values. In addition, the motor ability calculation unit (50) can receive distance information between the subject and the target (2) from the depth sensor (12). The motor ability calculation unit (50) can use the timing information to calculate the gaze error of the subject using the gaze information measurement values ​​and the distance information for each grid point corresponding to a specific timing.

[0070] FIG. 3 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0071] Referring to FIG. 3, an eye movement ability information generating device (1-1) according to another embodiment of the present specification may include a housing (10-1) including an RGB camera (11), a depth sensor (12), an eye tracking sensor (13), and a microphone (14), a coordinate calculation unit (20), a gaze information storage unit (30), a timing storage unit (40), and a movement ability calculation unit (50). Since the RGB camera (11), the depth sensor (12), the eye tracking sensor (13), the coordinate calculation unit (20), the gaze information storage unit (30), the timing storage unit (40), and the movement ability calculation unit (50) have been described above, a repeated description thereof will be omitted.

[0072] The above housing (10-1) can receive an audio signal through the microphone (14). The timing storage unit (40) can store the timing information using the audio signal or image information input from the RGB camera (11).

[0073] For example, the examiner may verbally command the subject to look at the first grid point. At this time, the microphone (14) may receive the command and output an audio signal to the timing storage unit (40). When the timing storage unit (40) receives the audio signal, the timing storage unit (40) may start recording timing information for the first grid point. Thereafter, the examiner may verbally command the subject to look at the second grid point. When the timing storage unit (40) receives the audio signal to look at the second grid point, the timing storage unit (40) may end recording timing information for the first grid point and start recording timing information for the second grid point. Since the method of storing timing information for the third to ninth grid points is the same, a repeated description will be omitted.

[0074] Thereafter, the examiner can command the subject, such as, "Cover the other eye." The timing storage unit (40) can begin recording timing information for the other eye upon receiving the audio signal. Since the method for storing timing information for the first to ninth grid points is the same, a repeated explanation will be omitted.

[0075] The above timing storage unit (40) may use speech to text (STT) technology to analyze information about voice from an input audio signal, which is an example and is not limited by a specific technology.

[0076] In addition, the examiner may inform the subject in advance of the gaze order for each grid point. Thereafter, the examiner may instruct the subject to gaze at each grid point in sequence when a beep sounds. Thereafter, the subject may gaze at the first to ninth grid points according to the beep sound. The timing storage unit (40) may receive the beep sound signal and store the timing information. This is an example and is not limited by the examination method.

[0077] As another example, the timing storage unit (40) may receive information on target images presented at the first to ninth grid points from the RGB camera (11). When the target receives an image presented at the first grid point, the timing storage unit (40) may start recording timing information for the first grid point. Thereafter, when the target receives an image presented at the second grid point, the timing storage unit (40) may end recording timing information for the first grid point and start recording timing information for the second grid point. Since the method of storing timing information for the third to ninth grid points is the same, a repeated description will be omitted.

[0078] When a target is presented again at the first grid point, the timing storage unit (40) can begin recording timing information for the opposite eye. Since the method for storing timing information for the first to ninth grid points is the same, a repetitive description will be omitted.

[0079] The timing storage unit (40) may use image segmentation, template matching, and / or feature extraction techniques to analyze that a target is presented at the grid point, which are examples and are not limited by the above method.

[0080] In addition, the timing storage unit (40) can receive images in which targets are presented at the first to ninth grid points from the RGB camera (11) and store timing information for each grid point. Thereafter, the timing storage unit (40) can receive a voice signal from the examiner, such as "Cover the other eye," from the microphone (14) and store timing information for the other eye.

[0081] FIG. 4 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0082] Referring to FIG. 4, an eye movement ability information generating device (1-2) according to another embodiment of the present specification may include a housing (10-2) including an RGB camera (11), a depth sensor (12), an eye tracking sensor (13), a microphone (14), and a field of view blocking device (15), a coordinate calculation unit (20), a gaze information storage unit (30), a timing storage unit (40), and a movement ability calculation unit (50). Since the RGB camera (11), the depth sensor (12), the eye tracking sensor (13), the microphone (14), the coordinate calculation unit (20), the gaze information storage unit (30), the timing storage unit (40), and the movement ability calculation unit (50) have been described above, a repeated description thereof will be omitted.

[0083] The housing (10-2) may further include a field of view blocking device (15) that individually blocks the field of view of both eyes of the subject. The gaze information storage unit (30) may output a signal to the field of view blocking device to block the field of view of one of the subject's eyes. When the field of view blocking device (15) receives the signal, it may block the field of view of the corresponding eye.

[0084] The above-described gaze information storage unit (30) can receive image information of a target presented at the first to ninth grid points from the RGB camera (11). The above-described gaze information storage unit (30) can receive an image in which the target is presented at the ninth grid point from the RGB camera (11). Thereafter, the above-described gaze information storage unit (30) can receive an image in which the target disappears at the ninth grid point from the RGB camera (11). At this time, the above-described gaze information storage unit (30) can output a signal to release the blocking of the field of vision of an eye whose field of vision is blocked by the field of vision blocking device (15) and to block the field of vision of the opposite eye.

[0085] FIG. 5 is an example image of items to be quantified to evaluate eye movement ability according to one embodiment of the present specification.

[0086] Referring to FIG. 5, the motor ability calculation unit (50) can receive distance information (a of FIG. 5) between the target (2) and the eye from the depth sensor (12). The motor ability calculation unit (50) can receive the timing information and the gaze direction measurement value. The motor ability calculation unit (50) can calculate an average value of the gaze direction using the gaze direction measurement value measured while the subject gazes at the target. The motor ability calculation unit (50) can calculate the coordinates of the estimated position of the subject's gaze on the target (2) using the average value. The motor ability calculation unit (50) can calculate the distance error (b of FIG. 5) between the estimated position and the target.

[0087] In addition, the above-mentioned exercise ability calculation unit (50) can calculate the direction error (θ in FIG. 5) between the estimated position and each grid point using the above-mentioned distance information.

[0088] In addition, the above-mentioned exercise ability calculation unit (50) can calculate the total error including both the distance error and the direction error.

[0089] FIG. 6 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0090] Referring to FIG. 6, an eye movement ability information generating device (1-3) according to another embodiment of the present specification may include a housing (10-2) including an RGB camera (11), a depth sensor (12), an eye tracking sensor (13), a microphone (14), and a field of view blocking device (15), a coordinate calculation unit (20), a gaze information storage unit (30), a timing storage unit (40), a motor ability calculation unit (50), and a graph output unit (60). Since the RGB camera (11), the depth sensor (12), the eye tracking sensor (13), the microphone (14), the field of view blocking device (15), the coordinate calculation unit (20), the gaze information storage unit (30), the timing storage unit (40), and the motor ability calculation unit (50) have been described above, a repeated description thereof will be omitted.

[0091] The above graph output unit (60) can output a graph of the position of the subject's gaze according to each grid point.

[0092] Figure 7 is an example image of a graph according to the gaze positions of the subject's left and right eyes.

[0093] Referring to Fig. 7, the graph output unit (60) can output a graph of the position of the gaze according to each grid point for the left and right eyes of the subject. The graph for the left eye may indicate that the inferior oblique muscle of the left eye is hyperactive. The graph for the right eye may indicate that the inferior rectus muscle of the right eye is hyperactive.

[0094] The housing (10, 10-1, 10-2) may further include an inertial measurement unit sensor and / or a speaker. When a subject wearing the housing (10, 10-1, 10-2) moves his / her head outside a predetermined angle range, the speaker may output a guidance voice to the subject so that the angle of the head falls within the range. Alternatively, when the angle of the subject's head falls outside the range, a warning sound may be generated, and the examiner may verbally command the subject to bring the angle of the head within the range. In addition, a guidance phrase may be output to the subject through the housing (10, 10-1, 10-2) so that the angle of the head falls within the range.

[0095] FIG. 8 is a schematic image of an eye movement ability information generation system according to one embodiment of the present specification.

[0096] Referring to Fig. 8, the eye movement ability information generation device (1, 1-1, 1-2, 1-3) may be a part of an eye movement ability information generation system (70) including the target (2), a computing device (3), a target generation device (4), and a communication device (5). The computing device (3) may be a computer, a smartphone, and / or a tablet computer, and is not limited by specific equipment.

[0097] Figure 9 is an example image showing the linkage between an eye movement ability information generation device and a computing device.

[0098] Referring to Fig. 9, the eye movement ability information generation device (1, 1-1, 1-2, 1-3) can be controlled using software installed in the computing device (3). The computing device (3) can output an operation signal to the eye movement ability information generation device (1, 1-1, 1-2, 1-3) through the communication device (5). In addition, the computing device (3) can receive data generated from the eye movement ability information generation device (1, 1-1, 1-2, 1-3) through the communication device (5).

[0099] The computing device (3) can output image data of the subject's left eye ((a) of FIG. 9) and right eye ((a') of FIG. 9) captured by the eye tracking sensor (13). In addition, the computing device (3) can output the angle of the subject's head ((b) of FIG. 9) measured by the inertial measurement sensor.

[0100] The computing device (3) can set the inspection cycle of the eye movement ability information generation device (1, 1-1, 1-1-2, 1-3) (Fig. 9 (c)). The inspection cycle may be the time at which a target is presented at each grid point. The computing device (3) can output data setting the cycle to the target generation device (4) via the communication device (5). The target generation device (4) can generate a target according to the cycle at the location of each grid point.

[0101] The computing device (3) can display the order in which targets are generated at each grid point ((d) of FIG. 9). In addition, the computing device (3) can set the order in which targets are generated at each grid point. In addition, the computing device (3) can output data setting the order to the target generation device (4). The target generation device (4) can generate targets at each grid point in the order set in the data.

[0102] The computing device (3) can output a signal to the gaze information storage unit (30) to block or unblock the visual field of the subject's left eye (Fig. 9 (e)). The gaze information storage unit (30) can output the signal to the visual field blocking device (15).

[0103] The computing device (3) can output a signal to the gaze information storage unit (30) to block or unblock the visual field of the subject's right eye ((e') of FIG. 9). The gaze information storage unit (30) can output the signal to the visual field blocking device (15).

[0104] When the above-mentioned visual field blocking device (15) blocks the visual field of the left or right eye of the subject, the computing device (3) can output a signal to the ocular movement ability information generating device (1, 1-1, 1-2, 1-3) to perform an ocular movement ability test of the left or right eye (Fig. 9 (f)).

[0105] During the above ocular movement ability information generation process, the computing device (3) can receive image data of the target (2) from the coordinate calculation unit (20) and measurement values ​​for the gaze direction from the gaze information storage unit (30). The computing device (3) can display the image of the target (2) and information for the gaze direction as images. In (g) of Fig. 9, the image of the target (2) and the estimated gaze position for the measurement value measured during the examination of the left eye of the subject can be displayed. In (g') of Fig. 9, the image of the target (2) and the estimated gaze position for the measurement value measured during the examination of the right eye of the subject can be displayed. When the ocular movement ability test for the left or right eye of the subject is completed, the computing device (3) can output a test completion signal to the ocular movement ability information generation device (1, 1-1, 1-2, 1-3). Thereafter, the computing device (3) can digitize the examination results of the left or right eye of the subject (Fig. 9 (h)).

[0106] The above computing device (3) can display the test results of the left eye of the subject ((i) of FIG. 9)) or the test results of the right eye (i' of FIG. 9)).

[0107] The computing device (3) can receive a graph of the gaze positions of the left and right eyes output from the graph output unit (60). The computing device (3) can output a result sheet ((j) of FIG. 9) including the graph, the deviation value of the left eye for each grid point, and / or the deviation value of the right eye for each grid point. The computing device (3) can output the result sheet in the form of a document file and / or an image file, which is an example and is not limited by a specific file format.

[0108] The computing device (3) can output a signal for initializing the generated eye movement ability information, the angle information of the subject's head, and the setting information of the RGB camera (11), the depth sensor (12), and / or the eye tracking sensor (13). In addition, the computing device (3) can output information on the communication status and / or the examination status with the eye movement ability information generating device (1, 1-1, 1-2, 1-3).

[0109] The method of linking the above computing device (3) and the eye movement ability information generation device (1, 1-1, 1-2, 1-3) is an example and is not limited by the above method.

[0110] This specification assumes that the subject's ocular motor ability is tested using the Hess Screen Chart test. However, this is not limited to the Hess Screen Chart test, and it is clear that various embodiments can be created depending on the type of ocular motor ability test, such as the Lancaster red-green test.

[0111] Below, a device for generating angle information for generating angle information will be described.

[0112] Fig. 10 is a block diagram of a perspective information generating device according to one embodiment of the present specification.

[0113] Referring to FIG. 10, a gaze angle information generating device (100) according to one embodiment of the present specification may include a housing (110), a coordinate calculation unit (120), a gaze information storage unit (130), a timing storage unit (140), and a gaze angle calculation unit (150).

[0114] The housing (110) may be mounted on the head of the subject. The housing (10) may be a head-mounted display (HMD) of an augmented reality device. The housing (110) may be any one of smart glasses, smart goggles, or a helmet-mounted display, which are merely examples and are not limited by a specific shape.

[0115] The housing (110) may include an RGB camera (111), a depth sensor (112), and an eye tracking sensor (113). The RGB camera (111) may capture a strabismus test target and store it as image data. The depth sensor (112) may output distance information from a subject wearing the housing (110) to the target. The eye tracking sensor (113) may track the direction of the subject's eyes looking at the target when the subject looks at the target. The eye tracking sensor (113) may be an infrared camera, which is merely an example and is not limited by specific equipment.

[0116] Figure 11 is an example image of performing a strabismus measurement test.

[0117] Referring to Fig. 11, a subject wearing the housing (110) is looking at the target. The target may correspond to a distance target (200) and / or a near target (210). The distance target (200) may be positioned at a distance of about 4 to 6 m from the front of the subject. The near target (210) may be positioned at a distance of about 33 cm from the front of the subject. The distance target (200) may be presented to the subject first. Thereafter, when the strabismus assessment using the distance target (200) is completed, the near target (210) may be presented to the subject.

[0118] The above-described distance marker (200) and near-distance marker (210) may have a predetermined shape. In addition, the above-described distance marker (200) and near-distance marker (210) may include a marker of a predetermined shape. This is merely an example and is not limited to a specific shape.

[0119] The RGB camera (111) can capture the long-distance target (200) and / or the short-distance target (210) and store them as image data. The coordinate calculation unit (120) can receive image data of the long-distance target (200) and / or the short-distance target (210) from the RGB camera (111) and calculate coordinates for each target.

[0120] Edge detection, template matching, feature extraction, Hough transform, and / or region-based segmentation algorithms may be used to calculate the above coordinates, which are examples and are not limited to a specific method.

[0121] The subject can gaze at the distance target (200) and / or the near target (210) while blocking the field of vision of one of the two eyes.

[0122] First, the subject can i) gaze at the distance target (200) for a predetermined period of time while blocking the visual field of the right eye. Next, the subject can ii) gaze at the distance target (200) for a predetermined period of time while blocking the visual field of the left eye. The subject can perform steps i) and ii) a predetermined number of times. The predetermined period of time may be 5 seconds or 3 seconds. The predetermined number of times may be 2. This is merely an example and is not limited by the time and number of times. In addition, step i) may be performed after step ii) is performed and is not limited by a specific order.

[0123] Thereafter, the subject may turn his / her head to the left by a first angle predetermined based on the distance target (200) located in front and then perform steps i) and ii) the above number of times. Next, the subject may turn his / her head to the right by the first angle based on the distance target (200) and then perform steps i) and ii) the above number of times. The first angle may correspond to about 20 to 25°, which is merely an example and is not limited by the angle. In addition, the subject may first perform steps i) and ii) after turning his / her head to the right and is not limited by a specific order.

[0124] Thereafter, the subject may tilt his / her head to the left by a second angle predetermined while looking at the distance target (200) located in front of him / her, and then perform steps i) and ii) the above number of times. Next, the subject may tilt his / her head to the right by the second angle while looking at the distance target (200) located in front of him / her, and then perform steps i) and ii) the above number of times. The second angle may correspond to about 30°, which is merely an example and is not limited by the angle. In addition, the subject may tilt his / her head to the right and then perform steps i) and ii) the above number of times, and is not limited by a specific order.

[0125] Thereafter, the subject can perform the above processes i) and ii) the above number of times using the near-distance indicator (210).

[0126] The subject may be verbally instructed by the examiner to perform each step. Furthermore, guidance text may be displayed through the housing (110) to direct the subject to perform each step. Alternatively, the housing (110) may further include a speaker (not shown) and output guidance voices to the subject to perform each step. This is merely an example and is not limited to a specific method.

[0127] The above eye tracking sensor (113) can track the direction of the subject's gaze during each of the above processes and calculate a measurement value for the gaze direction. The above gaze information storage unit (130) can store multiple measurement values ​​for the gaze direction of one eye with an open field of vision during each of the above processes.

[0128] When the subject gazes at the distance target (200) and / or the near target (210), the timing storage unit (140) can store the timing (hereinafter, 'timing information') at which the subject gazes at the distance target (200) and / or the near target (210) with an eye whose field of vision is not blocked.

[0129] The above-described strabismus calculation unit (150) can receive the timing information and the gaze information measurement values. In addition, the strabismus calculation unit (150) can receive distance information between the subject and the distance target (200) and / or the near target (210) from the depth sensor (12). The strabismus calculation unit (150) can calculate the strabismus of the subject using the gaze information measurement values ​​and the distance information for each process corresponding to a specific timing using the timing information. The strabismus calculation unit (150) can output the strabismus for each of the plurality of measurement values.

[0130] Fig. 12 is a block diagram of a perspective information generating device according to another embodiment of the present specification.

[0131] Referring to FIG. 12, a device (100-1) for generating angle information according to another embodiment of the present specification may include a housing (100-1) including an RGB camera (111), a depth sensor (112), an eye tracking sensor (113), and a field of view blocking device (114), a coordinate calculation unit (120), a gaze information storage unit (130), a timing storage unit (140), and a gaze angle storage unit (150). Since the RGB camera (111), the depth sensor (112), the eye tracking sensor (113), the coordinate calculation unit (120), the gaze information storage unit (130), the timing storage unit (140), and the gaze angle calculation unit (150) have been described above, a repeated description thereof will be omitted.

[0132] The above-described field of view blocking device (114) can individually block the field of view of both eyes of the subject. The above-described gaze information storage unit (130) can output a blocking signal to the field of view blocking device (114) so ​​that the field of view blocking device (114) can block the field of view of one eye of the subject for a predetermined period of time.

[0133] For example, the gaze information storage unit (130) can output a signal to the field of view blocking device (114) to block the field of view of the right eye of the subject. The field of view blocking device (114) can block the field of view of the right eye of the subject. After a predetermined time has passed since blocking the field of view of the right eye, the gaze information storage unit (130) can output a signal to the field of view blocking device (114) to block the field of view of the left eye of the subject. The field of view blocking device (114) can release the blocking of the field of view of the right eye and block the field of view of the left eye.

[0134] Fig. 13 is a block diagram of a perspective information generating device according to another embodiment of the present specification.

[0135] Referring to FIG. 13, a device for generating angle information (100-2) according to another embodiment of the present specification may include a housing (110-2) including an RGB camera (111), a depth sensor (112), an eye tracking sensor (113), a field of view blocking device (114), and an inertial measurement device (115), a coordinate calculation unit (120), a gaze information storage unit (130), a timing storage unit (140), and a gaze angle calculation unit (150). Since the RGB camera (111), the depth sensor (112), the eye tracking sensor (113), the field of view blocking device (114), the coordinate calculation unit (120), the gaze information storage unit (130), the timing storage unit (140), and the gaze angle calculation unit (150) have been described above, a repeated description thereof will be omitted.

[0136] The inertial measurement device (115) can measure the angle of the head of the subject. The angle of the head may correspond to the first angle at which the subject turns his / her head to the left and / or right with respect to the distance target (200). In addition, the angle of the head may correspond to the second angle at which the subject tilts his / her head to the left and / or right while looking at the distance target (200) positioned in front. The timing storage unit (140) may receive at least one of the angle information of the head of the subject from the inertial measurement device (115), the image information of the distance target (200) and / or the near target (210) from the RGB camera (111), and the field of view blocking information from the field of view blocking device (114), and store the timing information.

[0137] For example, when the subject gazes at the distance target (200) without tilting his / her head, the RGB camera (111) can output image information of the distance target (200) to the timing storage unit (140). At this time, the field of view blocking device (114) can receive a signal from the gaze information storage unit (130) to block the field of view of the subject's right eye and block the field of view of the right eye. The field of view blocking device (114) can transmit information indicating that the field of view of the right eye has been blocked to the timing storage unit (140). The timing storage unit (140) can store timing information indicating that the subject gazes at the distance target (200) with the left eye without tilting his / her head.

[0138] As another example, after the subject turns his / her head to the left by the first angle based on the distance target (200), he / she can gaze at the distance target (200). The RGB camera (111) can output image information of the distance target (200) to the timing storage unit (140). At this time, the field of view blocking device (114) can receive a signal to block the right eye of the subject from the gaze information storage unit (130) and block the field of view of the right eye. The field of view blocking device (114) can transmit information on blocking the field of view of the right eye to the timing storage unit (140). The inertial measurement device (115) can transmit information on turning the head to the left by the first angle based on the distance target (200) to the timing storage unit (140). The timing storage unit (140) can store timing information for the subject to turn his / her head to the left by the first angle and look at the distance target (200) with the left eye.

[0139] As another example, the subject may tilt his / her head to the left by the second angle while looking at the distance target (200) located in front. The RGB camera (111) may output image information of the distance target (200) to the timing storage unit (140). At this time, the field of view blocking device (114) may receive a signal to block the right eye of the subject from the gaze information storage unit (130) and block the field of view of the right eye. The field of view blocking device (114) may transmit information on blocking the field of view of the right eye to the timing storage unit (140). The inertial measurement device (115) may transmit information on the subject tilting his / her head to the left by the second angle to the timing storage unit (140). The timing storage unit (140) may store timing information on the subject tilting his / her head to the left by the second angle and looking at the distance target (200) with his / her left eye.

[0140] As another example, when the subject gazes at the near-distance target (210) without tilting his / her head, the RGB camera (111) can output image information of the near-distance target (210) to the timing storage unit (140). At this time, the field of view blocking device (114) can receive a signal from the gaze information storage unit (130) to block the field of view of the subject's right eye and block the field of view of the right eye. The field of view blocking device (114) can transmit information indicating that the field of view of the right eye has been blocked to the timing storage unit (140). The timing storage unit (140) can store timing information indicating that the subject gazes at the near-distance target (210) with the left eye without tilting his / her head.

[0141] The method of storing timing information after blocking the field of vision of the opposite eye, turning the head in the opposite direction, or tilting the head in the opposite direction is the same in each process, so a repeated explanation is omitted.

[0142] The above housing (110-2) can output the angle of the head measured from the inertial measurement device (115) to the subject through a display.

[0143] The above-described strabismus calculation unit (150) can calculate the strabismus angle for each eye using the distance information from the depth sensor (112) to the distance target (200) and / or the near target (210), the timing information, and the gaze tracking measurement value. The above-described strabismus calculation unit (150) can calculate the angles in the up / down and left / right directions formed by the subject's gaze direction and the distance target (200) and / or the near target (210) in each of the above-described processes.

[0144] Fig. 14 is a block diagram of a perspective information generating device according to another embodiment of the present specification.

[0145] Referring to FIG. 14, a device (100-3) for generating angle of view information according to another embodiment of the present specification may include a housing (110-2) including an RGB camera (111), a depth sensor (112), an eye tracking sensor (113), a field of view blocking device (114), and an inertial measurement device (115), a coordinate calculation unit (120), a gaze information storage unit (130), a timing storage unit (140), a gaze angle calculation unit (150), and a diopter conversion unit (160). Since the RGB camera (111), the depth sensor (112), the eye tracking sensor (113), the field of view blocking device (114), the inertial measurement sensor (115), the coordinate calculation unit (120), the gaze information storage unit (130), the timing storage unit (140), and the gaze angle calculation unit (150) have been described above, a repeated description thereof will be omitted.

[0146] The above diopter conversion unit (160) can convert the angle of astigmatism calculated by the angle of astigmatism calculation unit (150) into a unit of prism diopter. The formula for converting the angle of astigmatism into diopter is as shown in [Mathematical Formula 1].

[0147]

[0148] The above-described strabismus calculation unit (150) can calculate a plurality of strabismus angles using the line of sight measurement values ​​for each eye measured in each of the above-described processes. The above-described strabismus calculation unit (150) can output the plurality of strabismus angles to the diopter conversion unit (160). The diopter conversion unit (160) can determine which of the plurality of preset strabismus angle regions the plurality of strabismus angles fall into. The diopter conversion unit (160) can select the region that appears most frequently among the plurality of strabismus angle regions as a mode. The diopter conversion unit (160) can convert the mode into a diopter.

[0149] FIG. 15 is a schematic image of a perspective information generation system according to one embodiment of the present specification.

[0150] Referring to Fig. 15, the above-described angle information generating device (100, 100-1, 100-2, 100-3) may be a part of an angle information generating system (500) including the above-described distance indicator (200), the above-described near-distance indicator (210), the above-described computing device (300), and the above-described communication device (400). The above-described computing device may be a computer, a smartphone, and / or a tablet computer, and is not limited by specific equipment.

[0151] Figure 16 is an example image showing the linkage between a visual information generation device and a computing device.

[0152] Referring to FIG. 16, the angle information generation device (100, 100-1, 100-2, 100-3) can be controlled using software installed in the computing device (300). The computing device (300) can output an operation signal to the angle information generation device (100, 100-1, 100-2, 100-3) through the communication device (400). In addition, the computing device (300) can receive data generated from the angle information generation device (100, 100-1, 100-2, 100-3) through the communication device (400).

[0153] The computing device (300) can output image data of the subject's left eye ((a) of FIG. 16) and right eye ((a') of FIG. 16) captured by the eye tracking sensor (113). In addition, the computing device (300) can output the angle of the subject's head ((b) of FIG. 16) measured by the inertial measurement sensor (115).

[0154] The computing device (300) can set the inspection cycle of the visual angle information generating device (100, 100-1, 100-2, 100-3) (Fig. 16 (c)). The inspection cycle may be the time during which the left and right eyes of the subject are shielded.

[0155] The computing device (300) can output a signal to the gaze information storage unit (130) to block or unblock the visual field of the subject's left eye (Fig. 16 (d)). The gaze information storage unit (130) can output the signal to the visual field blocking device (114).

[0156] The computing device (300) can output a signal to the gaze information storage unit (130) to block or unblock the visual field of the subject's right eye ((d') of FIG. 16). The gaze information storage unit (130) can output the signal to the visual field blocking device (115).

[0157] The computing device (300) can output a signal to block the field of view of the left or right eye according to the inspection cycle.

[0158] When the above-mentioned visual field blocking device (114) blocks the visual field of the left or right eye of the subject, the computing device (300) can output a signal to the strabismus information generating device (100, 100-1, 100-2, 100-3) to perform a strabismus test of the left or right eye ((e) of FIG. 16).

[0159] During the above-described process of generating the angle of vision information, the computing device (300) may receive image data of the distance target (200) and / or the near target (210) from the coordinate calculation unit (120) and measurement values ​​for the gaze direction from the gaze information storage unit (130). The computing device (300) may display the image of the distance target (200) and / or the near target (210) and information for the gaze direction as images. In (f) of FIG. 16, the image of the distance target (200) and / or the near target (210) and the estimated gaze position for the measurement value measured when examining the left eye of the subject may be displayed. In (f') of FIG. 16, the image of the distance target (200) and / or the near target (210) and the estimated gaze position for the measurement value measured when examining the right eye of the subject may be displayed.

[0160] When the strabismus test of the subject is completed, the computing device (300) can output a test completion signal to the strabismus information generating device (100, 100-1, 100-2, 100-3). Thereafter, the computing device (300) can digitize the test results of the left and / or right eyes of the subject (Fig. 16 (g)).

[0161] The computing device (300) may receive the strabismus data of the left and right eyes output from the strabismus calculation unit (150) and / or the diopter conversion unit (160). The computing device (300) may output a strabismus measurement result sheet ((h) of FIG. 16) according to each of the above processes. The computing device (300) may output the result sheet in the form of a document file and / or an image file, which is an example and is not limited by a specific file format.

[0162] The computing device (300) can output a signal for initializing the generated squint information, the angle information of the subject's head, and the setting information of the RGB camera (111), the depth sensor (112), and / or the eye tracking sensor (113). In addition, the computing device (300) can output information on the communication status and / or the inspection status with the squint information generating device (100, 100-1, 100-2, 100-3).

[0163] The method of linking the computing device (300) and the visual information generating device (100, 100-1, 100-2, 100-3) is an example and is not limited by the method.

[0164] In this specification, we have described an augmented reality-based strabismus measurement test that replaces the prism occlusion test. However, it is not limited to replacing the prism occlusion test, and it is obvious that the method can be applied to various types of strabismus tests, such as the Maddox cross test, the Maddox rod test, and / or the double Maddox rod test.

[0165] Fig. 17 is a block diagram of a device for generating ocular movement ability information and strabismus information according to one embodiment of the present specification.

[0166] Referring to FIG. 17, the ocular movement ability information generation device (1, 1-1, 1-2, 1-3) and the strabismus information generation device (100, 100-1, 100-2, 100-3) may be a component of the ocular movement ability information and strabismus information generation device (1000). The above-described ocular movement ability information and strabismus information generating device (100) may include a housing (1100) of a predetermined shape including an RGB camera (1110), a depth sensor (1120), an eye tracking sensor (1130), a field of view blocking device (1140), an inertial measurement sensor (1150), and a microphone (1160), a coordinate calculation unit (1200), a gaze information storage unit (1300), a timing storage unit (1400), a strabismus calculation unit (1500), a diopter conversion unit (1600), a motor ability calculation unit (1700), and a graph output unit (1800). Since each component has been described above, a repetitive description thereof will be omitted.

[0167] FIG. 18 is a schematic structural diagram of software linked to an ocular movement ability information and strabismus information generation device according to one embodiment of the present specification.

[0168] Referring to FIG. 18, the ocular movement ability information generation and strabismus information generation software according to one embodiment of the present specification may be implemented as a single software. The software may be included in a computing device that is linked to the ocular movement ability information and strabismus information generation device (1000). The computing device including the software may output a calibration signal to the ocular movement ability information and strabismus information generation device (1000) to adjust each component of the housing (1100) according to the physical characteristics of the subject.

[0169] The computing device may receive preset data for generating ocular movement ability information and / or strabismus information. Thereafter, the computing device may output a signal to the ocular movement ability information and strabismus information generating device (1000) to perform a test for generating ocular movement ability information and / or strabismus information based on the preset data. Upon completion of the test, the computing device may output the respective results.

[0170] The adjustment of the above ocular movement ability information and strabismus information generation device (1000) and each test progress screen can be output in real time to the examiner and the subject through the display of the computing device and the housing (1100).

[0171] The above coordinate calculation unit (20, 120, 1200), gaze information storage unit (30, 130, 1300), timing storage unit (40, 140, 1400), strabismus calculation unit (150, 1500), diopter conversion unit (160, 1600), motor ability calculation unit (50, 1700) and graph output unit (60, 1800) may include a processor, ASIC (application-specific integrated circuit), other chipset, logic circuit, register, communication modem, data processing device, etc. known in the technical field to which the present invention belongs in order to execute calculation and various control logic. In addition, when the above-described control logic is implemented in software, the coordinate calculation unit (20, 120, 1200), gaze information storage unit (30, 130, 1300), timing storage unit (40, 140, 1400), astigmatism calculation unit (150, 1500), diopter conversion unit (160, 1600), motor ability calculation unit (50, 1700) and graph output unit (60, 1800) can be implemented as a set of program modules. At this time, the program modules can be stored in the memory device and executed by the processor.

[0172] Hereinafter, a method for generating eye movement ability information using an eye movement ability information generating device (1, 1-1, 1-2, 1-3) according to the present specification will be described. However, in describing the method for generating eye movement ability information according to the present specification, repetitive descriptions of each component are omitted.

[0173] The method for generating eye movement ability information according to the present specification is a method for generating eye movement ability information using a device including an RGB camera (11), a depth sensor (12), and an eye tracking sensor (13) in a housing (10) of a predetermined shape mounted on the head of a subject.

[0174] Fig. 19 is a flowchart of a method for generating eye movement ability information according to one embodiment of the present specification.

[0175] Referring to FIG. 19, when the subject wearing the housing (10) gazes at the target (2) in step S10, the processor may receive image data of the target from the RGB camera (11). The processor may calculate coordinates for a plurality of grid points from the image data. When the subject gazes at the target with one of the two eyes blocked in step S11, the processor may receive and store a measurement value for the gaze direction of the eye whose field of vision is blocked from the eye tracking sensor (13). When the subject gazes at a target presented at each grid point in step S12, the processor may store timing information for gaze at each grid point. In step S13, the processor may receive distance information between the subject and the target (2) from the depth sensor (12). The processor may calculate the gaze error of the subject for each grid point using the distance information, timing information, and gaze tracking measurement values.

[0176] In the above step S13, the processor can calculate an average value using the gaze tracking measurement values ​​stored at the time when the subject gazes at the grid points. The processor can calculate an estimated position on the target (2) using the average value. The processor can calculate a distance error between the estimated position and each grid point. In addition, the processor can calculate a direction error according to each grid point using the distance information and the estimated position. In addition, the processor can calculate a total error including both the distance error and the direction error.

[0177] FIG. 20 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0178] Referring to Fig. 20, steps S20, S21 and S23 are identical to steps S10, S11 and S13, so a repetitive description is omitted.

[0179] In step S22, the processor can receive image data in which a target is presented at each grid point from the RGB camera (11). The processor can store timing information according to the target presented at each grid point location.

[0180] FIG. 21 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0181] Referring to Fig. 21, steps S30, S31 and S33 are identical to steps S20, S21 and S23, so a repetitive description is omitted.

[0182] A housing (10-1) according to another embodiment of the present disclosure may further include a microphone (14) for receiving an audio signal. The processor may receive an audio signal from the microphone (14). The audio signal may correspond to an examiner's voice signal and / or a beep sound. Upon receiving the audio signal, the processor may store the timing at which the examinee gazes at each grid point.

[0183] Fig. 22 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0184] Referring to Fig. 22, steps S40 and S42 to S44 are identical to steps S30 to S33, so a repeated description is omitted.

[0185] According to another embodiment of the present specification, the housing (10-2) may further include a field of view blocking device (15) that individually blocks the field of view of the subject. In step S41, the processor may output an operating signal to the field of view blocking device to block the field of view of one eye of the subject.

[0186] Fig. 23 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0187] Referring to Fig. 23, steps S50 to S54 are identical to steps S40 to S44, so a repeated description is omitted.

[0188] In step S55, the processor can output information on the gaze of the subject's left and right eyes as a graph.

[0189] Hereinafter, a method for generating angle information using a perspective information generating device (100, 100-1, 100-2, 100-3) according to the present specification will be described. However, in describing the method for generating angle information according to the present specification, repetitive descriptions of each component are omitted.

[0190] A method for generating angle information according to this specification is a method for generating angle information using a device including an RGB camera (111), a depth sensor (112), and an eye tracking sensor (113) in a housing (110) of a predetermined shape mounted on the head of a subject.

[0191] Figure 24 is a flowchart of a method for generating angle information according to one embodiment of the present specification.

[0192] Referring to Fig. 24, when the subject wearing the housing (10) gazes at the distance target (200) and / or the near target (210) in step S60, the processor can receive image data of the distance target (200) and / or the near target (210) from the RGB camera (111). The processor can calculate coordinates for the distance target (200) and / or the near target (210) from the image data.

[0193] In step S61, when the subject gazes at the distance target (200) and / or the near target (210) with one of the two eyes blocked, the processor can receive and store multiple measurement values ​​for the gaze direction of the eye whose field of vision is not blocked from the eye tracking sensor (113).

[0194] The subject can i) gaze at the distance target (200) for a predetermined period of time while blocking the visual field of the right eye. Next, the subject can ii) gaze at the distance target (200) for a predetermined period of time while blocking the visual field of the left eye. The subject can repeat the steps i) and ii) a predetermined number of times.

[0195] Thereafter, the subject can turn his / her head to the left by a predetermined first angle based on the distance target (200) located in front and then perform steps i) and ii) the above number of times. Next, the subject can turn his / her head to the right by the first angle based on the distance target (200) and then perform steps i) and ii) the above number of times.

[0196] Thereafter, the subject can perform steps i) and ii) the number of times described above after tilting the head to the left by a predetermined second angle while looking at the distance target (200) located in front of the subject. Next, the subject can perform steps i) and ii) the number of times described above after tilting the head to the right by the second angle while looking at the distance target (200) located in front of the subject.

[0197] Thereafter, the subject can perform the above processes i) and ii) the above number of times using the near-distance indicator (210).

[0198] The above processor can store multiple measurement values ​​for the gaze direction of the subject in each of the above processes.

[0199] In step S62, the processor may store timing information at which the eye with the open field of vision gazes at the distance target (200) and / or the near target (210) in each of the above processes. In step S63, the processor may receive distance information between the subject and the distance target (200) and / or the near target (210) from the depth sensor (112). The processor may calculate a strabismus angle for each eye of the subject using the distance information, timing information, and gaze tracking measurement values. The processor may output a plurality of strabismus angles calculated using the plurality of measurement values.

[0200] In the above step S63, the processor can calculate the angles in the up / down and left / right directions formed by the subject's gaze direction and the long-distance target (200) and / or the short-distance target (210) in each of the above processes.

[0201] Fig. 25 is a flowchart of a method for generating angle information according to another embodiment of the present specification.

[0202] Referring to Fig. 25, steps S70 and S72 to S74 are identical to steps S60 to S63, and therefore, a repetitive description thereof is omitted.

[0203] A housing (100-1) according to another embodiment of the present disclosure may further include a field of view blocking device (114) that individually blocks the field of view of the subject. In step S71, the processor may output an operating signal to the field of view blocking device (114) to block the field of view of one eye of the subject for a predetermined period of time.

[0204] FIG. 26 is a flowchart of a method for generating angle information according to another embodiment of the present specification.

[0205] Referring to Fig. 26, steps S80 to S82 and S84 are identical to steps S70 to S72 and S74, and therefore, a repetitive description thereof is omitted.

[0206] A housing (100-2) according to another embodiment of the present specification may further include an inertial measurement device (115) for measuring an angle of the head of the subject. The processor may receive at least one of angle information of the head of the subject from the inertial measurement device (115), image information of the long-distance target (200) and / or the near-distance target (210) from the RGB camera (111), and field of view blocking information from the field of view blocking device (114), and store timing information for each process of storing the plurality of measurement values.

[0207] Figure 27 is a flowchart of a method for generating angle information according to another embodiment of the present specification.

[0208] Referring to Fig. 27, steps S90 to S94 are identical to steps S80 to S84, so a repetitive description is omitted.

[0209] In step S95, the processor can determine which of the plurality of squint angles output in step S94 falls within a preset plurality of squint angle regions. The processor can select the region with the highest frequency among the plurality of squint angle regions as the mode. The processor can convert the mode into a prism diopter using [Mathematical Formula 1].

[0210] The method for generating ocular movement ability information and the method for generating strabismus information according to the present specification may be implemented in the form of a computer program written to perform each step and recorded on a computer-readable recording medium. The aforementioned computer program may include codes coded in a computer language, such as C / C++, C#, JAVA, Python, or machine language, that can be read by the processor (CPU) of the computer through the device interface of the computer, so that the computer reads the program and executes the methods implemented as the program. Such codes may include functional codes related to functions that define functions necessary for executing the methods, and may include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. In addition, such codes may further include memory reference-related codes regarding which location (address address) of the internal or external memory of the computer should reference additional information or media necessary for the processor of the computer to execute the functions. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to send and receive during communication.

[0211] The above storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium can be distributed across network-connected computer systems, so that computer-readable code can be stored in a distributed manner.

[0212] While the embodiments of this specification have been described with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical spirit or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

[0213] [Explanation of symbols]

[0214] 100: Perspective information generation device

[0215] 110: Housing

[0216] 111: RGB Camera

[0217] 112: Depth sensor

[0218] 113: Eye tracking sensor

[0219] 114: Vision blocking device

[0220] 115: Inertial measurement sensor

[0221] 120: Coordinate calculation unit

[0222] 130: Gaze information storage unit

[0223] 140: Timing storage unit

[0224] 150: Angle calculation section

[0225] 160: Diopter converter

[0226] 200: Long-distance vision indicator

[0227] 210: Near-distance vision indicator

[0228] 500: Perspective Information Generation System

Claims

1. A method for generating strabismus information using a device including an RGB camera, a depth sensor, and an eye tracking sensor in a housing of a predetermined shape mounted on the head of a subject. (a) a step in which a processor calculates coordinates for a target from image data of a target of a predetermined shape input from the RGB camera; (b) a step of the processor receiving and storing a plurality of measurement values ​​for the gaze direction of one eye of the subject whose field of vision is open from the eye tracking sensor; (c) a step in which the processor stores the timing (hereinafter referred to as “timing information”) at which one of the subject’s two eyes with open visual field fixates on the target; and (d) A method for generating angle of strabismus information, comprising: a step of calculating the angle of strabismus of the subject for each eye using the distance information between the subject and the target, the timing information, and the gaze tracking information input from the depth sensor.

2. In claim 1, The housing further includes a field of view blocking device that individually blocks the field of view of both eyes of the subject; A method for generating strabismus information, characterized in that the step (b) above outputs a blocking signal to a field of vision blocking device so as to block the field of vision of one eye of the subject for a predetermined period of time.

3. In claim 1, The housing further includes an inertial measurement device for measuring the angle of the subject's head; Step (c) above, A method for generating angle information, wherein the processor receives at least one of angle information of the subject's head from an inertial measurement device, information of the target image from an RGB camera, and field of view blocking information from the field of view blocking device, and stores the timing information.

4. In claim 1, Step (d) above, A method for generating strabismus information, wherein the processor calculates the angle between the subject's gaze direction and the up / down and left / right directions formed by the target.

5. In claim 1, After step (d) above, (e) A method for generating angle of view information, further comprising a step of selecting an area output with the highest frequency among a plurality of preset angle of view areas as the most frequent value and converting it into a prism diopter.

6. A computer program written to perform each step of the method for generating angle information according to any one of claims 1 to 5 on a computer and recorded on a computer-readable recording medium.

7. A housing of a predetermined shape that is mounted on the subject's head and includes an RGB camera, a depth sensor, and an eye tracking sensor; A coordinate calculation unit that calculates coordinates for a target from image data of a target of a predetermined shape input from the RGB camera; A gaze information storage unit that receives multiple measurement values ​​for the gaze direction of one eye with an open field of vision among the subject's two eyes from the eye tracking sensor; A timing storage unit that stores the timing (hereinafter referred to as “timing information”) at which one of the subject’s eyes with open field of vision looks at the target; and A device for generating angle of vision information, comprising a strabismus calculation unit that calculates the angle of vision of the subject for each eye using distance information between the subject and the target, the timing information, and the gaze tracking information input from the depth sensor.

8. In claim 7, The housing further includes a field of view blocking device that individually blocks the field of view of both eyes of the subject; The above gaze information storage unit is, A strabismus information generating device characterized in that it outputs a blocking signal to a field of vision blocking device to block the field of vision of one eye of the subject for a predetermined period of time.

9. In claim 7, The housing further includes an inertial measurement device for measuring the angle of the subject's head; The above timing storage unit, A strabismus information generating device that receives at least one of the angle information of the subject's head from an inertial measurement device, information of the target image from an RGB camera, and field of view blocking information from the field of view blocking device, and stores the timing information.

10. In claim 7, The above angle calculation section, A device for generating strabismus information that calculates the angle between the subject's gaze direction and the up / down and left / right directions formed by the above-mentioned target.

11. In claim 7, A device for generating angle of vision information, further comprising a diopter conversion unit for selecting an area with the highest frequency of output from among a plurality of preset angle of vision areas as a mode and converting it into a prism diopter.

12. A device for generating angle information according to any one of claims 7 to 11; Multiple time markers located at predetermined locations; A computing device that outputs an operating signal of the above-mentioned angle information generating device and receives angle information; and A perspective information generation system including a communication device for transmitting and receiving signals or information between the perspective information generation device and a computing device.

Images