Method and apparatus for generating ocular movement ability information

Abstract

In the present specification, a method and apparatus for generating ocular movement ability information are disclosed. The method for generating ocular movement ability information, according to the present specification, is a method for generating ocular movement ability information by using a device including an RGB camera, a depth sensor, and an ocular tracking sensor in a housing of a predetermined shape mounted on the head of a subject, and may comprise the steps of: (a) calculating, by a processor, coordinates for a plurality of lattice points in image data obtained by photographing an ocular movement indicator input from the RGB camera; (b) receiving, from the eye tracking sensor by the processor, a measurement value for the gaze direction of one of the eyes of the subject of which the field of view is blocked, and storing the measurement value; (c) storing, by the processor, timing (hereinafter, referred to as "timing information") at which one of the eyes of the subject of which the field of view is open gazes at the lattice points included in the ocular movement indicator; and (d) calculating, by the processor, a gaze error of the subject with respect to each of the lattice points by using distance information between the subject and the indicator input from the depth sensor, the timing information, and gaze tracking information.

Description

Method and device for generating eye movement ability information

[0001] The present invention relates to a method and device for generating eye movement ability information, and more particularly, to a method and device for generating eye movement ability information using augmented reality equipment.

[0002] This application claims priority to Korean Patent Application No. 10-2023-0147215, filed in the Republic of Korea on October 30, 2023, the entire disclosure of which is incorporated herein by reference.

[0003] The material described in this section merely provides background information on the embodiments described herein and does not necessarily constitute prior art.

[0004] The Hess chart test is performed to assess the subject's ocular motility. The Hess chart test involves the subject wearing red-green glasses and fixing their face on a test chart. The examiner provides a target by shining a red light pointer on the chart's test points and asks the subject to shine a green light pointer on the target. The examiner then records the point the subject has pointed at with the pointer on a test sheet. For the other eye, the examiner and subject exchange pointers. Conventional Hess chart tests cannot track the subject's eyes, and errors can arise from the process of recording the point the examiner has pointed at with the pointer on the test sheet, as well as from slight head movements during the test.

[0005] Recently, along with virtual environment technology, technologies for testing eye movement abilities using virtual reality (VR) equipment are being developed. Subjects wear VR equipment and view a virtual test chart displayed on the VR device's display. However, due to the limited distance from the display device, the subject's field of view while viewing the virtual test chart is limited.

[0006] The purpose of this specification is to provide a method and device for generating eye movement ability information.

[0007] This specification is not limited to the above-mentioned tasks, and other tasks not mentioned will be clearly understood by those skilled in the art from the description below.

[0008] In order to solve the above-described problem, a method for generating ocular movement ability information according to the present specification is provided, which is a method for generating ocular movement ability information using a device including an RGB camera, a depth sensor, and an eye tracking sensor in a housing of a predetermined shape mounted on a head of a subject, the method including: (a) a step in which a processor calculates coordinates for a plurality of grid points from image data of an ocular movement target input from the RGB camera; (b) a step in which the processor receives, from the eye tracking sensor, a measurement value for a gaze direction of one of the subject's binoculars whose field of vision is blocked and stores the measurement value; (c) a step in which the processor stores a timing (hereinafter, "timing information") in which one of the subject's binoculars whose field of vision is open gazes at a grid point included in the ocular movement target; and (d) a step in which the processor calculates a gaze error of the subject for each grid point using distance information between the subject and the target input from the depth sensor, the timing information, and the gaze tracking information.

[0009] According to one embodiment of the present specification, the housing further includes a microphone for receiving an audio signal; and the step (c) may be a step in which the processor receives an audio signal from the microphone or information on a target image presented at the grid point from an RGB camera, and stores the timing information.

[0010] According to one embodiment of the present specification, the housing further includes a field of view blocking device for individually blocking the field of view of both eyes of the subject; and the step (b) may be characterized in that the processor outputs a blocking signal to the field of view blocking device to block the field of view of one eye of the subject.

[0011] According to one embodiment of the present specification, the step (d) may be a step in which the processor quantifies at least one of an average distance error between the subject's gaze positions for each grid point, a direction error for each grid point, and an overall error.

[0012] The method for generating eye movement ability information according to the present specification may further include, after step (d), a step (e) in which the processor outputs a graph of the position of the subject's gaze according to each grid point.

[0013] The method for generating eye movement ability information according to the present specification can be implemented in the form of a computer program written to perform each step on a computer and recorded on a computer-readable recording medium.

[0014] An ocular movement ability information generating device according to the present specification may include: a housing having a predetermined shape, which is mounted on the head of a subject and includes an RGB camera, a depth sensor, and an eye tracking sensor; a coordinate calculation unit that calculates coordinates for a plurality of grid points from image data of an ocular movement target input from the RGB camera; a gaze information storage unit that receives and stores a measurement value for a gaze direction of one eye with a occluded field of vision among the subject's two eyes from the eye tracking sensor; a timing storage unit that stores a timing (hereinafter, "timing information") at which one eye with an open field of vision gazes at a grid point included in an ocular movement target; and an eye movement ability calculation unit that calculates a gaze error of the subject for each grid point by using distance information between the subject and the target input from the depth sensor, the timing information, and the gaze tracking information.

[0015] According to one embodiment of the present specification, the housing further includes a microphone for receiving an audio signal, and the timing storage unit can receive an audio signal from the microphone or information on a target image presented at the grid point from an RGB camera, and store the timing information.

[0016] According to one embodiment of the present specification, the housing further includes a field of view blocking device for individually blocking the field of view of both eyes of the subject; and the gaze information storage unit may be characterized in that it outputs a blocking signal to the field of view blocking device to block the field of view of one eye of the subject.

[0017] According to one embodiment of the present specification, the motor ability calculation unit can quantify at least one of an average distance error between the subject's gaze positions for each grid point, a direction error for each grid point, and an overall error.

[0018] The device for generating eye movement ability information according to the present specification may further include a graph output unit that outputs a graph of the position of the subject's gaze according to each grid point.

[0019] The ocular movement ability information generation device of the present specification may be a component of an ocular movement ability information generation system including: an ocular movement chart on which a plurality of grid points are drawn; a target generation device that presents a target image to the grid points; a computing device that outputs an operation signal of the ocular movement ability information generation device and receives ocular movement ability information; and a communication device that transmits and receives signals or information among the ocular movement ability information generation device, the computing device, and the target generation device.

[0020] Other specific details of the present invention are included in the detailed description and drawings.

[0021] According to one aspect of the present disclosure, an augmented reality device can be used to track the eyes of a subject more accurately than before.

[0022] According to another aspect of the present specification, it is possible to reduce errors that occur in the process of an examiner transferring a point illuminated by a laser pointer by a subject to a result sheet.

[0023] According to another aspect of the present specification, the problem of narrowing of the field of view that occurs in eye examination using conventional VR equipment can be solved.

[0024] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0025] FIG. 1 is a block diagram of an eye movement ability information generation device according to one embodiment of the present specification.

[0026] Figure 2 is an example image of conducting an eye movement ability test.

[0027] FIG. 3 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0028] FIG. 4 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0029] FIG. 5 is an example image of items to be quantified to evaluate eye movement ability according to one embodiment of the present specification.

[0030] FIG. 6 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0031] Figure 7 is an example image of a graph according to the gaze positions of the subject's left and right eyes.

[0032] FIG. 8 is a schematic image of an eye movement ability information generation system according to one embodiment of the present specification.

[0033] Figure 9 is an example image showing the linkage between an eye movement ability information generation device and a computing device.

[0034] FIG. 10 is a flowchart of a method for generating eye movement ability information according to one embodiment of the present specification.

[0035] Fig. 11 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0036] Fig. 12 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0037] Fig. 13 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0038] Fig. 14 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0039] The advantages and features of the invention disclosed in this specification, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, this specification is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of this specification is complete and to fully inform those of ordinary skill in the art (hereinafter referred to as "skilled workers") of the scope of this specification, and the scope of rights of this specification is defined only by the scope of the claims.

[0040] The terminology used herein is for the purpose of describing embodiments and is not intended to limit the scope of the present disclosure. In this specification, singular forms also include plural forms, unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.

[0041] Throughout the specification, the same reference numerals refer to the same elements, and the term "and / or" includes each and every combination of the elements mentioned. Although terms such as "first," "second," etc. are used to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another. Therefore, it should be understood that a first element mentioned below may also be a second element within the technical scope of the present invention.

[0042] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those skilled in the art to which this specification pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0043] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0044] FIG. 1 is a block diagram of an eye movement ability information generation device according to one embodiment of the present specification.

[0045] Referring to FIG. 1, an eye movement ability information generating device (1) according to one embodiment of the present specification may include a housing (10), a coordinate calculation unit (20), a gaze information storage unit (30), a timing storage unit (40), and a movement ability calculation unit (50).

[0046] The housing (10) may be mounted on the subject's head. The housing (10) may be a head-mounted display (HMD) of an augmented reality device. The housing (10) may be any one of smart glasses, smart goggles, or a helmet-mounted display, which are merely examples and are not limited by a specific shape.

[0047] The housing (10) may include an RGB camera (11), a depth sensor (12), and an eye tracking sensor (13). The RGB camera (11) may capture an ocular movement ability test target and store it as image data. The depth sensor (12) may output distance information from a subject wearing the housing (10) to the target. The eye tracking sensor (13) may track the direction of the subject's eyes when the subject looks at a target presented on the target. The eye tracking sensor (13) may be an infrared camera, which is merely an example and is not limited by specific equipment.

[0048] Figure 2 is an example image of conducting an eye movement ability test.

[0049] Referring to Fig. 2, a subject wearing the housing (10) is looking at the target (2). The target (2) may be any one of a Hess screen chart, a Harms screen chart, and / or a Lancaster screen chart, which is an example and is not limited to a specific target.

[0050] The RGB camera (11) can capture the target (2) and store it as image data. The coordinate calculation unit (20) can receive image data of the target (2) from the RGB camera (11) and calculate coordinates for a plurality of grid points in the image of the target (2).

[0051] To calculate the above coordinates, line detection, grid search, edge detection algorithms, and / or deep learning models may be used, which are only examples and are not limited to a specific method.

[0052] To test eye movement ability, the examiner may present a target at any one of the grid points for the examinee to fixate on.

[0053] According to one embodiment of the present specification, the examiner may present the target in a predetermined order at the grid points of the target (2). The target may be presented first at the center point of the target (2) (hereinafter referred to as the "first grid point"). Next, the target may be presented at the grid point to the right of the center point (hereinafter referred to as the "second grid point"). Next, the target may be presented at the grid point at the upper right of the center point (hereinafter referred to as the "third grid point"). Next, the target may be presented at the grid point at the lower right of the center point (hereinafter referred to as the "fourth grid point"). Next, the target may be presented at the grid point below the center point (hereinafter referred to as the "fifth grid point"). Next, the target may be presented at the grid point above the center point (hereinafter referred to as the "sixth grid point"). Next, the target may be presented at the grid point at the upper left of the center point (hereinafter referred to as the "seventh grid point"). Next, the target may be presented at the grid point to the left of the central point (hereinafter referred to as the "eighth grid point"). Next, the target may be presented at the grid point to the lower left of the central point (hereinafter referred to as the "ninth grid point"). This is merely an example and is not limited by the above order.

[0054] The above target can be presented by the examiner using a laser pointer on the target (2). Alternatively, the examiner can verbally command the examinee to look at a specific grid point. Alternatively, the target (2) can include a light-emitting device at the grid point, and light can be emitted at each grid point in a predetermined order. Alternatively, a target generating device can be used to generate a target at a specific grid point. The target generating device can be an optical device that projects a specific marker onto each grid point. This is an example and is not limited to a specific method.

[0055] The subject can sequentially gaze at targets presented at the grid points while blocking the field of vision of one of the two eyes. The subject can gaze at the targets for a predetermined period of time. At this time, the eye tracking sensor (13) can track the direction of the subject's gaze and calculate a measurement value for the gaze direction. The gaze information storage unit (30) can store the measurement value for each of the grid points. At this time, the gaze information storage unit (30) can store a measurement value for the gaze direction of the eye whose field of vision is blocked among the two eyes of the subject. When the subject completes the examination for one eye, the measurement value for the other eye can be stored.

[0056] When the subject looks at the target, the timing storage unit (40) can store the timing (hereinafter, 'timing information') at which the subject looks at the target with an eye whose field of vision is not blocked.

[0057] The above motor ability calculation unit (50) can receive the timing information and gaze information measurement values. In addition, the motor ability calculation unit (50) can receive distance information between the subject and the target (2) from the depth sensor (12). The motor ability calculation unit (50) can use the timing information to calculate the gaze error of the subject using the gaze information measurement values ​​and the distance information for each grid point corresponding to a specific timing.

[0058] FIG. 3 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0059] Referring to FIG. 3, an eye movement ability information generating device (1-1) according to another embodiment of the present specification may include a housing (10-1) including an RGB camera (11), a depth sensor (12), an eye tracking sensor (13), and a microphone (14), a coordinate calculation unit (20), a gaze information storage unit (30), a timing storage unit (40), and a movement ability calculation unit (50). Since the RGB camera (11), the depth sensor (12), the eye tracking sensor (13), the coordinate calculation unit (20), the gaze information storage unit (30), the timing storage unit (40), and the movement ability calculation unit (50) have been described above, a repeated description thereof will be omitted.

[0060] The above housing (10-1) can receive an audio signal through the microphone (14). The timing storage unit (40) can store the timing information using the audio signal or image information input from the RGB camera (11).

[0061] For example, the examiner may verbally command the subject to look at the first grid point. At this time, the microphone (14) may receive the command and output an audio signal to the timing storage unit (40). When the timing storage unit (40) receives the audio signal, the timing storage unit (40) may start recording timing information for the first grid point. Thereafter, the examiner may verbally command the subject to look at the second grid point. When the timing storage unit (40) receives the audio signal to look at the second grid point, the timing storage unit (40) may end recording timing information for the first grid point and start recording timing information for the second grid point. Since the method of storing timing information for the third to ninth grid points is the same, a repeated description will be omitted.

[0062] Thereafter, the examiner can command the subject, such as, "Cover the other eye." The timing storage unit (40) can begin recording timing information for the other eye upon receiving the audio signal. Since the method for storing timing information for the first to ninth grid points is the same, a repeated explanation will be omitted.

[0063] The above timing storage unit (40) may use speech to text (STT) technology to analyze information about voice from an input audio signal, which is an example and is not limited by a specific technology.

[0064] In addition, the examiner may inform the subject in advance of the gaze order for each grid point. Thereafter, the examiner may instruct the subject to gaze at each grid point in sequence when a beep sounds. Thereafter, the subject may gaze at the first to ninth grid points according to the beep sound. The timing storage unit (40) may receive the beep sound signal and store the timing information. This is an example and is not limited by the examination method.

[0065] As another example, the timing storage unit (40) may receive information on target images presented at the first to ninth grid points from the RGB camera (11). When the target receives an image presented at the first grid point, the timing storage unit (40) may start recording timing information for the first grid point. Thereafter, when the target receives an image presented at the second grid point, the timing storage unit (40) may end recording timing information for the first grid point and start recording timing information for the second grid point. Since the method of storing timing information for the third to ninth grid points is the same, a repeated description will be omitted.

[0066] When a target is presented again at the first grid point, the timing storage unit (40) can begin recording timing information for the opposite eye. Since the method for storing timing information for the first to ninth grid points is the same, a repetitive description will be omitted.

[0067] The timing storage unit (40) may use image segmentation, template matching, and / or feature extraction techniques to analyze that a target is presented at the grid point, which are examples and are not limited by the above method.

[0068] In addition, the timing storage unit (40) can receive images in which targets are presented at the first to ninth grid points from the RGB camera (11) and store timing information for each grid point. Thereafter, the timing storage unit (40) can receive a voice signal from the examiner, such as "Cover the other eye," from the microphone (14) and store timing information for the other eye.

[0069] FIG. 4 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0070] Referring to FIG. 4, an eye movement ability information generating device (1-2) according to another embodiment of the present specification may include a housing (10-2) including an RGB camera (11), a depth sensor (12), an eye tracking sensor (13), a microphone (14), and a field of view blocking device (15), a coordinate calculation unit (20), a gaze information storage unit (30), a timing storage unit (40), and a movement ability calculation unit (50). Since the RGB camera (11), the depth sensor (12), the eye tracking sensor (13), the microphone (14), the coordinate calculation unit (20), the gaze information storage unit (30), the timing storage unit (40), and the movement ability calculation unit (50) have been described above, a repeated description thereof will be omitted.

[0071] The housing (10-2) may further include a field of view blocking device (15) that individually blocks the field of view of both eyes of the subject. The gaze information storage unit (30) may output a signal to the field of view blocking device to block the field of view of one of the subject's eyes. When the field of view blocking device (15) receives the signal, it may block the field of view of the corresponding eye.

[0072] The above-described gaze information storage unit (30) can receive image information of a target presented at the first to ninth grid points from the RGB camera (11). The above-described gaze information storage unit (30) can receive an image in which the target is presented at the ninth grid point from the RGB camera (11). Thereafter, the above-described gaze information storage unit (30) can receive an image in which the target disappears at the ninth grid point from the RGB camera (11). At this time, the above-described gaze information storage unit (30) can output a signal to release the blocking of the field of vision of an eye whose field of vision is blocked by the field of vision blocking device (15) and to block the field of vision of the opposite eye.

[0073] FIG. 5 is an example image of items to be quantified to evaluate eye movement ability according to one embodiment of the present specification.

[0074] Referring to FIG. 5, the motor ability calculation unit (50) can receive distance information (a of FIG. 5) between the target (2) and the eye from the depth sensor (12). The motor ability calculation unit (50) can receive the timing information and the gaze direction measurement value. The motor ability calculation unit (50) can calculate an average value of the gaze direction using the gaze direction measurement value measured while the subject gazes at the target. The motor ability calculation unit (50) can calculate the coordinates of the estimated position of the subject's gaze on the target (2) using the average value. The motor ability calculation unit (50) can calculate the distance error (b of FIG. 5) between the estimated position and the target.

[0075] In addition, the above-mentioned exercise ability calculation unit (50) can calculate the direction error (θ in FIG. 5) between the estimated position and each grid point using the above-mentioned distance information.

[0076] In addition, the above-mentioned exercise ability calculation unit (50) can calculate the total error including both the distance error and the direction error.

[0077] FIG. 6 is a block diagram of an eye movement ability information generation device according to another embodiment of the present specification.

[0078] Referring to FIG. 6, an eye movement ability information generating device (1-3) according to another embodiment of the present specification may include a housing (10-2) including an RGB camera (11), a depth sensor (12), an eye tracking sensor (13), a microphone (14), and a field of view blocking device (15), a coordinate calculation unit (20), a gaze information storage unit (30), a timing storage unit (40), a motor ability calculation unit (50), and a graph output unit (60). Since the RGB camera (11), the depth sensor (12), the eye tracking sensor (13), the microphone (14), the field of view blocking device (15), the coordinate calculation unit (20), the gaze information storage unit (30), the timing storage unit (40), and the motor ability calculation unit (50) have been described above, a repeated description thereof will be omitted.

[0079] The above graph output unit (60) can output a graph of the position of the subject's gaze according to each grid point.

[0080] Figure 7 is an example image of a graph according to the gaze positions of the subject's left and right eyes.

[0081] Referring to Fig. 7, the graph output unit (60) can output a graph of the position of the gaze according to each grid point for the left and right eyes of the subject. The graph for the left eye may indicate that the inferior oblique muscle of the left eye is hyperactive. The graph for the right eye may indicate that the inferior rectus muscle of the right eye is hyperactive.

[0082] The housing (10, 10-1, 10-2) may further include an inertial measurement unit sensor and / or a speaker. When a subject wearing the housing (10, 10-1, 10-2) moves his / her head outside a predetermined angle range, the speaker may output a guidance voice to the subject so that the angle of the head falls within the range. Alternatively, when the angle of the subject's head falls outside the range, a warning sound may be generated, and the examiner may verbally command the subject to bring the angle of the head within the range. In addition, a guidance phrase may be output to the subject through the housing (10, 10-1, 10-2) so that the angle of the head falls within the range.

[0083] FIG. 8 is a schematic image of an eye movement ability information generation system according to one embodiment of the present specification.

[0084] Referring to Fig. 8, the eye movement ability information generation device (1, 1-1, 1-2, 1-3) may be a part of an eye movement ability information generation system (70) including the target (2), a computing device (3), a target generation device (4), and a communication device (5). The computing device (3) may be a computer, a smartphone, and / or a tablet computer, and is not limited by specific equipment.

[0085] Figure 9 is an example image showing the linkage between an eye movement ability information generation device and a computing device.

[0086] Referring to Fig. 9, the eye movement ability information generation device (1, 1-1, 1-2, 1-3) can be controlled using software installed in the computing device (3). The computing device (3) can output an operation signal to the eye movement ability information generation device (1, 1-1, 1-2, 1-3) through the communication device (5). In addition, the computing device (3) can receive data generated from the eye movement ability information generation device (1, 1-1, 1-2, 1-3) through the communication device (5).

[0087] The computing device (3) can output image data of the subject's left eye ((a) of FIG. 9) and right eye ((a') of FIG. 9) captured by the eye tracking sensor (13). In addition, the computing device (3) can output the angle of the subject's head ((b) of FIG. 9) measured by the inertial measurement sensor.

[0088] The computing device (3) can set the inspection cycle of the eye movement ability information generation device (1, 1-1, 1-1-2, 1-3) (Fig. 9 (c)). The inspection cycle may be the time at which a target is presented at each grid point. The computing device (3) can output data setting the cycle to the target generation device (4) via the communication device (5). The target generation device (4) can generate a target according to the cycle at the location of each grid point.

[0089] The computing device (3) can display the order in which targets are generated at each grid point ((d) of FIG. 9). In addition, the computing device (3) can set the order in which targets are generated at each grid point. In addition, the computing device (3) can output data setting the order to the target generation device (4). The target generation device (4) can generate targets at each grid point in the order set in the data.

[0090] The computing device (3) can output a signal to the gaze information storage unit (30) to block or unblock the visual field of the subject's left eye (Fig. 9 (e)). The gaze information storage unit (30) can output the signal to the visual field blocking device (15).

[0091] The computing device (3) can output a signal to the gaze information storage unit (30) to block or unblock the visual field of the subject's right eye ((e') of FIG. 9). The gaze information storage unit (30) can output the signal to the visual field blocking device (15).

[0092] When the above-mentioned visual field blocking device (15) blocks the visual field of the left or right eye of the subject, the computing device (3) can output a signal to the ocular movement ability information generating device (1, 1-1, 1-2, 1-3) to perform an ocular movement ability test of the left or right eye (Fig. 9 (f)).

[0093] During the above ocular movement ability information generation process, the computing device (3) can receive image data of the target (2) from the coordinate calculation unit (20) and measurement values ​​for the gaze direction from the gaze information storage unit (30). The computing device (3) can display the image of the target (2) and information for the gaze direction as images. In (g) of Fig. 9, the image of the target (2) and the estimated gaze position for the measurement value measured during the examination of the left eye of the subject can be displayed. In (g') of Fig. 9, the image of the target (2) and the estimated gaze position for the measurement value measured during the examination of the right eye of the subject can be displayed. When the ocular movement ability test for the left or right eye of the subject is completed, the computing device (3) can output a test completion signal to the ocular movement ability information generation device (1, 1-1, 1-2, 1-3). Thereafter, the computing device (3) can digitize the examination results of the left or right eye of the subject (Fig. 9 (h)).

[0094] The above computing device (3) can display the test results of the left eye of the subject ((i) of FIG. 9)) or the test results of the right eye (i' of FIG. 9)).

[0095] The computing device (3) can receive a graph of the gaze positions of the left and right eyes output from the graph output unit (60). The computing device (3) can output a result sheet ((j) of FIG. 9) including the graph, the deviation value of the left eye for each grid point, and / or the deviation value of the right eye for each grid point. The computing device (3) can output the result sheet in the form of a document file and / or an image file, which is an example and is not limited by a specific file format.

[0096] The computing device (3) can output a signal for initializing the generated eye movement ability information, the angle information of the subject's head, and the setting information of the RGB camera (11), the depth sensor (12), and / or the eye tracking sensor (13). In addition, the computing device (3) can output information on the communication status and / or the examination status with the eye movement ability information generating device (1, 1-1, 1-2, 1-3).

[0097] The method of linking the above computing device (3) and the eye movement ability information generation device (1, 1-1, 1-2, 1-3) is an example and is not limited by the above method.

[0098] This specification assumes that the subject's ocular motor ability is tested using the Hess Screen Chart test. However, this is not limited to the Hess Screen Chart test, and it is clear that various embodiments can be created depending on the type of ocular motor ability test, such as the Lancaster red-green test.

[0099] The above coordinate calculation unit (20), gaze information storage unit (30), timing storage unit (40), motor ability calculation unit (50), and graph output unit (60) may include a processor, ASIC (application-specific integrated circuit), other chipset, logic circuit, register, communication modem, data processing device, etc. known in the technical field to which the present invention pertains in order to execute calculation and various control logic. In addition, when the above-described control logic is implemented in software, the coordinate calculation unit (20), gaze information storage unit (30), timing storage unit (40), motor ability calculation unit (50), and graph output unit (60) may be implemented as a set of program modules. At this time, the program modules may be stored in the memory device and executed by the processor.

[0100] Hereinafter, a method for generating eye movement ability information using an eye movement ability information generating device (1, 1-1, 1-2, 1-3) according to the present specification will be described. However, in describing the method for generating eye movement ability information according to the present specification, repetitive descriptions of each component are omitted.

[0101] The method for generating eye movement ability information according to the present specification is a method for generating eye movement ability information using a device including an RGB camera (11), a depth sensor (12), and an eye tracking sensor (13) in a housing (10) of a predetermined shape mounted on the head of a subject.

[0102] FIG. 10 is a flowchart of a method for generating eye movement ability information according to one embodiment of the present specification.

[0103] Referring to FIG. 10, when the subject wearing the housing (10) gazes at the target (2) in step S10, the processor may receive image data of the target from the RGB camera (11). The processor may calculate coordinates for a plurality of grid points from the image data. When the subject gazes at the target with one of the two eyes blocked in step S11, the processor may receive and store a measurement value for the gaze direction of the eye whose field of vision is blocked from the eye tracking sensor (13). When the subject gazes at a target presented at each grid point in step S12, the processor may store timing information for gaze at each grid point. In step S13, the processor may receive distance information between the subject and the target (2) from the depth sensor (12). The processor may calculate the gaze error of the subject for each grid point using the distance information, timing information, and gaze tracking measurement values.

[0104] In the above step S13, the processor can calculate an average value using the gaze tracking measurement values ​​stored at the time when the subject gazes at the grid points. The processor can calculate an estimated position on the target (2) using the average value. The processor can calculate a distance error between the estimated position and each grid point. In addition, the processor can calculate a direction error according to each grid point using the distance information and the estimated position. In addition, the processor can calculate a total error including both the distance error and the direction error.

[0105] Fig. 11 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0106] Referring to Fig. 11, steps S20, S21 and S23 are identical to steps S10, S11 and S13, so a repetitive description is omitted.

[0107] In step S22, the processor can receive image data in which a target is presented at each grid point from the RGB camera (11). The processor can store timing information according to the target presented at each grid point location.

[0108] Fig. 12 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0109] Referring to Fig. 12, steps S30, S31 and S33 are identical to steps S20, S21 and S23, so a repetitive description is omitted.

[0110] A housing (10-1) according to another embodiment of the present disclosure may further include a microphone (14) for receiving an audio signal. The processor may receive an audio signal from the microphone (14). The audio signal may correspond to an examiner's voice signal and / or a beep sound. Upon receiving the audio signal, the processor may store the timing at which the examinee gazes at each grid point.

[0111] Fig. 13 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0112] Referring to Fig. 13, steps S40 and S42 to S44 are identical to steps S30 to S33, so a repeated description is omitted.

[0113] According to another embodiment of the present specification, the housing (10-2) may further include a field of view blocking device (15) that individually blocks the field of view of the subject. In step S41, the processor may output an operating signal to the field of view blocking device to block the field of view of one eye of the subject.

[0114] Fig. 14 is a flowchart of a method for generating eye movement ability information according to another embodiment of the present specification.

[0115] Referring to Fig. 14, steps S50 to S54 are identical to steps S40 to S44, so a repetitive description is omitted.

[0116] In step S55, the processor can output information on the gaze of the subject's left and right eyes as a graph.

[0117] The method for generating ocular movement ability information according to the present specification may be implemented in the form of a computer program written to perform each step and recorded on a computer-readable recording medium. The aforementioned computer program may include codes coded in a computer language, such as C / C++, C#, JAVA, Python, or machine language, that can be read by the processor (CPU) of the computer through the device interface of the computer, so that the computer reads the program and executes the methods implemented as a program. Such codes may include functional codes related to functions that define functions necessary for executing the methods, and may include control codes related to execution procedures necessary for the processor of the computer to execute the functions according to a predetermined procedure. In addition, such codes may further include memory reference-related codes regarding which location (address address) of the internal or external memory of the computer should reference additional information or media necessary for the processor of the computer to execute the functions. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to send and receive during communication.

[0118] The above storage medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the program can be stored in various recording media on various servers that the computer can access or in various recording media on the user's computer. In addition, the medium can be distributed across network-connected computer systems, so that computer-readable code can be stored in a distributed manner.

[0119] While the embodiments of this specification have been described with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical spirit or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

[0120] [Explanation of symbols]

[0121] 1: Eye movement ability information generation device

[0122] 2: Timetable

[0123] 10: Housing

[0124] 11: RGB camera

[0125] 12: Depth sensor

[0126] 13: Eye tracking sensor

[0127] 14: Microphone

[0128] 15: Vision blocking device

[0129] 20: Coordinate calculation section

[0130] 30: Gaze information storage unit

[0131] 40: Timing storage unit

[0132] 50: Athletic ability calculation section

[0133] 60: Graph output section

[0134] 70: Eye movement ability information generation system

Claims

1. A method for generating eye movement ability information using a device including an RGB camera, a depth sensor, and an eye tracking sensor in a housing of a predetermined shape mounted on the head of a subject. (a) a step in which a processor calculates coordinates for a plurality of grid points from image data captured by a motion chart input from the RGB camera; (b) a step in which the processor receives from the eye tracking sensor a measurement value of the gaze direction of one of the subject's two eyes whose field of vision is blocked and stores the measurement value; (c) a step of the processor storing the timing (hereinafter referred to as “timing information”) at which one of the subject’s two eyes with open field of vision fixates on a grid point included in the ocular movement chart; and (d) A method for generating eye movement ability information, comprising: a step of calculating the eye movement error of the subject for each grid point using the distance information between the subject and the target, the timing information, and the eye tracking information input from the depth sensor.

2. In claim 1, The above housing further includes a microphone for receiving an audio signal; Step (c) above, A method for generating eye movement ability information, wherein the processor receives an audio signal from a microphone or information on a target image presented at the grid point from an RGB camera, and stores the timing information.

3. In claim 1, The housing further includes a field of view blocking device that individually blocks the field of view of both eyes of the subject; A method for generating eye movement ability information, characterized in that the step (b) above comprises the processor outputting a blocking signal to a field of view blocking device to block the field of view of one eye of the subject.

4. In claim 1, Step (d) above, A method for generating eye movement ability information, wherein the processor is a step of quantifying at least one of the average distance error between the subject's gaze positions for each grid point, the direction error for each grid point, and the overall error.

5. In claim 1, After step (d) above, (e) A method for generating eye movement ability information, further comprising a step of the processor outputting a graph of the position of the subject's gaze according to each grid point.

6. A computer program written to perform each step of the method for generating eye movement ability information according to any one of claims 1 to 5 on a computer and recorded on a computer-readable recording medium.

7. A housing of a predetermined shape that is mounted on the subject's head and includes an RGB camera, a depth sensor, and an eye tracking sensor; A coordinate calculation unit that calculates coordinates for a plurality of grid points from image data captured by a motion chart input from the RGB camera; A gaze information storage unit that receives and stores measurements of the gaze direction of one of the subject's two eyes, the visual field of which is blocked, from the eye tracking sensor; A timing storage unit that stores the timing (hereinafter referred to as “timing information”) at which one of the subject’s eyes with open visual field gazes at a grid point included in the ocular movement chart; and An eye movement ability information generating device, comprising: a movement ability calculation unit that calculates the eye movement error of the subject for each grid point using distance information between the subject and the target, the timing information, and the eye tracking information input from the depth sensor.

8. In claim 7, The above housing further includes a microphone for receiving an audio signal; The above timing storage unit, An eye movement ability information generating device that receives an audio signal from a microphone or information on a target image presented at the grid point from an RGB camera and stores the timing information.

9. In claim 7, The housing further includes a field of view blocking device that individually blocks the field of view of both eyes of the subject; The above gaze information storage unit is, An ocular movement ability information generating device characterized in that it blocks the field of vision of one eye of the subject by outputting a blocking signal using a field blocking device.

10. In claim 7, The above exercise ability calculation unit is, An eye movement ability information generating device that quantifies at least one of the average distance error between the subject's gaze positions for each grid point, the direction error for each grid point, and the overall error.

11. In claim 7, An eye movement ability information generating device further comprising a graph output unit that outputs a graph of the position of the subject's gaze according to each grid point.

12. An eye movement ability information generating device according to any one of claims 7 to 11; An intraocular movement chart with multiple grid points; A target generating device that presents a target image to the above grid points; A computing device that outputs an operation signal of the above eye movement ability information generating device and receives eye movement ability information; and An eye movement ability information generation system, comprising: a communication device for transmitting and receiving signals or information between the eye movement ability information generation device, a computing device, and a target generation device.

Images