Computer system for screening, evaluating and rehabilitating oculomotor imbalances and for improving and training visual comfort and performance

Abstract

The invention relates to a computer system for analyzing, correcting, and / or training human vision, comprising a virtual reality device (1i); a computer terminal (2k) capable of executing a computer program (W); a computer server (4) comprising a message agent (41), characterized in that the server is configured to, upon receipt by the message agent of signaling messages emitted by the device and the terminal, establish a direct bidirectional communication channel (WebRTC) between the device and the terminal; and in that, in response to a notification of a message triggering an eye exercise (Ej) transmitted by the terminal to the message agent, the device is configured to trigger the broadcast of a stream (Fj) containing this exercise and to transmit this stream to the terminal, via said channel, for display on the computer program. Figure to be published with the abbreviation: Fig. 1

Description

Title of the invention: Computer system for screening, evaluating and rehabilitating oculomotor imbalances and for improving and training visual comfort and performance

[0001] The invention relates to the field of vision, and in particular to ophthalmology, orthoptics, optics and optometry. More specifically, the invention relates to the field of screening, assessment and rehabilitation of oculomotor disorders and imbalances, assessment and improvement of visual comfort, analysis and training of visual performance and more generally to understanding the visual behavior of a subject by means of virtual reality devices.

[0002] It is known to diagnose and / or re-educate an oculomotor imbalance in a patient by eye exercises which are performed by the patient through a virtual reality environment.

[0003] Document EP3986245 describes, for example, a method for measuring a patient's eye convergence, implemented using a virtual reality headset equipped with an eye tracker through which the patient is immersed in a virtual environment. In this environment, an animation creating the effect of an object leaping towards the patient is displayed, which elicits a convergence demand from the patient's eyes. The patient's eye convergence response can then be measured by the eye tracker to facilitate the practitioner's diagnosis.

[0004] This type of solution is thus clearly successful, given the advantages it offers in terms of practitioner concentration during eye exercises, the precision and objectivity of the data enabling diagnosis, and the modernity it brings to the fields of ophthalmology and orthoptics. However, its adoption remains limited due to the cost of equipment that can immerse a patient in a virtual environment and the difficulty of developing an eye test within such a virtual environment.

[0005] Furthermore, there is a need to be able to control the exercises that the patient must perform and to visualize what the patient sees within the virtual environment, and even to visualize the data collected by the various sensors of the virtual reality device. It is therefore necessary, in addition to the device itself, to provide software that can be run on a computer terminal and that allows the practitioner to trigger and control eye exercises within the device, at their request, and to provide a rendering of these exercises on this computer terminal.

[0006] Given the resources it requires and its complexity, this type of software cannot be developed on demand for each practitioner, nor installed directly on the practitioner's computer terminal, but must instead be offered as a service accessible via the practitioner's computer terminal from a remote server. However, this IT architecture presents several problems.

[0007] On the one hand, the solution must be usable by different people and for different purposes. Indeed, beyond orthoptists and ophthalmologists, the solution can be used by pharmacists or opticians for screening purposes, or even by patients themselves for home rehabilitation. Therefore, the architecture must be able to manage a fleet of virtual reality devices and link each device to the service on the remote server, while adapting the service's functionalities to the user's profile. The modularity and number of service functions relative to the number of devices in the fleet can thus complicate the IT architecture.

[0008] There are still other use cases involving the use of virtual reality in the field of human vision that can further complicate the computer architecture if their management were to be centralized. These include assisting an optician in choosing a client's vision correction, helping to improve a subject's visual comfort, and analyzing and training an athlete's visual performance.

[0009] Furthermore, to avoid overloading the computer terminal's memory, it is preferable for the data streams corresponding to the exercises to be stored in the virtual reality device itself. However, the rendering functionality, on the computer terminal, of the exercise that is broadcast in the virtual reality device implies that the broadcast data stream must be transmitted to the computer terminal for display. In the case of a fleet of virtual reality devices, the IT architecture thus requires that each data stream broadcast on a device pass through the server to be redirected to the appropriate computer terminal. The amount of data that the server must then manage is therefore far too large for the IT architecture to be feasible.

[0010] There is therefore a need for a computer system for analyzing, correcting and / or training human vision, of simple design, offering, for a fleet of virtual reality devices, a centralized service for controlling the distribution of eye exercises or tests in a virtual reality device and for visualizing these exercises or tests and their results on a computer terminal associated with this device, with reasonable resource consumption and which allows any professional, particularly in the field of human vision, to orthoptics, ophthalmology or optometry, to use a virtual reality device through this computer system.

[0011] The present invention falls within this context and aims to meet this need.

[0012] To this end, the invention relates to a computer system for analyzing, correcting and / or training human vision, characterized in that it comprises:

[0013] a virtual reality device intended to be mounted on the head of a subject and comprising a control unit, a first communication unit and a memory in which is stored at least a first stream intended to generate a virtual reality environment in which the subject can be immersed and comprising an exercise and / or an eye test in the virtual reality environment requiring eye movement of the subject;

[0014] a computer terminal comprising a display device and a second communication unit, the computer terminal being capable of executing a computer program for controlling and visualizing the execution of eye tests and / or exercises by the subject using the virtual reality device;

[0015] a computer server comprising a third communication unit equipped with a message agent, each of the first and second communication units being capable of sending messages to the message agent, the message agent being arranged to notify a message received from one of the first and second communication units to the other of the first and second communication units.

[0016] The system according to the invention is characterized in that:

[0017] the third communication unit is arranged so that, upon receipt by the agent of signaling messages issued by the first and second communication units, it establishes a direct bidirectional communication channel between the first communication unit and the second communication unit;

[0018] in response to a notification of a trigger message for said exercise and / or eye test transmitted by the second communication unit to the message agent, the control unit is arranged to trigger the broadcast of the first stream on the virtual reality device and the first communication unit is arranged to transmit to the second communication unit, via said channel, the first stream broadcast for its display on a computer program visualization interface on the display device of the computer terminal.

[0019] It is thus understood that the invention proposes to retain the advantages of a virtual reality device control solution based on access, via a computer terminal, to a computer program, such as a service that can, for example, be located on a remote server. In order to control the virtual reality device from the computer terminal, each of these two elements is signaled with a message agent on a computer server. This message agent acts as a relay, forwarding messages transmitted to it by one communication unit to other communication units. For example, a user action within the computer program could trigger the transmission of a signaling message from the second communication unit to the message agent, which would then notify the first communication unit of this message. This message constitutes a coupling request. The first communication unit can then, in response, send its own signaling message to the message agent, this message constituting a positive response to the coupling request.

[0020] The computer server can then trigger the opening of a bidirectional communication channel between the virtual reality device and the computer terminal, in a peer-to-peer manner. This channel allows the virtual reality device and the computer terminal to exchange data streams in real time, without an intermediary server. Thus, the user of the computer terminal can access all the functionalities of the computer program and can therefore control the delivery of one or more exercises and / or eye tests in the virtual reality device via the computer terminal. For example, it could be stipulated that an action by the user with the computer program triggers the transmission of a message from the second communication unit to the message agent to initiate an exercise and / or eye test, which then notifies the first communication unit of this message.The control unit can then trigger, in response, the broadcast of the corresponding data stream.

[0021] When the eye exercise and / or test is broadcast in the device, the data stream corresponding to this exercise and / or test is transmitted to the computer terminal via the communication channel, which then allows the computer program to display this data stream on an interface. The user thus has a tool for visualizing the exercises and tests and their results without consuming significant resources on a computer system server.

[0022] In the present invention, the term "virtual reality device" means, without limitation, a virtual, augmented, or hybrid reality device, such as a headset, a pair of glasses, or any other device intended to be worn on the subject's head and allowing them to be immersed in a virtual reality. Such a headset may, for example, be equipped with a stereoscopic head-mounted display (HMD). Where applicable, the device may include a single display for both of the subject's eyes, or conversely, two separate displays, each intended for one of the subject's eyes.

[0023] In the present invention, the term "computer terminal" means, without limitation, a tablet, a smartphone, a laptop computer, or any electronic device comprising a screen, or a display and a control and / or input interface capable of being manipulated by a user, such as a touch interface or a keyboard and mouse.

[0024] In the present invention, "communication unit" means, without limitation, any electronic and / or logic component or any set of electronic and / or logic components enabling the exchange of data, in an analog or digital manner, with another communication unit, via a wireless communication network, for example of the Wi-Fi, 4G or LTE, or 5G type.

[0025] In the present invention, the term "message agent" means, without limitation, one or more computer programs installed on one or more servers and capable of receiving, storing, and relaying or routing messages it receives to one or more receivers. A message agent, also called a "message broker," may, for example, manage a queue. The message agent may, for example, operate in publish / subscribe mode, in which a communication unit may subscribe to a topic of the message agent in order to receive messages published on that topic, and / or publish messages on a topic of the message agent so that these messages are notified to the units subscribed to that topic.

[0026] In the present invention, the term "computer program" means a set of instructions intended to be executed by a computer or computer system to enable a user to control a virtual reality device and view content displayed on that device or data originating from the device. It may be software installed, partially or fully, on the computer terminal and / or hosted, partially or fully, on one or more remote servers, and with which a user can interact via a user interface accessible on the computer terminal. The user interface may, in particular, be an application designed to be used via a web browser.

[0027] In the present invention, the term "stream" means, without limitation, an immersive video stream intended to be displayed on a screen. The immersive video stream may include a 360° video comprising views of the same scene or environment from a plurality of directions, or even from a plurality of positions, at the same instant. It may consist of live-captured images or computer-generated images. The immersive video stream may be intended to be displayed alone, in which case the stream defines both a scene and an exercise and / or an eye test, or it may be intended to be superimposed on another stream, the stream defining the exercise and / or the eye test and the other stream defining a scene. In the case of superimposition, the immersive video stream may include only one or more objects intended to be positioned in a configurable manner within a scene defined by another video stream. The parameters defining, in particular, the position of the object(s) in the scene over time, the trajectory and speed of movement of the object(s) in the scene over time, and the size of the object(s) over time. This immersive video stream can be stored in the virtual reality device's memory in various forms. For example, the immersive video stream can be stored as one or more image sequences, notably in MPEG4 format. Alternatively, the immersive video stream can be calculated in real time by a 3D engine in the control unit, from one or more 3D objects stored in memory, notably in polygonal or vector form.Alternatively, the immersive video stream could be composed of elements calculated in real time by a 3D engine of the control unit from 3D objects stored in memory, and pre-calculated elements stored in memory, such as scenes or 360° backgrounds onto which the elements calculated in real time are superimposed.

[0028] In the present invention, the term "virtual reality environment" means, without limitation, an immersive environment of a virtual, augmented, or hybrid type. A virtual reality device generates, from an immersive video stream, images that simulate the physical presence of an individual in a virtual reality environment and / or that superimpose virtual objects onto a real environment. It is thus possible to explore a virtual reality environment, to a certain extent, for example by directing one's gaze in different directions or by moving within the environment, for example, using a controller.

[0029] In the present invention, "eye exercise and / or test" means, without limitation, the display of an animation of one or more objects in the virtual reality environment. The animation allows for the evaluation of an ocular mechanism, the detection of an oculomotor disorder, or the evaluation and / or training of the movement of one or both of the subject's eyes. The animation may, for example, include the appearance or disappearance of an object, a change in the object's visual appearance, such as its shape, dimensions, color, or transparency, the movement of an object, or a change in the object's position in space. An exercise or test may be predetermined, with all the animation parameters being set beforehand and stored in the virtual reality device's memory, or it may be configurable, with at least one of the animation parameters being set or controlled by the professional via the computer terminal.

[0030] Advantageously, the message agent is a "message queuing telemetry transport" (MQTT) type agent. MQTT stands for "Message Queuing Telemetry Transport". This type of agent offers several advantages, including low bandwidth requirements and relatively low latency, which make it particularly relevant for managing a fleet of virtual reality devices. virtual.

[0031] Advantageously, the third communication unit is arranged so that, upon the agent's receipt of signaling messages emitted by the first and second communication units, it establishes a direct, bidirectional communication channel of the "Web Real-Time Communication" (WebRTC) type between the first and second communication units. This technology allows for peer-to-peer transmission and reception of data from a web browser, making it advantageous when the computer program includes a web application.

[0032] In one embodiment of the invention, the messaging agent is of the publish / subscribe type. When the virtual reality device is activated, the first communication unit is configured to subscribe to a predetermined topic of the messaging agent associated with an identifier of the virtual reality device. The computer program may include an interface for selecting the virtual reality device, and the second communication unit is configured to publish, in response to the device selection via said selection interface, a first reporting message on the topic of the messaging agent associated with the identifier of the selected device. If necessary, the first communication unit may be configured to publish, upon receiving a notification of said first reporting message on the topic to which it is subscribed, a second reporting message on that topic.Preferably, the second communication unit is also arranged to subscribe to the subject of the message agent associated with the virtual reality device identifier.

[0033] According to one embodiment of the invention, the computer program can be configured to display, on the selection interface, a list of virtual reality devices associated with a user account that has logged into the computer program. During the initial setup of a virtual reality device, a user account identifier can, for example, be provided to the virtual reality device, so that the user account and the device are associated with each other in a database of the computer system. The second communication unit can thus be configured to publish, in response to the selection of the device via said selection interface when the computer program is executed with a given user account, a reporting message incorporating an identifier of said user account on the subject of the message agent associated with the identifier of the virtual reality device.The server can thus control the association, at the database level, of the user account connected to the computer program executed at the computer terminal and the virtual reality device, and if necessary establish a bidirectional communication channel. direct between the first communication unit of this computer terminal and the second communication unit of this virtual reality device.

[0034] Advantageously, the computer terminal can be arranged to control the virtual reality device through predetermined subjects of the message agent associated with an identifier of said virtual reality device, the second communication unit being thus arranged to publish control messages, such as exercise and / or eye test trigger messages, on a subject of the message agent associated with the identifier of said virtual reality device to which the first communication unit is subscribed.

[0035] In one embodiment of the invention, the computer system comprises a plurality of virtual reality devices, each intended to be mounted on the head of a subject. The first communication unit of each virtual reality device can be configured, upon commissioning of said virtual reality device, to subscribe to a predetermined subject of the message agent associated with an identifier of said virtual reality device, and the second communication unit is configured to publish, in response to the selection of a virtual reality device from a list of devices via said selection interface, a first reporting message on the subject of the message agent associated with the identifier of the selected device.Where appropriate, each first communication unit is arranged to publish, upon receipt of a notification of said first reporting message on the subject to which it is subscribed, a second reporting message on that subject.

[0036] It is thus understood that the computer system can manage, via a single messaging agent, a plurality of virtual reality devices, each device having, for example, a distinct identifier known to the computer system at the server level and being associated with a user account in a server database. If necessary, for each pair of virtual reality devices and computer terminals, the server can establish a direct bidirectional communication channel between the communication units of that pair.

[0037] In one embodiment of the invention, the virtual reality device includes at least one head motion sensor and / or an eye tracker. During the transmission of the first stream on the virtual reality device, the first communication unit can be configured to transmit, via said channel, the first stream and the data acquired by the motion sensor and / or the eye tracker to the second communication unit for display on the computer program's display interface on the computer terminal's display device. Beyond providing the professional with a rendering of the exercise and / or test being transmitted on the virtual reality device, this feature allows the display of data enabling the professional to monitor the exercise and / or test and to visualize the results of the exercise and / or test. The data displayed may be raw data acquired by the sensor and / or eye tracker or data calculated from this raw data, at the level of the virtual reality device or the computer terminal.

[0038] Advantageously, the computer program is arranged to display, on the program's display interface on the computer terminal's display device, a graphical representation of the subject, and to modify the orientation and / or gaze of said graphical representation as the second communication unit receives data acquired by the motion sensor and / or eye tracker. The data can thus be displayed both on a graph allowing, for example, monitoring of its evolution over time during the exercise and / or test, and on a real-time digital avatar of the subject.

[0039] In one embodiment of the invention, the control unit of the virtual reality device is capable of assuming a plurality of predetermined states. The first communication unit can be arranged to transmit a state change message to the message agent at each change of state of the control unit, and the computer program is arranged, in response to each notification of a state change message transmitted by the first communication unit to the message agent, to modify the visualization interface of the computer program displayed on the display device of the computer terminal according to said state change.The software can thus automatically modify the visualization interface according to the state of the virtual reality device, in order to offer an interface adapted depending on whether the device is being calibrated by the subject, displaying an exercise and / or test, in an inactive or powered-off state, or in a standby state. The use of the messaging agent to transmit the device's state allows for a virtually instantaneous response from the software to a change in the device's state.

[0040] Advantageously, a plurality of streams are stored in said memory, each stream being intended to generate a virtual reality environment in which the subject can be immersed and comprising a predetermined exercise and / or eye test within the virtual reality environment requiring eye movement by the subject. The computer program may include an interface for selecting one or more exercises and / or eye tests from a predetermined list of exercises and / or eye tests; and the second communication unit is arranged to transmit to the message agent, in response to the selection of at least one exercise and / or eye test on said interface, a trigger message for said selected exercise and / or eye test. Preferably, in response to notification by the agent of messaging to the first communication unit of said trigger message, the control unit is arranged to trigger the broadcast, on the virtual reality device, of the stream stored in said memory and comprising said exercise and / or selected eye test.

[0041] Preferably, said predetermined list of eye exercises and / or tests may be recorded in a database of the server hosting the computer program by being associated with an identifier of the user account that logged into the computer program.

[0042] Each stream of said plurality of streams may include one or more or a combination of the following eye exercises and / or tests:

[0043] A visual acuity exercise and / or test, in which a sequence of letters or patterns is displayed in the virtual reality environment, the dimensions of the letters or patterns changing during the display of the sequence, for example by decreasing. It may be provided, in particular, that during the display of the sequence, only one of the displays of the virtual reality device, corresponding to one of the subject's eyes, is active, the other being covered or inactive;

[0044] A stereoscopic vision exercise and / or test, in which an object is successively displayed in different locations of the virtual reality environment;

[0045] A convergence exercise and / or test, in which an object is displayed in the virtual reality environment while moving along a depth axis, for example by gradually approaching the subject and then moving away;

[0046] An exercise and / or a fusion or amplitude test in convergence and divergence, in which an object is displayed in the virtual reality environment moving along different trajectories and / or speeds for each display of the virtual reality device, and therefore for each eye of the subject;

[0047] An eye-tracking exercise and / or test, in which an object is displayed in the virtual reality environment moving along a curved trajectory, in particular a circular one;

[0048] An exercise and / or test of eye saccades, in which two juxtaposed objects are displayed in the virtual reality environment by blinking alternately;

[0049] An exercise and / or test of physiological diplopia, in which two objects are displayed in the virtual reality environment, one behind the other along a depth axis, blinking alternately.

[0050] In the event that several exercises and / or eye tests are selected via the selection interface, the second communication unit can be configured to automatically transmit to the message agent a trigger message for each exercise and / or eye test selected according to their order of selection, each trigger message being transmitted in response to the receipt of a notification of an end-of-exercise message from the previous exercise and / or test by the messaging agent. Alternatively, the second communication unit can be configured to transmit to the messaging agent a trigger message for each exercise and / or eye test selected following validation submitted by the professional via the selection interface.

[0051] In one embodiment of the invention, the computer system is characterized in that:

[0052] Following the commissioning of the virtual reality device, the control unit is arranged to assume a standby state for an exercise and / or test; and, in response to a notification of a message changing the state to a standby state, the computer program is arranged to add to the computer program's viewing interface said interface for selecting one or more eye exercises and / or tests from a predetermined list of eye exercises and / or tests;

[0053] In response to a notification of a trigger message for an exercise and / or eye test selected via the selection interface, the control unit is arranged to borrow a broadcast state of an exercise and / or test and to trigger the broadcast on the virtual reality device of a stream stored in said memory and comprising said exercise and / or eye test selected; and, in response to a notification of a message to change state to a broadcast state of an exercise and / or test, the computer program is arranged to modify the computer program's visualization interface according to a predetermined configuration associated with the selected exercise and / or eye test.

[0054] For example, a waiting stream could be stored in the memory of the virtual reality device, the stream being intended to generate a virtual reality waiting environment in which the subject can be immersed. The control unit can thus be configured, when it enters the waiting state, to trigger the broadcast of said waiting stream on the virtual reality device.

[0055] Preferably, the computer program may be arranged to add an interface for visualizing the data acquired by the motion sensor and / or the eye tracker of the virtual reality device and transmitted to the computer terminal via the channel.

[0056] Advantageously, at the end of the broadcast of the last selected exercise and / or eye test, the control unit is configured to assume the standby state. The first communication unit can thus be configured to transmit, to the messaging agent, a broadcast status message for an exercise and / or test, said message being able, for example, to indicate the start or end of the broadcast of the exercise and / or test, that the exercise and / or test is being broadcast, or that a response from the subject to a question is expected.

[0057] In one embodiment of the invention, at least one variable stream for generating a virtual reality environment in which the subject can be immersed and which includes a configurable exercise and / or eye test is stored in said memory. Where applicable:

[0058] in response to a notification of a trigger message for said configurable exercise and / or eye test transmitted by the second communication unit to the message agent, the control unit is arranged to trigger the broadcast of the variable stream on the virtual reality device;

[0059] in response to a notification of a message of change of state to a broadcast state of an exercise and / or a variable test, the computer program is arranged to add to the visualization interface of the computer program an interface for controlling the variable parameters of the selected configurable exercise and / or eye test;

[0060] Following a request to modify a parameter according to a given instruction via this control interface, the second communication unit is configured to transmit to the message agent a modification message for said parameter containing said instruction; and;

[0061] In response to a notification of a parameter modification message containing a setpoint, the control unit is arranged to modify the diffusion of the variable stream on the virtual reality device according to said parameter and said setpoint.

[0062] This embodiment thus makes it possible to broadcast a stream without time limit, for example to immerse the subject in a looping scene, and to allow the profession to carry out a personalized exercise or test in which a parameter can be controlled in real time, for example to modify the size of an object, the direction of its trajectory or its speed of movement.

[0063] In one embodiment of the invention, the virtual reality device includes at least one sensor capable of detecting whether the virtual reality device is mounted on the subject's head. If so, in response to the sensor's detection that the subject is wearing the virtual reality device, the control unit is configured to assume a calibration state and to trigger the transmission to the virtual reality device of a calibration stream stored in said memory and comprising a calibration procedure for the virtual reality device. This calibration procedure may be a calibration procedure for the motion sensor of the virtual reality device and / or a calibration procedure for the eye tracker of the virtual reality device.

[0064] If desired, the computer program and / or the control unit of the virtual reality device can be arranged to display, respectively on the computer terminal and in the virtual reality environment, a proposal interface initialization of the calibration procedure, at the end of each broadcast of a stream containing an exercise and / or an eye test.

[0065] The invention also relates to a method of associating a virtual reality device and a computer terminal, implemented by means of a computer system according to the invention.

[0066] The invention also relates to a method of displaying, on a display device of a computer terminal, a stream broadcast on a virtual reality device and / or data acquired by one or more sensors of a virtual reality device, implemented by means of a computer system according to the invention.

[0067] The present invention is now described by means of purely illustrative and in no way limiting examples of the scope of the invention, and from the accompanying drawings, in which the various figures represent:

[0068] [Fig-1] represents, schematically and partially, a view of an in system format for analyzing, correcting and / or training human vision according to an embodiment of the invention;

[0069] [Fig.2] represents, schematically and partially, a view of the system in format of [Fig.1] during a step of associating a virtual reality device and a computer terminal;

[0070] [Fig.3] represents, schematically and partially, a view of the system in format of the [Fig.1] at the beginning of the broadcast of a stream on the virtual reality device;

[0071] [Fig.4] represents, schematically and partially, a view of the system in format of [Fig.1] at the end of the broadcast of a stream on the virtual reality device.

[0072] In the following description, identical elements, whether structural or functional, appearing in different figures retain, unless otherwise specified, the same reference numerals. Of course, various other modifications may be made to the invention within the scope of the appended claims.

[0073] A computer system for analyzing, correcting and / or training human vision is represented in [Fig.1] according to an example of an embodiment of the invention.

[0074] The computer system comprises a plurality of virtual reality devices 1; each intended to be mounted on the head of a subject.

[0075] In the example described, it refers to one of these devices 1; This device is a helmet equipped with a visor head-mounted display 11 comprising two separate displays, each intended for one of the subject's eyes, it being understood that other types of virtual, augmented or hybrid reality devices may be used without departing from the scope of the present invention.

[0076] The device h also includes a control unit 12 equipped with a memory in which different video streams Fj are stored. Each video stream Fj is intended to generate a virtual reality environment in which the subject can be immersed when the stream is broadcast on the displays of the head-mounted display 11. Some of these streams Fj also include an exercise and / or an eye test Ej, namely an animation of one or more objects in the virtual reality environment intended to generate a response or a movement of one or both of the subject's eyes, the animation then making it possible to evaluate an ocular mechanism or to screen for an oculomotor disorder or to evaluate and / or trigger a movement of one or both of the subject's eyes.

[0077] In the description that follows, reference will be made to a flow Fi comprising an eye exercise Ei of the convergence exercise type.

[0078] The control unit 12 can take on a plurality of predetermined states indicating the status of the virtual reality device 1; such as an inactive state, a state waiting for an exercise and / or a test, a state of broadcasting an exercise and / or a test or a calibration state.

[0079] In order to allow the user of the computer system to detect an oculomotor disorder in the subject or to monitor the exercises and / or tests performed by the subject, the virtual reality device 1 includes various sensors 14, namely sensors for the subject's head movement, such as accelerometers and gyroscopes, and eye trackers. An eye tracker is defined as any device that detects eye movement and measures the direction of gaze. For example, it may consist of one or more cameras, including infrared cameras, connected to a computer, which may or may not be integrated into the headset 11.

[0080] The virtual reality device 1 also includes a communication unit 13, capable of exchanging data with another communication unit via a wireless communication network, for example of the WiFi, 4G or LTE, or 5G type.

[0081] The computer system also includes a plurality of 2k computer terminals, each intended to be operated by a user of the computer system, such as an orthoptist, an ophthalmologist, an optician or even a pharmacist, or even the subject himself.

[0082] In the example described, it refers to one of these 2k terminals. This terminal is a laptop computer equipped with a 21 screen and a keyboard and mouse (not shown) allowing the user to interact with the 2k terminal, it being understood that other types of computer terminals, such as a touch tablet or a smartphone, may be used without departing from the scope of the present invention.

[0083] The computer terminal is capable of executing a computer program W for controlling and visualizing the execution of eye tests and / or exercises by the subject by means of a virtual reality device 1;

[0084] In the example described, this program W is hosted on a computer server 3 and can be executed by the computer terminal 2k through a web browser, as a service made available by the server 3.

[0085] This type of computer program is also called a web application or The "WebApp" requires no installation on the computer terminal 2k. To access the computer program W, the computer terminal 2k and the computer server 3 are each equipped with a communication unit, 22 and 31 respectively, capable of exchanging information via a wireless communication network. The various control interfaces for the virtual reality device 1, and for viewing the streams Fj and exercises and / or tests Ej broadcast on this device 1, are thus stored on the server 3 and transmitted to the computer terminal 2k via the communication units 2 and 31, after the terminal 2k has requested it, to be viewed on the screen of the computer terminal 2k via the web browser.

[0086] In other words, in the example described, each computer terminal 2k of the computer system can request the computer program W hosted on the server 3. In order to be able to identify and authenticate the users of the computer program W, the server 3 has a database 32, provided on this server or on another dedicated server, in which are stored user account identifiers Ck through which users can connect to the computer program W.

[0087] The computer system also includes a computer server 4 hosting a message agent 41 and a communication unit 42 capable of exchanging data with another communication unit via a wireless communication network. Servers 3 and 4 may be part of the same server group, be a single server, or, conversely, be separate servers.

[0088] In the example described, the message agent 41 is a publish / subscribe agent of the "queueed message transmission" type, also known as an "MQTT broker". This publish agent manages topics, to which devices called "subscribers" can subscribe to be notified when a message is published, and to which devices called "publishers" can publish messages. The message agent 41 can thus notify or relay, via the communication unit 42, a message published by a publisher on a given topic to the subscribers of that topic.

[0089] In the invention, the communication units 13 of each virtual reality device h and 22 of each computer terminal 2k will be able to act both as publishers as subscribers of message agent 41, which is unique to the computer system.

[0090] Still in the example described, the subjects managed by the message agent 41 are stored in a database of the server 4, being organized according to two hierarchical levels, namely on the one hand according to different types of predetermined subjects and on the other hand according to an identifier id; of a virtual reality device 1;.

[0091] More specifically, a subject may relate to signaling messages enabling the coupling or association of a virtual reality device 1 to a computer terminal 2k. The subject will thus be filed in a list of subjects of type "signaling" and will be identified by an identifier id of the virtual reality device 1, for example by being filed as follows: " / signaling / id;"

[0092] A subject may also be related to trigger messages for an exercise and / or an eye test, allowing the broadcast of a stream Fj containing an exercise and / or a test Ej, on a virtual reality device 1;. The subject will thus be placed in a list of subjects of type “trigger” and will be identified by an identifier id; of the virtual reality device 1;, for example by being placed in the following way: “ / control / id;”.

[0093] A subject may also be related to state change messages of the control unit 12 of a virtual reality device U. The subject will thus be placed in a list of subjects of type "state" and will be identified by an identifier id; of the virtual reality device 1;, for example by being placed in the following way: " / state / id;".

[0094] Other types of subjects or another subject hierarchy may be provided without departing from the scope of the present invention.

[0095] In connection with [Fig.2], we will now describe an example of a method for associating a virtual reality device 1 with a computer terminal 2k of the computer system of [Fig.1].

[0096] In advance, during the first commissioning of the virtual reality device 1;, an identifier of a user account Ck, intended to control this device through a computer terminal, may for example be provided to the virtual reality device 1 i, which transmits it to the server 3 so that the user account Ck and the identifier id; of the device 1; are associated with each other at the level of the database 32 of the server 3.

[0097] When the virtual reality device I is put into service, the communication unit 13 subscribes to a signaling subject identified by the device identifier id;, namely the subject / signaling / id; of the message agent 4L

[0098] When the computer program W is executed on a web browser, via a computer terminal 2k, the user authenticates to their user account Ck. The computer program W then displays a viewing interface comprising various elements including a visualization interface W1 of a stream broadcast on the virtual reality device 1; and an interface W2 for selecting a virtual reality device from a list of virtual reality devices associated with the user account Ck at the database level 32.

[0099] When device 1; is selected by the user on interface W2, computer program W retrieves the identifier id; of this device 1; and publishes, via communication unit 22, a first signaling message incorporating an identifier of the user account Ck on the subject / signaling / id; of message agent 41.

[0100] Message agent 41 notifies this message to the subscribers of the subject / signaling / id;, namely the communication unit 13, which publishes in response a second signaling message on this subject / signaling / id; from message agent 4L. The first message thus forms a request for association and the second message thus forms a confirmation of association.

[0101] In response to these reporting messages, the server 4 can then, through the communication unit 42, establish a direct bidirectional WebRTC communication channel between the communication units 13 and 22 of the device h and the computer terminal 2k. This direct bidirectional WebRTC communication channel is of the "real-time communication for the Web" type and allows data exchange between the web browser running on the computer terminal 2k and the control unit 12 of the device h.

[0102] This WebRTC channel will thus allow real-time data exchange, without going through the server 4. Thus, the user of the computer terminal 2k can control the broadcast of exercises and / or eye tests in the virtual reality device 1; and view the results via the computer program W.

[0103] Following the association of the virtual reality device 1 with the computer terminal 2k, the communication units 13 and 22 each subscribe to the subjects / control / idi and / state / id of the message agent 4L

[0104] The control unit 12 then assumes a waiting state for an exercise and / or a test. A waiting stream, stored in the memory of the virtual reality device 1, can be provided for and broadcast on this virtual reality device 1 as long as the control unit 12 is in this waiting state. The subject can thus familiarize themselves with the virtual reality environment in which they are immersed.

[0105] The communication unit 13 then publishes a state change message, indicating this waiting state, on the subject / state / id; of the message agent 4L. In response to the notification of this state change message by the message agent 41 received by the communication unit 22, the computer program W adds to its visualization interface a selection interface W3 of one or more eye exercises and / or tests from a predetermined list of eye exercises and / or tests Ej.

[0106] It may be provided that one or more lists of exercises and / or eye tests Ej are stored in the database and associated with the user account Ck. The lists may also be associated with a particular subject profile, a subject history, or a sub-account of the user counter Ck. The computer program W can thus select one of the lists of exercises and / or eye tests Ej based on the various pieces of information provided by the user during their browsing and display this selected list in the W3 interface.

[0107] Without limitation, the list of eye exercises and / or tests Ej may include a visual acuity exercise or test, a stereoscopic vision exercise or test, a convergence exercise or test, a fusion or amplitude exercise or test in convergence and divergence, an eye tracking exercise or test, a saccade exercise or test, and a physiological diplopia exercise or test. For each of these exercises and / or tests Ej, a stream Fj containing said exercise or test Ej will be stored in the memory of device 1.

[0108] When the user selects via the W3 interface an exercise or a test Ej, such as the convergence exercise Eb, the computer program W transmits, via the communication unit 22, a trigger message of said eye exercise Ei on the subject / control / idi of the message agent 41 and the message agent 41 notifies this message to the communication unit 13.

[0109] In response to this notification, the control unit 12 then assumes the broadcast state of an exercise and / or an exercise and triggers the broadcast, on the virtual reality device 1, of the stream Fi stored in its memory and containing said eye exercise Eh

[0110] Figure 3 shows the computer system at the beginning of the broadcast of the Fi stream and Figure 4 shows the computer system at the end of the broadcast of the Fb stream

[0111] When the state of the control unit 12 changes, the communication unit 13 publishes a state change message, indicating the broadcast state, on the subject / state / id; of the message agent 4L. In response to the notification of this state change message by the message agent 41 received by the communication unit 22, the computer program W adds to its visualization interface a W4 visualization interface of a graphical representation of the subject and a W5 visualization interface of the data acquired by the eye trackers of the virtual reality device 1;.

[0112] It should be noted that other configurations of the display interface of the computer program W can be envisaged, or that the configuration of the display interface of the computer program W can be adapted according to the exercise and / or the eye test selected. In particular, it may be possible to stipulate that certain elements are present from the start of the computer program W, or from the beginning of the association of the computer terminal 2k with the device 1; or that others elements are added, such as elements intended to give instructions to the user.

[0113] As shown at the top of [Fig.3] and [Fig.4], the broadcast stream Fi includes a convergence exercise Eb in which an object O is displayed in the virtual reality environment moving along a depth axis, gradually approaching the subject.

[0114] Throughout the transmission of the Fb stream, the communication unit 13 transmits in real time to the communication unit 22, via the WebRTC channel, the Fi stream broadcast on the displays of the head-mounted display 11 for display on the visualization interface Wl. The user thus has a real-time rendering of the exercise Eb as it is viewed by the subject.

[0115] Concurrently, the communication unit 13 transmits in real time to the communication unit 22, via the WebRTC channel, the data D acquired by the sensors 14 of the device h during the broadcast of the Fp stream

[0116] Data D acquired by the eye trackers, representing in the described example the instantaneous direction of gaze of each eye of the subject, are displayed as they are received in the form of a graph at the W5 interface.

[0117] In addition, a graphic representation of the subject's head is displayed in the W4 interface, and the W computer program modifies the orientation and gaze of this graphic representation as it receives data acquired by the sensors 14.

[0118] The communication unit 13 also publishes, on the subject / state / id; of the message agent 41, messages indicating the broadcast status of an exercise and / or a test, said message being able, for example, to indicate the start or end of the broadcast of the exercise and / or the test, that the exercise and / or the test is being broadcast, or that a response by the subject to a question is expected.

[0119] It should be noted that if several exercises and / or eye tests Ej are selected via the selection interface W3, the communication unit 22, upon receiving a notification of an end-of-broadcast message for an exercise and / or test published on the subject / state / id; of the message agent 41 by the communication unit 13, can automatically publish a trigger message for the next selected exercise and / or test Ej on the subject / control / id; of the message agent 41, according to the order selected by the user. Alternatively, the broadcast order can be defined according to a sequence of exercises and / or tests stored in the database 32 of the server 3 or defined manually by the user. Manual validation at the computer program level W can also be provided to proceed with the broadcast of the next exercise and / or test.

[0120] At the end of the broadcast of exercise Eb or the last exercise and / or eye test Selected, control unit 12 again assumes the waiting state of an exercise and / or test. The waiting stream is broadcast to device 1. A state change message is published by communication unit 13 on the subject / state / id of message agent 41, and the visualization interface of computer program W is again adapted in response to the notification of this message by message agent 41. The user can thus trigger the broadcast of new exercises and / or tests on device 1.

[0121] In an example not shown, it may be foreseen that a calibration procedure for the virtual reality device 1 and its sensors can be initiated by the subject and / or the user each time the control unit 12 enters the standby state for broadcasting exercise and / or test.

[0122] The preceding description clearly explains how the invention achieves its stated objectives, namely, to provide a simple computer system for analyzing, correcting, and / or training human vision, offering, for a fleet of virtual reality devices, a centralized service for controlling the distribution of eye exercises and / or tests within a virtual reality device and for visualizing these exercises and / or tests and their results on a computer terminal associated with this device, with reasonable resource consumption. These objectives are achieved in particular through the use of a single messaging agent capable of linking virtual reality devices and computer terminals to establish a direct bidirectional communication channel between the communication units of each pair of virtual reality devices and computer terminals.Each virtual reality device can thus be controlled by the computer terminal to which it is associated via the messaging agent, while this device and terminal can exchange data directly with each other.

[0123] In any event, the invention is not limited to the embodiments specifically described in this document, and extends in particular to all equivalent means and to any technically feasible combination of these means. In particular, other configurations of virtual reality devices, other exercises and / or eye tests, other types of messaging agents or communication channels may be envisaged.

Claims

1. Demands A computer system for analyzing, correcting and / or training human vision, characterized in that it comprises: a. a virtual reality device (1;) intended to be mounted on the head of a subject and comprising a control unit (12), a first communication unit (13) and a memory in which is stored at least a first stream (Fj) intended to generate a virtual reality environment in which the subject can be immersed and comprising an exercise and / or an eye test (Ej) in the virtual reality environment requiring eye movement of the subject; b. a computer terminal (2k) comprising a display device (21) and a second communication unit (22), the computer terminal being capable of executing a computer program (W) for controlling and visualizing the execution of eye tests and / or exercises by the subject using the virtual reality device; c. a computer server (4) comprising a third communication unit (42) equipped with a message agent (41), each of the first and second communication units (13, 22) being capable of sending messages to the message agent (41), the message agent being arranged to notify a message received from one of the first and second communication units to the other of the first and second communication units; characterized in that: a. the third communication unit (42) is arranged so that, upon receipt by the agent of messages (41) of signaling messages issued by the first and second communication units (13, 22), it establishes a direct bidirectional communication channel (WebRTC) between the first communication unit and the second communication unit; b. in response to a notification of a trigger message for said exercise and / or eye test (Ej) transmitted by the second communication unit (22) to the message agent (41), the control unit (1) is arranged to trigger the broadcast of the first stream (Fj) on the virtual reality device (h) and the first communication unit (13) is arranged to transmit to the second communication unit (22), via said channel (WebRTC), the first stream broadcast for its display on a visualization interface (Wl) of the computer program (W) on the display device (21) of the computer terminal (2k).

2. Computer system according to the preceding claim, characterized in that the message agent (42) is a "message transmission in queue mode" or MQTT type agent.

3. Computer system according to any one of the preceding claims, characterized in that the third communication unit (42) is arranged to, upon receipt by the message agent (41) of signaling messages issued by the first and second communication units (13, 22), establish a direct bidirectional communication channel (WebRTC) of the type "Real-Time Communication for the Web", or WebRTC, between the first communication unit and the second communication unit.

4. A computer system according to any one of the preceding claims, characterized in that the message agent (41) is of the publish / subscribe type; in that, upon commissioning of the virtual reality device (h), the first communication unit (13) is arranged to subscribe to a predetermined subject ( / signaling / id;) of the message agent (41) associated with an identifier (id;) of the virtual reality device (h); in that the computer program (W) comprises a selection interface (W2) for the virtual reality device; in that the second communication unit (22) is arranged to publish, in response to the selection of the device through said selection interface, a first signaling message on the subject ( / signaling / idi) of the message agent (41) associated with the identifier of the selected device;and in that the first communication unit (13) is arranged to publish, upon receipt of a notification of said first alert message on the subject to which it is subscribed, a second alert message on that subject.;

5. Computer system according to the preceding claim, characterized in that it comprises a plurality of virtual reality devices (1;) each intended to be mounted on the head of a subject; in that the first communication unit (13) of each virtual reality device is arranged, when said virtual reality device is put into service, to subscribe to a predetermined subject ( / signaling / id;) of the message agent (41) associated with an identifier (id;) of said virtual reality device; in that the second communication unit (22) is arranged to publish, in response to the selection of a virtual reality device from a list of devices through said selection interface (W2), a first signaling message on the subject of the message agent associated with the identifier of the selected device;and in that each first communication unit (13) is arranged to publish, upon receipt of a notification of said first alert message on the subject to which it is subscribed, a second alert message on that subject.;

6. Computer system according to any one of the preceding claims, characterized in that the virtual reality device (h) comprises at least one subject head motion sensor (14) and / or an eye tracker (14) and in that, during the broadcast of the first stream (Fj) on the virtual reality device, the first communication unit (13) is arranged to transmit to the second communication unit (22), via said channel (WebRTC), the first broadcast stream and the data acquired by the motion sensor and / or the eye tracker (14) for display on the visualization interface (Wl) of the computer program (W) on the display device (21) of the computer terminal (2k).

7. Computer system according to the preceding claim, characterized in that the computer program (W) is arranged to display, on the viewing interface (W1) of the computer program displayed on the display device (21) of the computer terminal (2k), a graphic representation (W4) of the subject, and to modify the orientation and / or the gaze of said graphic representation as the second communication unit (22) receives data acquired by the motion sensor and / or the eye tracker (14).

8. A computer system according to any one of the preceding claims, characterized in that the control unit (12) of the virtual reality device (h) is capable of assuming a plurality of predetermined states, in that the first communication unit (13) is arranged to transmit to the message agent (41), at each change of state of the control unit, a change of state message and in that the computer program (W) is arranged, in response to each notification of a change of state message transmitted by the first communication unit (13) to the message agent (41), to modify the visualization interface (W1) of the computer program (W) displayed on the display device (21) of the computer terminal according to said change of state.

9. A computer system according to the preceding claim, characterized in that a plurality of streams (Fj) are stored in said memory, each stream being intended to generate a virtual reality environment in which the subject can be immersed and comprising a predetermined eye exercise and / or test (Ej) in the virtual reality environment requiring eye movement of the subject; in that the computer program (W) comprises a selection interface (W3) for one or more eye exercises and / or tests from a predetermined list of eye exercises and / or tests; and in that the second communication unit (22) is arranged to transmit to the message agent (41), in response to the selection of at least one eye exercise and / or test on said interface, a trigger message for said selected eye exercise and / or test.

10. A computer system according to the preceding claim, characterized in that: a. following the commissioning of the virtual reality device (li), the control unit (12) is arranged to assume a standby state of an exercise and / or test (Ej); and, in response to a notification of a message of change of state to a standby state, the computer program (W) is arranged to add to the viewing interface (Wl) of the computer program said selection interface (W3) of one or more eye exercises and / or tests (Ej) from a predetermined list of eye exercises and / or tests; b. in response to a notification of a trigger message for an exercise and / or eye test (Ej) selected via the selection interface (W3), the control unit (12) is arranged to assume a broadcast state of an exercise and / or test and to trigger the broadcast on the virtual reality device (h) of a stored stream (Fj) in said memory and comprising said selected exercise and / or eye test; and, in response to a notification of a message of change of state to a broadcast state of an exercise and / or test, the computer program (W) is arranged to modify the viewing interface (W1) of the computer program according to a predetermined configuration associated with the selected exercise and / or eye test.

11. A computer system according to any one of claims 8 to 10, characterized in that the virtual reality device (11) comprises at least one sensor capable of detecting whether the virtual reality device is mounted on the subject's head; and in that, in response to the sensor's detection of the subject wearing the virtual reality device, the control unit (12) is arranged to assume a calibration state and to trigger the broadcast on the virtual reality device of a calibration stream stored in said memory and comprising a calibration procedure for the virtual reality device.

Images