Computer system for screening for, evaluating and rehabilitating oculomotor dysfunction and for improving and training visual comfort and performance

Abstract

The invention relates to a computer system for analysing, correcting and / or training human vision, comprising: a virtual reality device (1i); a computer terminal (2k) capable of executing a computer program (W); and a computer server (4) comprising a message agent (41). The invention is characterised in that the server is designed, upon receipt by the message agent of a report transmitted by the device and the terminal, to establish a direct bidirectional communication channel (WebRTC) between the device and the terminal, and in that, in response to a notification of a message for triggering an ocular exercise (Ej) transmitted by the terminal to the message agent, the device is designed to trigger the broadcasting of a stream (Fj) comprising this exercise and to transmit this broadcast stream via the channel to the terminal, in order for the stream to be displayed on the computer program.

Description

Description Title of the invention: Computer system for screening, evaluating and rehabilitating oculomotor imbalances and improving and training visual comfort and performance

[0001] The invention relates to the field of vision, and in particular ophthalmology, orthoptics, optics and optometry. More specifically, the invention relates to the field of screening, evaluation and rehabilitation of oculomotor disorders and imbalances, evaluation and improvement of visual comfort, analysis and training of visual performance and more generally understanding the visual behavior of a subject by means of virtual reality devices.

[0002] It is known to diagnose and / or re-educate a patient's oculomotor imbalance through eye exercises that are performed by the patient through a virtual reality environment.

[0003] Document EP3986245 describes, for example, a method for measuring the convergence of a patient's eyes, implemented by a virtual reality headset equipped with an eye tracker through which the patient is immersed in a virtual environment. In this environment, an animation creating an effect of an object springing towards the patient is broadcast, which causes a vergential demand for convergence of the patient's eyes. The convergence response of the patient's eyes can then be measured by the eye tracker, in order to facilitate the practitioner's diagnosis.

[0004] This type of solution is therefore clearly successful, given the advantages it provides in terms of the practitioner's concentration during eye exercises, the precision and objectivity of the data enabling diagnosis, and the modernity it brings to the fields of ophthalmology and orthoptics. However, its distribution remains limited given the cost of equipment that can immerse a patient in a virtual environment and the difficulty that developing an eye test within such a virtual environment can represent.

[0005] Furthermore, there is a need to be able to control the exercises that the patient must perform and to visualize what the patient sees within the virtual environment, or even to visualize the data collected by the various sensors of the virtual reality device. It is therefore necessary, in addition to the device, to provide manipulative software on a computer terminal that allows triggering and controlling eye exercises in the device, at the request of the practitioner, and to offer a rendering of these exercises on this computer terminal.

[0006] Given the resources it requires and its complexity, this type of software cannot be developed on demand for each practitioner, nor installed directly on the practitioner's computer terminal, but must instead be offered as a service accessible by the practitioner's computer terminal from a remote server. However, this IT architecture poses several problems.

[0007] On the one hand, the solution must be able to be used by different people and for different purposes. Indeed, beyond orthoptists and ophthalmologists, the solution can be used by pharmacists or opticians, as part of screening operations, or even by patients themselves, for home use for rehabilitation. The architecture must therefore allow for the management of a fleet of virtual reality devices and the link between each device and the service on the remote server, while adapting the service's functionalities to the profile of the device's user. The modularity and the number of functions of the service in relation to the number of devices in the fleet can thus complicate the IT architecture.

[0008] There are still other use cases involving the use of virtual reality in the field of human vision that can further complicate the IT architecture in the case where their management had to be centralized. These include assistance in choosing a customer's vision correction by an optician, assistance in improving a subject's visual comfort or even the analysis and training of an athlete's visual performance.

[0009] Furthermore, in order to avoid overloading the memory of the computer terminal, it is preferable for the data streams corresponding to the exercises to be stored in the virtual reality device itself. However, the functionality of rendering, on the computer terminal, the exercise that is broadcast in the virtual reality device, implies that the broadcast data stream is transmitted to the computer terminal to be displayed there. In the case of a fleet of virtual reality devices, the computer architecture thus requires that each data stream broadcast on a device passes through the server to be redirected to the appropriate computer terminal. The quantity of data that the server must then manage is thus much too large for the computer architecture to be feasible.

[0010] There is thus a need for a computer system for analyzing, correcting and / or training human vision, of simple design, offering, for a fleet of virtual reality devices, a centralized service for controlling the broadcasting of eye exercises or tests in a virtual reality device and for viewing these exercises or tests and their results on a computer terminal associated with this device, the resource consumption of which is reasonable and which allows any professional, in particular of human vision in the field of orthoptics, ophthalmology or optometry, to use a virtual reality device through this computer system.

[0011] The present invention is placed in this context and aims to meet this need.

[0012] For these purposes, the invention relates to a computer system for analyzing, correcting and / or training human vision, characterized in that it comprises:

[0013] a virtual reality device intended to be mounted on the head of a subject and comprising a control unit, a first communication unit and a memory in which is stored at least a first stream intended to generate a virtual reality environment in which the subject can be immersed and comprising an exercise and / or an eye test in the virtual reality environment requiring eye movement of the subject;

[0014] a computer terminal comprising a display device and a second communication unit, the computer terminal being capable of executing a computer program for controlling and visualizing the execution of tests and / or eye exercises by the subject by means of the virtual reality device;

[0015] a computer server comprising a third communication unit equipped with a message agent, each of the first and second communication units being capable of sending messages to the message agent, the message agent being arranged to notify a message received from one of the first and second communication units to the other of the first and second communication units.

[0016] The system according to the invention is characterized in that:

[0017] the third communication unit is arranged to, upon receipt by the agent of signaling messages transmitted by the first and second communication units, establish a direct two-way communication channel between the first communication unit and the second communication unit;

[0018] in response to a notification of a message triggering said exercise and / or eye test transmitted by the second communication unit to the message agent, the control unit is arranged to trigger the broadcasting of the first stream on the virtual reality device and the first communication unit is arranged to transmit to the second communication unit, via said channel, the first broadcast stream for its display on a computer program viewing interface on the display device of the computer terminal.

[0019] It is thus understood that the invention proposes to retain the advantages of a solution for controlling a virtual reality device based on access, via a computer terminal, to a computer program, such as a service which can for example be located on a remote server. In order to be able to control the virtual reality device from the computer terminal, each of these two elements is signaled to a message agent of a computer server. This message agent acts as a relay by retransmitting messages transmitted to it by a communication unit to other communication units. For example, it may be provided that a user action with the computer program triggers the transmission of a signaling message by the second communication unit to the message agent, which notifies this message to the first communication unit. This message thus constitutes a coupling request. This first communication unit can then, in response, itself transmit a signaling message to the message agent, this message constituting a positive response to the coupling request.

[0020] The computer server can then trigger the opening of a two-way communication channel between the virtual reality device and the computer terminal, in peer-to-peer mode. This channel thus allows the virtual reality device and the computer terminal to exchange data streams in real time, without a server being interposed. Thus, the user of the computer terminal can access all of the functionalities of the computer program and can therefore control the broadcasting of one or more exercises and / or eye tests in the virtual reality device via the computer terminal. For example, it may be provided that an action by the user with the computer program triggers the transmission of a message triggering an exercise and / or eye test by the second communication unit to the message agent, which notifies this message to the first communication unit.The control unit can thus trigger, in response, the broadcasting of the corresponding data stream.

[0021] When the eye exercise and / or test is broadcast in the device, the data stream corresponding to this exercise and / or test is transmitted to the computer terminal via the communication channel, which then allows the computer program to display this data stream on an interface. The user thus has a tool for viewing the exercises and tests and their results without consuming significant resources at the level of a server of the computer system.

[0022] In the present invention, the term "virtual reality device" means, without limitation, a virtual, augmented or hybrid reality device, such as a headset, a pair of glasses or any other device intended to be worn on the subject's head and allowing the subject to be immersed in a virtual reality. Such a headset may, for example, be equipped with a stereoscopic head-mounted display (HMD). Where appropriate, the device may include a single display for both eyes of the subject, or, on the contrary, two separate displays, each intended for one of the subject's eyes.

[0023] In the present invention, the term "computer terminal" means, without limitation, a touchscreen tablet, a smartphone, a laptop, or any electronic device comprising a screen, or a display and a control and / or input interface capable of being manipulated by a user, such as a touch interface or a keyboard and mouse.

[0024] In the present invention, the term "communication unit" means, without limitation, any electronic and / or logical component or any set of electronic and / or logical components allowing data to be exchanged, in analog or digital form, with another communication unit, via a wireless communication network, for example of the Wi-Fi, 4G or LTE, or 5G type.

[0025] In the present invention, the term "message agent" means, without limitation, one or more computer programs, installed on one or more servers and capable of receiving, storing and relaying or routing messages that it receives to one or more receivers. A message agent, also called a "message broker", may for example manage a queue. The message agent may for example operate in publish / subscribe or publish / subscribe mode, in which a communication unit may subscribe to a topic of the message agent, in order to receive messages published on this topic, and / or publish messages on a topic of the message agent, so that these messages are notified to the units subscribed to this topic.

[0026] In the present invention, the term "computer program" means a set of instructions intended to be executed by a computer or a computer system in order to allow a user to control a virtual reality device and to view content broadcast on this device or data from the device. It may be software installed, partially or totally, on the computer terminal and / or hosted, partially or totally, on one or more remote servers, and with which a user can interact via a user interface accessible on the computer terminal. The user interface may in particular be an application designed to be used via a web browser.

[0027] In the present invention, the term "stream" means, without limitation, an immersive video stream intended to be broadcast on a display. The immersive video stream may comprise a 360° video comprising views of the same scene or the same environment from a plurality of directions, or even from a plurality of positions, at the same time. These may be images acquired live or computer-generated images. The immersive video stream may be intended to be broadcast alone, the stream defining in this case both a scene and an exercise and / or an eye test, or be intended to be superimposed on another stream, the stream defining the exercise and / or the eye test and this other stream defining a scene. In the case of a superposition, the immersive video stream may comprise only one or more objects intended to be positioned in a configurable manner in a scene defined by another video stream, the parameters defining in particular the position of the object or objects in the scene over time, the trajectory and speed of movement of the object or objects in the scene over time, the size of the object or objects over time. Said immersive video stream may be stored in the memory of the virtual reality device in different forms. The immersive video stream may for example be stored in the form of one or more image sequences, in particular in MPEG4 format. Alternatively, the immersive video stream may be calculated in real time by a 3D engine of the control unit, from one or more 3D objects stored in the memory, in particular in polygonal form or in vector form.Alternatively, the immersive video stream may be composed of elements calculated in real time by a 3D engine of the control unit from 3D objects stored in the memory, and pre-calculated elements stored in the memory, such as scenes or 360° backgrounds on which the elements calculated in real time are superimposed.

[0028] In the present invention, the term "virtual reality environment" means, without limitation, an immersive environment of the virtual, augmented or hybrid type. A virtual reality device generates, from an immersive video stream, images which simulate the physical presence of an individual in a virtual reality environment and / or which superimpose virtual objects in a real environment. It is thus possible to explore a virtual reality environment, to a certain extent, for example by directing one's gaze in different directions or by moving in the environment, for example by means of a controller.

[0029] In the present invention, the term "eye exercise and / or test" means, without limitation, the display of an animation of one or more objects in the virtual reality environment, the animation making it possible to evaluate an ocular mechanism, to detect an oculomotor disorder, or to evaluate and / or cause a movement of one or each eye of the subject. The animation may, for example, include the appearance or disappearance of an object, a change in the visual appearance of the object, such as its shape, dimensions, color or transparency, a movement of an object or a modification of the position in space of the object. An exercise or test may be predetermined, all of the parameters of the animation being fixed in advance and stored in the memory of the virtual reality device or be configurable, at least one of the parameters of the animation being able to be fixed or controlled by the professional via the computer terminal.

[0030] Advantageously, the message agent is a "message queue transmission" or MQTT agent. MQTT stands for "Message Queuing Telemetry Transport." This type of agent offers several advantages, including low bandwidth requirements and relatively low latency, which make it particularly relevant in the case of managing a fleet of reality devices. virtual.

[0031] Advantageously, the third communication unit is arranged to, upon receipt by the agent of signaling messages sent by the first and second communication units, establish a direct bidirectional communication channel of the "Web Real Time Communication" or WebRTC type between the first communication unit and the second communication unit. This technology allows data to be sent and received from a web browser, peer-to-peer, which makes it advantageous in the case where the computer program includes a web application.

[0032] In one embodiment of the invention, the message agent is of the publish / subscribe type. When the virtual reality device is put into service, the first communication unit is arranged to subscribe to a predetermined topic of the message agent associated with an identifier of the virtual reality device. The computer program may comprise an interface for selecting the virtual reality device and the second communication unit is arranged to publish, in response to the selection of the device through said selection interface, a first reporting message on the topic of the message agent associated with the identifier of the selected device. Where appropriate, the first communication unit may be arranged to publish, upon receipt of a notification of said first reporting message on the topic to which it is subscribed, a second reporting message on this topic.Preferably, the second communication unit is also arranged to subscribe to said subject of the message agent associated with the identifier of the virtual reality device.

[0033] According to an exemplary embodiment of the invention, the computer program may be arranged to display, on the selection interface, a list of virtual reality devices associated with a user account that has logged into the computer program. When a virtual reality device is first put into service, an identifier of a user account may, for example, be provided to the virtual reality device, so that the user account and the device are associated with each other at the level of a database of the computer system. The second communication unit may thus be arranged to publish, in response to the selection of the device through said selection interface when the computer program is executed with a given user account, a reporting message incorporating an identifier of said user account on the subject of the message agent associated with the identifier of the virtual reality device.The server can thus control the association, at the database level, of the user account connected to the computer program executed at the computer terminal and the virtual reality device, and if necessary establish a two-way communication channel. direct between the first communication unit of this computer terminal and the second communication unit of this virtual reality device.

[0034] Advantageously, the computer terminal may be arranged to control the virtual reality device through predetermined subjects of the message agent associated with an identifier of said virtual reality device, the second communication unit thus being arranged to publish control messages, such as messages for triggering exercises and / or eye tests, on a subject of the message agent associated with the identifier of said virtual reality device to which the first communication unit is subscribed.

[0035] In one embodiment of the invention, the computer system comprises a plurality of virtual reality devices each intended to be mounted on the head of a subject. The first communication unit of each virtual reality device can be arranged, when said virtual reality device is put into service, to subscribe to a predetermined topic of the message agent associated with an identifier of said virtual reality device and the second communication unit is arranged to publish, in response to the selection of a virtual reality device from a list of devices through said selection interface, a first reporting message on the topic of the message agent associated with the identifier of the selected device.Where applicable, each first communication unit is arranged to publish, upon receipt of a notification of said first reporting message on the subject to which it is subscribed, a second reporting message on this subject.

[0036] It is thus understood that the computer system can manage, via a single message agent, a plurality of virtual reality devices, each device having for example a distinct identifier known to the computer system at the server level and being associated with a user account in a database of the server. Where appropriate, for each pair of virtual reality device and computer terminal, the server can establish a direct bidirectional communication channel between the communication units of this pair.

[0037] In one embodiment of the invention, the virtual reality device comprises at least one subject's head movement sensor and / or an eye tracker. When broadcasting the first stream on the virtual reality device, the first communication unit can be arranged to transmit to the second communication unit, via said channel, the first broadcast stream and the data acquired by the movement sensor and / or the eye tracker for their display on the computer program's visualization interface on the computer terminal's display device. Beyond offering the professional a rendering of the exercise and / or test currently being broadcast on the virtual reality device, this characteristic makes it possible to display data allowing the professional to control the exercise and / or test and to visualize the results of the exercise and / or test. The data displayed may be raw data acquired by the sensor and / or eye tracker or data calculated from this raw data, at the level of the virtual reality device or the computer terminal.

[0038] Advantageously, the computer program is arranged to display, on the visualization interface of the computer program displayed on the display device of the computer terminal, a graphical representation of the subject, and to modify the orientation and / or the gaze of said graphical representation as and when the second communication unit receives the data acquired by the motion sensor and / or the eye tracker. The data can thus be displayed both on a graph making it possible to control, for example, their temporal evolution during the exercise and / or the test and on a digital avatar of the subject in real time.

[0039] In one embodiment of the invention, the control unit of the virtual reality device is capable of adopting a plurality of predetermined states. The first communication unit can be arranged to transmit to the message agent, at each change of state of the control unit, a change of state message and the computer program is arranged, in response to each notification of a change of state message transmitted by the first communication unit to the message agent, to modify the visualization interface of the computer program displayed on the display device of the computer terminal as a function of said change of state.The computer program can thus automatically modify the visualization interface according to the state of the virtual reality device, in order to propose an interface adapted according to whether the device is being calibrated by the subject, broadcasting an exercise and / or a test, in an inactive or switched off state, or in a waiting state. The use of the messaging agent for the transmission of the state of the device makes it possible to obtain a substantially instantaneous response from the computer program to a change in the state of the device.

[0040] Advantageously, a plurality of streams is stored in said memory, each stream being intended to generate a virtual reality environment in which the subject can be immersed and comprising a predetermined exercise and / or eye test in the virtual reality environment requiring eye movement of the subject. The computer program may comprise an interface for selecting one or more eye exercises and / or tests from a predetermined list of eye exercises and / or tests; and the second communication unit is arranged to transmit to the message agent, in response to the selection of at least one eye exercise and / or test on said interface, a message for triggering said selected eye exercise and / or test. Preferably, in response to the notification by the agent of messaging to the first communication unit of said trigger message, the control unit is arranged to trigger the broadcasting, on the virtual reality device, of the stream stored in said memory and comprising said selected exercise and / or eye test.

[0041] Preferably, said predetermined list of exercises and / or eye tests may be recorded in a database of the server hosting the computer program by being associated with an identifier of the user account which has connected to the computer program.

[0042] Each stream of said plurality of streams may include one or more or a combination of the following eye exercises and / or tests:

[0043] An exercise and / or a visual acuity test, in which a sequence of letters or patterns is displayed in the virtual reality environment, the dimensions of the letters or patterns changing during the display of the sequence, for example by decreasing. In particular, it may be provided that during the display of the sequence, only one of the displays of the virtual reality device, corresponding to one of the subject's eyes, is active, the other being blocked or inactive;

[0044] A stereoscopic vision exercise and / or test, in which an object is successively displayed in different locations of the virtual reality environment;

[0045] A convergence exercise and / or test, in which an object is displayed in the virtual reality environment moving along a depth axis, for example gradually moving closer to the subject and then further away;

[0046] An exercise and / or test of fusion or amplitude in convergence and divergence, in which an object is displayed in the virtual reality environment moving along different trajectories and / or speeds for each display of the virtual reality device, and therefore for each eye of the subject;

[0047] An eye tracking exercise and / or test, in which an object is displayed in the virtual reality environment moving along a curved, particularly circular, path;

[0048] A saccade eye movement exercise and / or test, in which two juxtaposed objects are displayed in the virtual reality environment by flashing alternately;

[0049] A physiological diplopia exercise and / or test, in which two objects are displayed in the virtual reality environment, one behind the other along a depth axis, flashing alternately.

[0050] In the event that several exercises and / or eye tests are selected via the selection interface, the second communication unit may be arranged to automatically transmit to the message agent a trigger message for each exercise and / or eye test selected according to their order of selection, each trigger message being transmitted in response to the receipt of a notification of an end message of the previous exercise and / or test by the message agent. Alternatively, the second communication unit may be arranged to transmit to the message agent a trigger message for each exercise and / or eye test selected following validation submitted by the professional via the selection interface.

[0051] In one embodiment of the invention, the computer system is characterized in that:

[0052] following the commissioning of the virtual reality device, the control unit is arranged to adopt a waiting state for an exercise and / or test; and, in response to a notification of a message of change of state to a waiting state, the computer program is arranged to add to the visualization interface of the computer program said interface for selecting one or more exercises and / or eye tests from a predetermined list of exercises and / or eye tests;

[0053] in response to a notification of a message triggering a selected exercise and / or eye test via the selection interface, the control unit is arranged to adopt a broadcast state of an exercise and / or test and to trigger the broadcast on the virtual reality device of a stream stored in said memory and comprising said selected exercise and / or eye test; and, in response to a notification of a message changing state to a broadcast state of an exercise and / or a test, the computer program is arranged to modify the display interface of the computer program according to a predetermined configuration associated with the selected exercise and / or eye test.

[0054] For example, it may be possible to provide for a waiting stream to be stored in the memory of the virtual reality device, the stream being intended to generate a waiting virtual reality environment in which the subject can be immersed. The control unit may thus be arranged, when it adopts the waiting state, to trigger the broadcasting of said waiting stream on the virtual reality device.

[0055] Preferably, the computer program may be arranged to add an interface for viewing the data acquired by the motion sensor and / or the eye tracker of the virtual reality device and transmitted to the computer terminal via the channel.

[0056] Advantageously, at the end of the broadcast of the last selected exercise and / or eye test, the control unit is arranged to adopt the waiting state. The first communication unit can thus be arranged to transmit, to the messaging agent, a broadcast status message for an exercise and / or a test, said message being able for example to indicate the start or the end of broadcast of the exercise and / or the test, that the exercise and / or the test is being broadcast or that a response by the subject to a question is expected.

[0057] In an exemplary embodiment of the invention, at least one variable stream intended to generate a virtual reality environment in which the subject can be immersed and comprising a configurable exercise and / or eye test is stored in said memory. Where applicable:

[0058] in response to a notification of a message triggering said exercise and / or configurable eye test transmitted by the second communication unit to the message agent, the control unit is arranged to trigger the broadcasting of the variable stream on the virtual reality device;

[0059] in response to a notification of a state change message to a broadcast state of a variable exercise and / or test, the computer program is arranged to add to the visualization interface of the computer program an interface for controlling the variable parameters of the selected configurable eye exercise and / or test;

[0060] following a request to modify a parameter according to an instruction given via this control interface, the second communication unit is arranged to transmit to the message agent a message modifying said parameter containing said instruction; and;

[0061] in response to a notification of a message modifying a parameter containing an instruction, the control unit is arranged to modify the broadcasting of the variable flow on the virtual reality device according to said parameter and said instruction.

[0062] This embodiment thus makes it possible to broadcast a stream without time limit, for example to immerse the subject in a looping scene, and to allow the professional to carry out a personalized exercise or test in which a parameter can be controlled in real time, for example to modify the size of an object, the direction of its trajectory or its speed of movement.

[0063] In one embodiment of the invention, the virtual reality device comprises at least one sensor capable of detecting whether the virtual reality device is mounted on the subject's head. If applicable, in response to the sensor detecting the subject wearing the virtual reality device, the control unit is arranged to adopt a calibration state and to trigger the broadcasting on the virtual reality device of a calibration stream stored in said memory and comprising a procedure for calibrating the virtual reality device. Said calibration procedure may be a procedure for calibrating the motion sensor of the virtual reality device and / or a procedure for calibrating the eye tracker of the virtual reality device.

[0064] If desired, the computer program and / or the control unit of the virtual reality device may be arranged to display, respectively on the computer terminal and in the virtual reality environment, a proposal interface initialization of the calibration procedure, at the end of each broadcast of a stream containing an exercise and / or an eye test.

[0065] The invention also relates to a method for associating a virtual reality device and a computer terminal, implemented by means of a computer system according to the invention.

[0066] The invention also relates to a method for displaying, on a display device of a computer terminal, a stream broadcast on a virtual reality device and / or data acquired by one or more sensors of a virtual reality device, implemented by means of a computer system according to the invention.

[0067] The present invention is now described using examples which are purely illustrative and in no way limitative of the scope of the invention, and from the appended drawings, drawings in which the various figures represent:

[0068] [Fig.l] represents, schematically and partially, a view of a computer system for analyzing, correcting and / or training human vision according to an embodiment of the invention;

[0069] [Fig.2] represents, schematically and partially, a view of the computer system of [Fig.l] during a step of associating a virtual reality device and a computer terminal;

[0070] [Fig.3] represents, schematically and partially, a view of the computer system of [Fig.l] at the start of the broadcasting of a stream on the virtual reality device;

[0071] [Fig.4] represents, schematically and partially, a view of the computer system of [Fig.l] at the end of the broadcasting of a stream on the virtual reality device.

[0072] In the following description, elements which are identical in structure or function and which appear in different figures retain, unless otherwise specified, the same references. Of course, various other modifications may be made to the invention within the scope of the appended claims.

[0073] [Fig.l] shows a computer system for analyzing, correcting and / or training human vision according to an exemplary embodiment of the invention.

[0074] The computer system comprises a plurality of virtual reality devices I,, each intended to be mounted on the head of a subject.

[0075] In the example described, it refers to one of these devices F. This device is a headset equipped with a video headset 11 comprising two separate displays, each intended for one of the subject's eyes, it being understood that other types of virtual, augmented or hybrid reality devices may be used without departing from the scope of the present invention.

[0076] The device F further comprises a control unit 12 provided with a memory in which different video streams Fj are stored. Each video stream Fj is intended to generate a virtual reality environment in which the subject can be immersed when the stream is broadcast on the displays of the head-mounted display 11. Some of these streams Fj also include an exercise and / or an ocular test Ej, namely an animation of one or more objects in the virtual reality environment intended to generate a response or a movement of one or both eyes of the subject, the animation then making it possible to evaluate an ocular mechanism or to detect an oculomotor disorder or to evaluate and / or trigger a movement of one or each eye of the subject.

[0077] In the following description, reference will be made to a flow F! comprising an eye exercise E, of the convergence exercise type.

[0078] The control unit 12 can adopt a plurality of predetermined states indicating the status of the virtual reality device 1 ; , such as an inactive state, a state waiting for an exercise and / or a test, a state broadcasting an exercise and / or a test or a calibration state.

[0079] In order to allow the user of the computer system to detect an oculomotor disorder in the subject or to monitor the exercises and / or tests performed by the subject, the virtual reality device L comprises various sensors 14, namely sensors for the movement of the subject's head, such as accelerometers and gyrometers, and eye trackers. An eye tracker is understood to mean any device making it possible to detect an eye movement and to measure the direction of gaze. It could, for example, be one or more cameras, in particular infrared cameras, associated with a computer, which may or may not be integrated into the head-mounted display 11.

[0080] The virtual reality device 1 ; also comprises a communication unit 13, capable of exchanging data with another communication unit via a wireless communication network, for example of the WiFi, 4G or LTE, or 5G type.

[0081] The computer system also comprises a plurality of computer terminals 2 k , each intended to be manipulated by a user of the computer system, such as an orthoptist, an ophthalmologist, an optician or even a pharmacist, or even the subject himself.

[0082] In the example described, it refers to one of these 2 terminals k This terminal is a portable computer equipped with a screen 21 and a keyboard and a mouse (not shown) allowing the user to interact with the terminal 2 k, it being understood that other types of computer terminals may be used, such as a touch tablet or a smartphone, without departing from the scope of the present invention.

[0083] The computer terminal is capable of executing a computer program W for controlling and visualizing the execution of tests and / or eye exercises by the subject using a virtual reality device 1 ; .

[0084] In the example described, this program W is hosted on a computer server 3 and can be executed by the computer terminal 2 k through a web browser, as a service made available by the server 3.

[0085] This type of computer program is also called a web application or “WebApp” and does not require any installation on the computer terminal 2 k . In order to access the computer program W, the computer terminal 2k and the computer server 3 are each equipped with a communication unit, respectively 22 and 31, capable of exchanging information together through a wireless communication network. The different control interfaces of the virtual reality device 1 ; , and visualization of Fj streams and Ej exercises and / or tests broadcast on this device 1 ; are thus stored on server 3 and transmitted to computer terminal 2 k via communication units 2 and 31, after terminal 2 k has requested it, to be viewed on the screen of the computer terminal 2 k via the web browser.

[0086] In other words, in the example described, each computer terminal 2 kof the computer system can request the computer program W hosted on the server 3. In order to be able to identify and authenticate the users of the computer program W, the server 3 has a database 32, provided on this server or on another dedicated server, in which identifiers C are stored k user accounts through which users can log in to the W computer program.

[0087] The computer system also comprises a computer server 4 hosting a message agent 41 and a communication unit 42 capable of exchanging data with another communication unit via a wireless communication network. It is possible for the servers 3 and 4 to be part of the same group of servers, to be the same server or, on the contrary, to be servers distinct from each other.

[0088] In the example described, the message agent 41 is a publication / subscription agent of the “message transmission in queue mode” type, also called an “MQTT broker”. This publication agent manages subjects, also called “topics”, to which so-called “subscriber” devices can subscribe to be notified when a message is published there, and in which so-called “publisher” devices can publish messages. The message agent 41 can thus notify or relay, via the communication unit 42, a message published by a publisher on a given subject to the subscribers to this subject.

[0089] In the invention, the communication units 13 of each virtual reality device F and 22 of each computer terminal 2 k will be able to act as as publishers as well as subscribers of the message agent 41, which is unique to the computer system.

[0090] Still in the example described, the subjects managed by the message agent 41 are stored in a database of the server 4 by being organized according to two hierarchical levels, namely on the one hand according to different types of predetermined subjects and on the other hand according to an identifier idj of a virtual reality device F-

[0091] More specifically, a subject may relate to signaling messages enabling a virtual reality device F to be coupled or associated with a computer terminal 2 k The subject will thus be placed in a list of “report” type subjects and will be identified by an idj identifier of the virtual reality device l b for example by being stored as follows: " / signaling / id ; »

[0092] A subject may also relate to messages triggering an exercise and / or an eye test, allowing the broadcasting of a stream Fj containing an exercise and / or a test Ej to be triggered, on a virtual reality device F- The subject will thus be placed in a list of subjects of the “trigger” type and will be identified by an identifier id ; of the virtual reality device F, for example by being stored as follows: “ / control / id ; ".

[0093] A subject can also be related to state change messages of the control unit 12 of a virtual reality device F- The subject will thus be stored in a list of subjects of type “state” and will be identified by an identifier id ; of the virtual reality device F, for example by being stored as follows: “ / state / id;”.

[0094] Other types of subjects or another hierarchy of subjects may be provided without departing from the scope of the present invention.

[0095] In connection with [Fig.2], we will now describe an example of a method of associating a virtual reality device F with a computer terminal 2 k of the computer system of [Fig.l].

[0096] Beforehand, when first putting the virtual reality device into service, b an identifier of a user account C k , intended to control this device through a computer terminal, could for example be provided to the virtual reality device 1 i, which transmits it to the server 3 so that the user account C k and the identifier id ; of the device F are associated with each other at the level of the database 32 of the server 3.

[0097] When commissioning the virtual reality device F, the communication unit 13 subscribes to a reporting subject identified by the identifier id ; of the device, namely the subject / signal i rig / id, of the message agent 41.

[0098] When running the computer program W on a web browser, via a computer terminal 2 k , the user authenticates on his user account C k . The computer program W then displays a visualization interface including different elements including a visualization interface W1 of a stream broadcast on the virtual reality device 1, and an interface W2 for selecting a virtual reality device from a list of virtual reality devices associated with the user account C k at database level 32.

[0099] When device 1 ;is selected by the user on the interface W2, the computer program W retrieves the identifier id, of this device 1, and publishes, via the communication unit 22, a first reporting message incorporating an identifier of the user account C k on the subject / signaling / id; of message agent 41.

[0100] The message agent 41 notifies this message to the subscribers of the subject / signaling / id;, namely the communication unit 13, which publishes in response a second signaling message on this subject / signaling / id; of the message agent 4L. The first message thus forms an association request and the second message thus forms an association confirmation.

[0101] In response to these signaling messages, the server 4 can then, through the communication unit 42, establish a direct bidirectional WebRTC communication channel between the communication units 13 and 22 of the device 1 ;and computer terminal 2 k . This direct two-way WebRTC communication channel is of the “real-time communication for the Web” type and allows data exchanges between the web browser running on the computer terminal 2 k and the control unit 12 of the device L-

[0102] This WebRTC channel will thus allow data exchanges in real time, without going through the server 4. Thus, the user of the computer terminal 2 k can control the broadcasting of exercises and / or eye tests in the virtual reality device I, and visualize the results via the computer program W.

[0103] Following the association of the virtual reality device 1 with the computer terminal 2 k , communication units 13 and 22 each subscribe to the topics / control / id; and / statc / id, of the message agent 4L

[0104] The control unit 12 then adopts a waiting state for an exercise and / or a test. It may be provided that a waiting flow, stored in the memory of the virtual reality device 1 ; , be broadcast on this virtual reality device 1 ; as long as the control unit 12 is in this waiting state. The subject can thus familiarize himself with the virtual reality environment in which he is immersed.

[0105] The communication unit 13 then publishes a state change message, indicating this waiting state, on the subject / state / id; of the message agent 4L. In response to the notification of this state change message by the message agent 41 received by the communication unit 22, the computer program W adds to its display interface a selection interface W3 of one or more eye exercises and / or tests from a predetermined list of eye exercises and / or tests Ej.

[0106] It will be possible to provide for one or more lists of exercises and / or eye tests Ej to be stored in the database and associated with the user account C k . The lists can also be associated with a particular profile of the subject, a history of the subject, a sub-account of the user counter C k The computer program W can thus select one of the lists of exercises and / or eye tests Ej according to the different information provided by the user during their navigation and display this selected list at the level of the W3 interface.

[0107] In a non-limiting manner, the list of eye exercises and / or tests Ej may include a visual acuity exercise or test, a stereoscopic vision exercise or test, a convergence exercise or test, a fusion or amplitude exercise or test in convergence and divergence, an eye pursuit exercise or test, an eye saccade exercise or test and a physiological diplopia exercise or test. For each of these exercises and / or tests Ej, a stream Fj comprising said exercise or test Ej will be stored in the memory of the device h.

[0108] When the user selects via the W3 interface an exercise or a test Ej, such as the convergence exercise E b the computer program W transmits, via the communication unit 22, a message triggering said eye exercise Ei on the subject / control idi of the message agent 41 and the message agent 41 notifies this message to the communication unit 13.

[0109] In response to this notification, the control unit 12 then borrows the broadcast state of an exercise and / or an exercise and triggers the broadcast, on the virtual reality device 1;, of the stream F] stored in its memory and comprising said ocular exercise E r

[0110] [Fig.3] shows the computer system at the start of the broadcast of the flow F, and [Fig.4] shows the computer system at the end of the broadcast of the flow F.

[0111] When the state of the control unit 12 changes, the communication unit 13 publishes a state change message, indicating the broadcast state, on the subject / state / id; of the message agent 4L. In response to the notification of this state change message by the message agent 41 received by the communication unit 22, the computer program W adds to its visualization interface an interface W4 for visualizing a graphical representation of the subject and an interface W5 for visualizing the data acquired by the eye trackers of the virtual reality device 1. ; .

[0112] It will be noted that other configurations of the display interface of the computer program W may be envisaged or that the configuration of the display interface of the computer program W may be adapted according to the exercise and / or the eye test selected. In particular, it may be provided that certain elements are present from the start-up of the computer program W, or from the start of the association of the computer terminal 2 k to device 1 ; , or even others elements are added, such as elements intended to give instructions to the user.

[0113] As shown at the top of [Fig.3] and [Fig.4], the diffused flux F! includes a convergence exercise E 1? in which an object O is displayed in the virtual reality environment moving along a depth axis, gradually approaching the subject.

[0114] Throughout the diffusion of the F stream b the communication unit 13 transmits in real time to the communication unit 22, via the WebRTC channel, the stream Fj broadcast on the displays of the head-mounted display 11 for its display on the visualization interface W1. The user thus has a real-time rendering of the exercise E b as visualized by the subject.

[0115] Concomitantly, the communication unit 13 transmits in real time to the communication unit 22, via the WebRTC channel, the data D acquired by the sensors 14 of the device I, during the broadcasting of the stream Fj.

[0116] Data D acquired by the eye trackers, representing in the example described the instantaneous direction of gaze of each eye of the subject, are displayed as they are received in graphic form at the W5 interface.

[0117] In addition, a graphical representation of the subject's head is displayed in the interface W4, and the computer program W modifies the orientation and gaze of this graphical representation as the data acquired by the sensors 14 is received.

[0118] The communication unit 13 also publishes, on the subject / state / id, of the message agent 41, messages indicating the broadcast status of an exercise and / or a test, said message being able for example to indicate the start or the end of broadcasting of the exercise and / or the test, that the exercise and / or the test is being broadcast or that a response by the subject to a question is expected.

[0119] It should be noted that in the case where several exercises and / or eye tests Ej are selected via the selection interface W3, the communication unit 22 may, upon receipt of a notification of an end of broadcast exercise and / or test message published on the subject / state / id; of the message agent 41 by the communication unit 13, automatically publish on the subject / control / id, of the message agent 41 a message triggering the exercise and / or test Ej selected next according to the order of selection by the user. Alternatively, the broadcast order may be defined according to a sequence of exercises and / or tests stored in the database 32 of the server 3 or defined manually by the user. Manual validation may also be provided at the level of the computer program W in order to move on to the broadcast of the next exercise and / or test.

[0120] At the end of the broadcast of the Ei exercise, or of the last exercise and / or eye test se- selected, the control unit 12 again borrows the waiting state of an exercise and / or test. The waiting flow is broadcast at the device I,. A state change message is published by the communication unit 13 on the subject / state / id ;of the message agent 41 and the visualization interface of the computer program W is again adapted in response to the notification of this message by the message agent 41. The user can thus trigger the broadcasting of new exercises and / or tests on the device 1 ; .

[0121] In an example not shown, it may be provided that a calibration procedure of the virtual reality device 1 ; and its sensors can be initiated by the subject and / or the user each time the control unit 12 enters the exercise and / or test broadcast waiting state.

[0122] The preceding description clearly explains how the invention makes it possible to achieve the objectives it has set itself, namely to provide a computer system for analyzing, correcting and / or training human vision of simple design, proposing, for a fleet of virtual reality devices, a centralized service for controlling the broadcasting of eye exercises and / or tests in a virtual reality device and for viewing these exercises and / or tests and their results on a computer terminal associated with this device and whose resource consumption is reasonable. These objectives are notably achieved using a single message agent capable of associating virtual reality devices and computer terminals to establish a direct two-way communication channel between the communication units of each pair of virtual reality devices and computer terminals.Each virtual reality device can thus be controlled by the computer terminal to which it is associated via the messaging agent while this device and this terminal can exchange data between them directly.

[0123] In any event, the invention cannot be limited to the embodiments specifically described in this document, and extends in particular to all equivalent means and to any technically effective combination of these means. In particular, other configurations of the virtual reality devices, other exercises and / or eye tests, other types of messaging agents or communication channels may be provided.

Claims

Claims

1. Computer system for analyzing, correcting and / or training human vision, characterized in that it comprises: a. a virtual reality device (1 ; ) intended to be mounted on the head of a subject and comprising a control unit (12), a first communication unit (13) and a memory in which is stored at least a first stream (Fj) intended to generate a virtual reality environment in which the subject can be immersed and comprising an exercise and / or an eye test (Ej) in the virtual reality environment requiring an eye movement of the subject; b. a computer terminal (2 k) comprising a display device (21) and a second communication unit (22), the computer terminal being capable of executing a computer program (W) for controlling and visualizing the execution of tests and / or eye exercises by the subject by means of the virtual reality device; c. a computer server (4) comprising a third communication unit (42) equipped with a message agent (41), each of the first and second communication units (13, 22) being capable of transmitting messages to the message agent (41), the message agent being arranged to notify a message received from one of the first and second communication units to the other of the first and second communication units; characterized in that: a.the third communication unit (42) is arranged to, upon receipt by the message agent (41) of signaling messages transmitted by the first and second communication units (13, 22), establish a direct two-way communication channel (WebRTC) between the first communication unit and the second communication unit; b. in response to a notification of a trigger message of said exercise and / or eye test (Ej) transmitted by the second communication unit (22) to the message agent (41),. the control unit (1) is arranged to trigger the broadcasting of the first stream (Fj) on the virtual reality device (F) and the first communication unit (13) is arranged to transmit to the second communication unit (22), via said channel (WebRTC), the first stream broadcast for its display on a visualization interface (Wl) of the computer program (W) on the display device (21) of the computer terminal (2 k).

2. Computer system according to the preceding claim, characterized in that the message agent (42) is an agent of the “message transmission in queue mode” type, or MQTT.

3. Computer system according to one of the preceding claims, characterized in that the third communication unit (42) is arranged to, upon receipt by the agent of messages (41) of signaling messages sent by the first and second communication units (13, 22), establish a direct bidirectional communication channel (WebRTC) of the "real-time communication for the Web" type, or WebRTC, between the first communication unit and the second communication unit.

4. Computer system according to one of the preceding claims, characterized in that the message agent (41) is of the publication / subscription type; in that, when the virtual reality device (1; ), the first communication unit (13) is arranged to subscribe to a predetermined subject ( / signaling / idj) of the message agent (41) associated with an identifier (idj) of the virtual reality device (1 ; ) ; in that the computer program (W) comprises a selection interface (W2) of the virtual reality device; in that the second communication unit (22) is arranged to publish, in response to the selection of the device through said selection interface, a first signaling message on the subject ( / signaling / id;) of the message agent (41) associated with the identifier of the selected device; and in that the first communication unit (13) is arranged to publish, upon receipt of a notification of said first signaling message on the subject to which it is subscribed, a second signaling message on this subject.

5. Computer system according to the preceding claim, characterized in that it comprises a plurality of virtual reality devices (1 ;) each intended to be mounted on the head of a subject; in that the first communication unit (13) of each virtual reality device is arranged, when said virtual reality device is put into service, to subscribe to a predetermined subject t / signal i ng / id of the message agent (41) associated with an identifier (idj) of said virtual reality device; in that the second communication unit (22) is arranged to publish, in response to the selection of a virtual reality device from a list of devices through said selection interface (W2), a first signaling message on the subject of the message agent associated with the identifier of the selected device; and in that each first communication unit (13) is arranged to publish, upon receipt of a notification of said first signaling message on the subject to which it is subscribed, a second signaling message on this subject.

6. Computer system according to one of the preceding claims, characterized in that the virtual reality device (1 ; ) comprises at least one sensor (14) for movement of the subject's head and / or an eye tracker (14) and in that, when broadcasting the first stream (F,) on the virtual reality device, the first communication unit (13) is arranged to transmit to the second communication unit (22), via said channel (WebRTC), the first broadcast stream and the data acquired by the movement sensor and / or the eye tracker (14) for their display on the visualization interface (Wl) of the computer program (W) on the display device (21) of the computer terminal (2 k ).

7. Computer system according to the preceding claim, characterized in that the computer program (W) is arranged to display, on the viewing interface (Wl) of the computer program displayed on the display device (21) of the computer terminal (2 k ), a graphical representation (W4) of the subject, and to modify the orientation and / or the gaze of said graphical representation as and when the second communication unit (22) receives the data acquired by the motion sensor and / or the eye tracker (14).

8. Computer system according to one of the preceding claims, characterized in that the control unit (12) of the virtual reality device (1 ; ) is capable of adopting a plurality of predetermined states, in that the first communication unit (13) is arranged to transmitting to the message agent (41), at each change of state of the control unit, a change of state message and in that the computer program (W) is arranged, in response to each notification of a change of state message transmitted by the first communication unit (13) to the message agent (41), to modify the display interface (Wl) of the computer program (W) displayed on the display device (21) of the computer terminal as a function of said change of state.

9. Computer system according to the preceding claim, characterized in that a plurality of streams (Fj) is stored in said memory, each stream being intended to generate a virtual reality environment in which the subject can be immersed and comprising a predetermined exercise and / or eye test (Ej) in the virtual reality environment requiring an eye movement of the subject; in that the computer program (W) comprises an interface (W3) for selecting one or more exercises and / or eye tests from a predetermined list of exercises and / or eye tests; and in that the second communication unit (22) is arranged to transmit to the message agent (41), in response to the selection of at least one exercise and / or eye test on said interface, a message for triggering said selected exercise and / or eye test.

10. Computer system according to the preceding claim, characterized in that: a. following the commissioning of the virtual reality device (lj), the control unit (12) is arranged to adopt a waiting state for an exercise and / or a test (Ej); and, in response to a notification of a message of change of state to a waiting state, the computer program (W) is arranged to add to the display interface (Wl) of the computer program said selection interface (W3) of one or more exercises and / or eye tests (Ej) from a predetermined list of exercises and / or eye tests; b. in response to a notification of a message of triggering an exercise and / or eye test (Ej) selected via the selection interface (W3), the control unit (12) is arranged to adopt a broadcasting state for an exercise and / or test and to trigger the broadcasting on the virtual reality device (1 ; ) of a stored stream (Fj) in said memory and comprising said selected exercise and / or eye test; and, in response to a notification of a state change message to a broadcast state of an exercise and / or test, the computer program (W) is arranged to modify the viewing interface (Wl) of the computer program according to a predetermined configuration associated with the selected exercise and / or eye test.

11. Computer system according to one of claims 8 to 10, characterized in that the virtual reality device (lj) comprises at least one sensor capable of detecting whether the virtual reality device is mounted on the head of the subject; and in that, in response to the detection by the sensor of the wearing of the virtual reality device by the subject, the control unit (12) is arranged to adopt a calibration state and to trigger the broadcasting on the virtual reality device of a calibration stream stored in said memory and comprising a procedure for calibrating the virtual reality device.

Images