Apparatus, process, and computer program for simulating mammalian physiological behavior using virtual mammals

Abstract

To provide a device, process and program for simulating physiological behavior of a mammal using a virtual mammal.SOLUTION: The device including a mobile head and mobile eyes comprises: a visual stream for successive data representative of eye poses; instructions for mobility action instructions; a memory for storing information on an environment and on stabilization constraints; and a processor assessing and recording a current part of the environment, triggering successive movements of the head and of the eyes according to a function of the mobility action, the successive data and the information on the environment, and controlling the successive movements of the eyes with respect to the successive movements of the head by using the stabilization constraints according to a function of the successive data. The device further comprises an input for training movement sequence data representative of a training movement sequence of the head and the eyes associated with a training environment. The processor learns the stabilization constraints by triggering, in the training environment, the successive movements of the head corresponding to the training movement sequence, and assessing and recording a current part of the training environment from the sequence data.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The invention relates to a device for simulating the physiological behavior of a mammal, more precisely the behavior of the visual system of a mammal placed in an environment, using a virtual mammal. The invention can be implemented in the field of computing and has many applications, for example in the field of ophthalmology for the evaluation of vision correction or the testing of ophthalmic products. [Background technology]

[0002] Much research has been conducted on the visual systems of living organisms, particularly mammals, and humans in general among mammals, from the structure and behavior of the eye and light transduction to the neural structures and behaviors associated with vision. Based on such knowledge and the behavior of the visual system, simulation models have been realized on computers. Tools (e.g., CarlSim), application blocks (e.g., software body avatars from www.makehumancommunity.org), or global application systems by combining blocks have been proposed to simulate the behavior of living organisms in the field of robotics, but these are generally quite limited, especially with regard to the structures to be simulated and the environments in which they are created.

[0003] Moreover, in humans, the visual system is generally evaluated under very constrained conditions with respect to the images provided to the visual system, in particular fixed images, with the head and eyes stationary, which can be referred to as stationary or quasi-stationary conditions.

[0004] However, in reality, the visual system is exposed to a highly variable visual environment, and any effective evaluation of visual performance should take such a variable environment into account. For example, Patent Document 1 describes a method for objectively or subjectively determining or selecting a visual device, particularly a lens, in which the visual device, the visual environment, and the wearer can be virtually simulated. The importance of considering the visual motility, particularly the mobility of the head and eyes due to changing visual environments, can be further emphasized in relation to the fact that such changes and mobility are essential during the development of an organism's visual system to generate a properly functioning visual system.

[0005] Furthermore, eye movements can be attributed to two main sources: reflex movements and other movements as opposed to reflex movements. These other movements can be termed voluntary movements as opposed to reflex movements. In particular, when the head is moving, the visual system responds to the vestibular system's detection of head movement with a reflex response that also depends on the currently visualized environment. This is called the vestibulo-ocular reflex (VOR), which allows stabilization of the image on the retina during head rotation by contralateral eye movements that maintain the image in the center of the visual field. The VOR is mediated by the cerebellum, a brain region whose adaptation is directly driven by sensorimotor errors. The cerebellar adaptation process, driven by an error signal in fixation signaled by retinal slip, minimizes retinal slip, which ultimately reduces vestibular output to generate compensatory motor commands that drive eye movements. The vestibulo-ocular reflex is operable regardless of visual conditions and functions in both bright and dark conditions. The vestibulo-ocular reflex has been the subject of the following conference and related papers: (Non-Patent Document 1)

[0006] Such advanced research into vision systems has in any case proven to be complex to integrate satisfactorily into realistic implementations of robots or digital avatars moving in a given environment: existing simulation systems usually either fall short of a realistically sufficient vision mechanism or prove to produce more or less substantial distortions for real-world mammals.

[0007] The document (Patent Document 2) is also known. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] European Patent Application Publication No. 1862110 [Patent Document 2] International Publication No. 2018 / 005891 [Non-patent literature]

[0009] [Non-Patent Document 1] "Exploring vestibulo-ocular adaptation in a closed-loop neuro-robotic experiment using STDP.A simulation study" Francisco Naveros et al., 2018 IEEE / RSJ International Conf. on Intelligent Robots and Systems (IROS), 10.1109 / IROS.2018.8594019. Summary of the Invention [Means for solving the problem]

[0010] One object of the present invention is to provide an apparatus for simulating the physiological behavior of a mammal in an environment by means of a virtual mammal, said virtual mammal comprising: a head that moves relative to the environment; at least one eye that moves in rotation relative to the head, The device comprises: - at least one first input adapted to receive continuous data representative of a posture of said at least one eye relative to an environment; at least one second input adapted to receive instructions for at least one movement behavior by the virtual mammal in the environment; at least one memory adapted to store information about the environment and stabilization constraints between the at least one eye movement and the head movement; at least one processor, assessing a current portion of the environment from the continuous data, and if the current portion is not identified in the at least one memory, recording information about the current portion in the at least one memory; triggering continuous head and at least one eye movement in function of the at least one locomotion behavior, the continuous data, and stored information about the environment; at least one processor configured to control dynamic adjustment of the continuous movement of the at least one eye to the continuous movement of the head in function of the continuous data by using a stabilization constraint; The device further comprises: - at least one third input adapted to receive, during a training phase, training movement sequence data (= training data) representing a training movement sequence of the head and the at least one eye associated with a training environment, The at least one processor, during the training stage, - triggering a series of head movements in a training environment corresponding to said training movement sequence data; - assessing a current portion of a training environment from said training data and, if said current portion is not identified in said at least one memory, recording information about said current portion in said at least one memory; - determining information about at least a portion of stabilization constraints in function of the training data and the continuous head movement taking into account evaluating the current portion of the training environment and recording the information about the current portion, and recording the information about the at least a portion of stabilization constraints in the at least one memory; configured to learn at least a portion of the stabilization constraints by As a result, the virtual mammal provides a simulated behavior of the mammal to be simulated when performing the at least one locomotion behavior in the environment through continuous dynamically adjusted current movements of the head and the at least one eye, and through evaluating the environment.

[0011] In certain implementations, such devices may behave more or less like real creatures, in this example mammals, thereby enabling accurate simulation of such mammals by virtual mammals, or equivalently mammal avatars. In this regard, such virtual mammals may enable, among other things, personalized simulation, monitoring, and prediction.

[0012] In some disclosed embodiments, this is further achieved by such a device learning to behave in the same way as a real mammal through a deep learning process. The latter may have an advantage over other methods of producing such virtual mammals in that it does not require the creation of a complete and accurate working model of the visual system or precise knowledge of the structures and functions involved in a real visual system. This deep learning process is advantageously carried out by a neural network that is used to simulate at least a portion of the visual system, and once the neural network is trained, it can be used for the simulation.

[0013] In any event, the disclosed device is not limited to visual aspects such as the neuronal aspects and ocular capabilities of an organism, but phototransduction capabilities can also be considered, either on their own or through more general conditions such as age, degeneration, or other physiological conditions.

[0014] In addition to other uses, the virtual mammals simulated by the disclosed apparatus may also be used as dummies for training ophthalmologists, surgeons, or other personnel involved in ophthalmology or visual equipment.

[0015] In addition, the control of dynamic adjustments of eye movements is not limited to the VOR mechanism: it can in particular involve intentional eye fixation and / or stabilization in reflexive visual movements (when following a moving object) instead of or in combination with them.

[0016] During the training phase, learning at least a portion of the stabilization constraints may depend, inter alia, on implicit relationships between received training data representing head movements, on the one hand, and eye movements, on the other hand, these relationships depending on the stabilization constraints. The processor may, inter alia, consider stabilization mechanisms underlying images corresponding to at least one visual stream as seen by a mammal in an environment.

[0017] For example, during a training phase, head movements and at least some of such images are mimicked for a virtual mammal in an environment from received training data. The latter may include, for example, data directly related to such visual streams or data related to eye movements to which such visual streams are guided. In this example, no eye movements need to be triggered when learning the stabilization constraints.

[0018] In another example, during the training phase, at least some of the head and eye movements are mimicked for a virtual mammal in an environment from received training data, and such a visual stream is then derived from that simulation.

[0019] Furthermore, during the training phase of the virtual mammal, environmental awareness can be taken into account in learning at least some of the stabilization constraints, making it possible to reflect an effective coupling between environmental awareness and the mammal-specific eye movement stabilization mechanism.

[0020] Indeed, the disclosed device can potentially achieve substantial realism in incorporating combined environmental recognition and eye movement stabilization mechanisms, including a learning phase, an improvement that may prove particularly important for simulations of visual systems.

[0021] The data representing the eye posture and / or the training movement sequence data representing the eye training movement sequence may in particular include data directed to the eye posture and / or one or more visual streams corresponding to the eye posture, which may in fact represent the eye posture or the eye training movement sequence.

[0022] In an advantageous implementation, the virtual mammal includes a movable body relative to the environment as well as a movable head.

[0023] The following features are also envisaged which can optionally be used together with the inventive device either alone or according to any technical combination of the following features: the processor comprises software and computing hardware; - a training movement sequence of the head and said at least one eye is acquired on a simulated mammal, the training movement sequences of the head and said at least one eye are obtained by computation and are synthetic; The learning process during the training phase is carried out in a data-driven regime of the device, the device can be used in a data-driven regime in which a training phase can be carried out, The device can be used in a model-driven regime, - if the current part is not identified in the at least one memory, instead of recording information about the current part of the environment in the at least one memory, the at least one processor is configured to replace previously recorded information in the at least one memory with information about the current part; the recorded information about the current part of the environment in the at least one memory is erasable; The device is also for simulating pathological behavior, - The virtual mammals of the device are completely virtual, the virtual mammal of the device has substantial parts; - the physical part includes a head, the substantial part comprises at least one eye; - the substance part contains the body, -The solid parts include the limbs, -The head of the device is virtual, -The head of the device is a material element, at least one eye of the device is virtual; at least one eye of the device is a material element; - at least one eye of the device is capable of seeing; at least one eye of the device is equipped with a camera; the command for the at least one movement behavior includes a command to move the head; the instructions for the at least one movement behavior include an instruction to move the eyes; the instructions for at least one movement behavior include instructions to move both the head and the eyes; the instructions for the at least one movement behavior include an instruction to search for an object; the instructions for the at least one movement behavior include an instruction to proceed to a predetermined location within the environment; "pose" of said at least one eye means the position together with the orientation relative to a coordinate system of said at least one eye; the "pose" of said at least one eye can be expressed as position and orientation values; the "pose" of said at least one eye can be represented as a vector, the movement sequence of the head and the at least one eye results in successive postures over time; - data representing pose (position + orientation) relate to the eye pose itself or to the visual stream corresponding to the eye pose relative to the environment, -Stabilization constraints are related to the vestibulo-ocular reflex, -Stabilization constraints relate to intentional eye fixation, -Stabilization constraints are related to the optokinetic reflex, the configuration for learning of the at least one processor is configured to perform a deep learning process with data representing training movement sequences of a head and the at least one eye associated with a training environment to learn at least a portion of the stabilization constraints; -Mammals are humans, -Mammals are animals, - the simulated mammal is a defined, real individual, the simulated mammal is a theoretical mammal; -Theoretical mammals are mammals that represent a group, -Theoretical mammals are statistically defined, -Theoretical mammals are research mammals, During the training phase, the training environment in which the virtual mammal is located is preferably the same as the environment in which the training movement sequences of the head and said at least one eye of the simulated mammal are obtained; During the training phase, the training environment in which the virtual mammal is located is not the same as the environment in which the training movement sequences of the head and said at least one eye of the simulated mammal are obtained; Once the virtual mammal is trained in the training phase, the acquired environmental information / knowledge of the training environment is retained for further use of the virtual mammal; Once the virtual mammal is trained in the training phase, the acquired environmental information / knowledge of the training environment is erased, so that the virtual mammal so configured begins to discover the environment from scratch; - The environment of the virtual mammal is realistic and - The environment of the virtual mammal is virtual, - to evaluate a current portion of said environment from said training data and, if said current portion is not identified in said at least one memory, to record information about said current portion in said at least one memory, which may also be considered as training of a virtual mammal device, but which training may be performed both in a data-driven regime and in a model-driven regime; at least one action of assessing the current portion of an environment, recording information, and triggering the continuous movement is associated with at least one task parameter specific to the mammal to be simulated, and the at least one processor is further configured to learn the at least one task parameter in a training phase in function of the training data and the continuous movement of a head, taking into account assessing the current portion of a training environment and recording information about the current portion; The operating parameters are selected from any type of parameters that can be measured or derived from external characteristics, The working parameters relate, for example, to the behavior of the eyelids in relation to the light level and the range of vision, The working parameters relate, for example, to the level of ambient light in the environment, the at least one processor further comprising: - acquiring at least one visual stream corresponding to said continuous data representative of said at least one eye posture, assessing said current part of the environment from said at least one visual stream, triggering said continuous movement and controlling said dynamic adjustment in particular in function of said at least one visual stream, - configured during the training phase to obtain at least one training visual stream corresponding to the training movement sequence data representing a training movement sequence, and to evaluate the current portion of the training environment and identify the information in particular in function of the at least one training visual stream; the data representing the training movement sequence includes position and gaze direction data of parts of the virtual mammal (e.g., the body and / or legs of the virtual mammal, etc.) over time; the at least one processor is further configured to condition the acquired at least one visual stream and the at least one training visual stream in relation to at least one conditioning parameter specific to the mammal being simulated, and the at least one processor is further configured to learn, during a training phase, the at least one conditioning parameter in function of the at least one training visual stream and the continuous head movement, taking into account the evaluation of the current portion of a training environment and the recording of the information related to the current portion; the at least one processor further comprising: - deriving successive images from said at least one visual stream; - configured to control said dynamic adjustment by stabilizing at least a portion of said derived successive images; the at least one processor is further configured to control the dynamic adjustment by suppressing sequential movements of the at least one eye relative to the at least some of the sequential images; the apparatus comprises at least one fifth input adapted to receive information regarding a modified value of a stabilization constraint, said at least one processor being configured to replace information regarding a previous value of the stabilization constraint in a memory with information regarding said modified value; the previous and revised values ​​of the stabilization constraints correspond to at least one of a pair of different mammal ages and a pair of different health states; the at least one processor comprises at least one first neural network configured to control the dynamic adjustment by a feedback loop; the at least one first neural network configured to control the dynamic adjustment by a feedback loop is constructed based on a cerebellar-vestibular-ocular reflex control structure of a mammal to be simulated; - said at least one first neural network is a simulated neural network; at least one of assessing the current portion of the environment, recording the information, triggering the continuous movement, and controlling the dynamic adjustment is associated with a task parameter, the device comprising at least one input adapted to receive reference motion data corresponding to the continuous movement of a head and the at least one eye, the reference motion data being associated with reference visual data corresponding to the received at least one visual stream, the at least one processor being configured to learn at least one of the task parameters from the reference motion data and at least a portion of the reference visual data; - if a virtual mammal device learns at least one of said task parameters from at least a portion of said baseline motion data and said baseline visual data, it is in a data-driven regime; - when the virtual mammal device has learned at least one of said task parameters from at least a portion of said baseline motion data and said baseline visual data, it is in a data-driven regime and is placed in a model-driven regime; at least one visual stream of the reference visual data is synthetic; at least one visual stream of reference visual data is acquired on the simulated mammal; at least one visual stream of reference visual data is derived from a reference gaze direction obtained on the simulated mammal; at least one visual stream of reference visual data is obtained from a camera positioned near an eye of the simulated mammal; the at least one processor comprises at least one second neural network configured to evaluate the current portion of an environment and record information about the current portion; - at least one second neural network configured to evaluate the current portion of the environment and record information about the current portion is constructed based on a hippocampal structure of a simulated mammal, the hippocampal structure including position coding; at least one second neural network is a simulated neural network; the at least one processor is configured to perform at least one of the following operations: evaluating, by simulated spiking neurons, the current portion of an environment; recording the information about the current portion; and controlling dynamic adjustment of continuous movements of the at least one eye; the device comprises at least one fourth input adapted to receive at least one simulated visual characteristic, the at least one processor being further configured for dynamically conditioning the at least one visual stream in function of the at least one simulated visual characteristic; the at least one processor is further configured to dynamically condition the at least one visual stream in function of the at least one simulated visual property by virtue of at least one dedicated neural network simulating the mammalian retina and early visual cortex; The dedicated neural network simulating the retina and early visual cortex is a simulated neural network; the at least one processor is further configured for dynamically conditioning the at least one visual stream in function of the at least one simulated visual characteristic by virtue of at least one customizable spatiotemporal filter that filters the at least one visual stream; the at least one processor is further configured to dynamically condition the at least one visual stream in function of the at least one simulated visual characteristic by means of a set of cascaded customizable spatiotemporal filters; - Customizable spatiotemporal filters simulate visual instruments, - if the eye of the virtual mammal is a material element having a camera, a visual device, which is also material, is placed on the physical eye of the virtual mammal; - Customizable spatiotemporal filters simulate deformations of the eye, especially the eye's optical interface, - Customizable spatiotemporal filters simulate eye diseases, - the at least one simulated visual characteristic is at least partially related to at least one physiological behavior associated with the at least one eye; - physiological behavior is related to the corresponding functions of the eye itself or the visual cortex, - said at least one simulated visual characteristic relates at least in part to the visual equipment of said at least one eye;

[0024] A further object of the invention is to provide a process for simulating the physiological behavior of a mammal in an environment by means of a virtual mammal in a device according to the invention, said mammal comprising: a head that is movable relative to the environment; at least one eye that is rotatably movable relative to the head, The process comprises: receiving continuous data representative of a posture of said at least one eye relative to an environment; receiving an instruction for at least one movement behavior by a virtual mammal in an environment; - evaluating, by at least one processor, a current portion of the environment from said continuous data using information about the environment stored in at least one memory, and if said current portion is not identified in said at least one memory, recording information about said current portion in said at least one memory; - triggering, by said at least one processor, continuous movements of the head and said at least one eye in function of said at least one movement behavior, said continuous data, and stored information about the environment; - controlling, by the at least one processor, a dynamic adjustment of the continuous movement of the at least one eye to the continuous movement of the head in function of the continuous data by using a stabilization constraint between the movement of the at least one eye and the movement of the head; The process further comprises: - during a training phase, receiving training data representing training movement sequences of the head and the at least one eye associated with a training environment; learning, by the at least one processor, during the training phase; - triggering, in a training environment, a series of head movements corresponding to said training movement sequence; - assessing a current portion of a training environment from said training data and, if said current portion is not identified in said at least one memory, recording information about said current portion in said at least one memory; - determining information about at least a portion of the stabilization constraints in function of the training data and the continuous movement of the head, taking into account evaluating the current portion of the training environment and recording the information about the current portion, and recording the information about the at least a portion of the stabilization constraints in the at least one memory; configured to learn at least a portion of the stabilization constraints by As a result, the virtual mammal provides a simulated behavior of the mammal to be simulated when performing the at least one locomotion behavior in the environment through continuous dynamically adjusted current movements of the head and the at least one eye, and through evaluating the environment.

[0025] The process can be carried out according to all the methods described, particularly for the above mentioned devices, and possibly combined together.

[0026] The invention also relates to a computer program comprising instructions which, when executed by a processor, cause said processor to carry out the process of the invention.

[0027] The computer program is advantageously configured to perform any of the disclosed process modes of execution separately or in combination.

[0028] The invention also relates to a non-transitory computer readable medium comprising computer program instructions which, when executed by at least one processor, cause said at least one processor in an apparatus to perform the processes of the invention.

[0029] Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0030] [Figure 1] The major structural, functional, and environmental characteristics and relationships of a virtual mammal simulated by a device according to the disclosure are symbolically depicted. [Figure 2] 1 shows a schematic diagram of an apparatus for simulating a virtual mammal as operating in a model-driven regime. [Figure 3] 1 illustrates a preferred operational implementation of data and functions of a device having functional modules. [Figure 4] 1 shows an example of different elements that may be considered to model / simulate the mammalian retina and early visual cortex. [Figure 5A] 1 shows an example of a mathematical model of a VOR controller that can be inferred. [Figure 5B] 1 shows the simplified structure of the mammalian VOR neural system being modeled / simulated. [Figure 6] We present a simplified mammalian hippocampal neuronal structure, which is the basis for modeling / simulating the processing of visual and self-motion information during the construction of internal spatial representations. [Figure 7] 1 shows a schematic diagram of the material elements of an apparatus for simulating a virtual mammal as it operates in a data-driven regime. [Figure 8] FIG. 1 is a flow diagram of the disclosed process. DETAILED DESCRIPTION OF THE INVENTION

[0031] The disclosed device and process aims to simulate the physiological behavior of a mammal, which may be in a healthy or unhealthy state (and therefore also in a pathological state) of at least the mammal's visual system (it will be seen that additional functions, parts, and organs may easily be added to the virtual mammal), using a virtual mammal, which is simulated in the device. The simulated physiological behavior is therefore derived from a healthy mammal or a mammal with some pathological condition. The mammal referred to is preferably a human, especially when the device is applied to the selection of visual equipment, in particular lenses or goggles.

[0032] The visual system in the virtual mammal is simulated using a set of functions, e.g., functions related to the action of the eye on light entering the eye (e.g., related to the curvature and length of the eye), the transformation of light / photons in the retina, and different operations performed on the transformed light in parallel with different neuron structure functions of the visual system of the real mammal that is the reference for the virtual mammal. These neuron structure functions are, for example, early visual processing structures, vestibulo-ocular reflex (VOR) neuron structures. These functions, with respect to those involved in the neuron structures in real mammals, are most preferably implemented / executed as neural networks in the virtual mammal to simulate real neuron structures. However, if the function of the real neuron structure is extremely simple and / or does not vary much between mammals, it can be more easily simulated by mathematical and / or logical functions that can be predetermined and fixed.

[0033] In the following description, the visual stream is considered as a vehicle / support of data representing pose (position + orientation). The visual stream corresponds to the eye pose relative to the environment and may be received within the device or determined by the device from known eye pose and from environmental data (the latter may be stored in the memory / memories of the virtual mammal, and is not the same as information about the environment that is related to information available to the mammal).

[0034] Moreover, the following description is mainly based on the control of dynamic adjustment of eye movements in the VOR mechanism, which may further include intentional eye fixation and / or stabilization related to the optokinetic reflex (when tracking a moving object), either separately or in combination. Therefore, these modes, VOR, intentional eye fixation, and optokinetic reflex, can be implemented in the device, either alone or in combination (as will be described later, using corresponding functional modules). Preferably, at least the vestibulo-ocular reflex (VOR) is implemented to simulate a virtual mammal.

[0035] 1 , a virtual mammal 1 simulated by an apparatus 11 according to the disclosure is located in an environment 2 and has a visual relationship with the environment 2, in the present example primarily in the form of a visual stream 10 of data corresponding to what is seen or assumed to be seen in the environment 2 by the virtual mammal's eyes 4, which corresponds to the information contained in the received light / photons. The eyes and part of the visual system process the visual information contained in the visual stream 10 to obtain environment information 8.

[0036] The virtual mammal has a head 3 on which eyes 4 are located, and both the head 3 and eyes 4 are movable as represented by rotating arrows 5, 6, with the eyes movable 6 relative to the head and the head movable 5 relative to the environment 2. These movements can be "voluntary" movements as opposed to reflex movements, with "voluntary" movements being under the control of movement behavior commands 9. "Voluntary" movements under the control of movement behavior commands 9 can be for "voluntary" movement of the head 3 or for "voluntary" movement of the eyes 4, depending on the type of movement behavior command. The eyes 4 are also subject to reflex movements in response to stabilization constraints 7.

[0037] A virtual mammal may be entirely virtual, or may comprise a movable physical head with some material elements and a movable material eye, e.g., a camera, all within a real and / or displayed physical environment on a screen. A virtual mammal may also be virtually placed within an entirely virtual environment, especially if it is itself entirely virtual. It may also comprise a body (either entirely virtual or with material elements) that may be movable within the environment. A virtual mammal may be implemented as a robot.

[0038] The material elements of the virtual mammal are actually extensions of the simulated mammal. In terms of output / response, the simulated mammal also simulates the material elements, and the simulation is used to control effectors that move or activate the material elements. The movement of the material elements is therefore the result of the simulation, including responses to voluntary or reflex movements, particularly through movement behavior commands. Input, particularly the visual stream, can be synthetic (i.e., computer-generated) or obtained from a camera, particularly a camera at the eye position of a virtual mammal with a physical head and eyes. Of course, the simulated mammal can be augmented with additional input and / or output means, such as eyes, arms, legs, hearing, etc., and therefore additional corresponding functional modules (see below) are added.

[0039] A virtual mammal can therefore be a highly detailed 3D model of a real environment and can be placed in a virtual environment or a synthetic / virtual 3D environment that can be used to replicate recorded behavior of mammals, particularly humans, and can be used to predict what a person for whom the virtual mammal is defined would do if placed in a similar real environment.

[0040] FIG. 2 shows a schematic diagram of the main hardware elements implemented in an apparatus for simulating the visual behavior of a mammal in an environment. This FIG. 2 relates to an apparatus in which a virtual mammal is readily constructed from a mammal model, which is then referred to as being implemented in a model-driven regime. The virtual mammal 1 comprises an apparatus for simulating a mammal 11. The processor 12 is the heart of the apparatus and can be implemented in several ways. It can be a computer, typically a microcomputer, an array of computers, or a network, having at least one general-purpose processor or a combination of a general-purpose processor and special-purpose processing means such as neuronal computing hardware or a reprogrammable gate array. The processor 12 is under software control and communicates with a memory 13 and its surroundings via input / output links. The memory stores, among other things, environmental information 8 and stabilization constraints 7. Again, the memory can be of any type: a general-purpose circuit, such as a RAM or EEPROM, or a specialized circuit capable of storing information either physically or by the structure of the circuit, such as the neural circuit being simulated.

[0041] The software on which the processor 12 runs is configured to simulate a mammal and is configured to simulate and control (via output links) head movements 5 and eye movements 6 according to received (via input links) commands to locomotion behavior 9 ("voluntary" movements) and visual stream 10, which includes processing / generation by the processor 12 of "reflex" eye movements to adapt the eye movements according to stabilization constraints 7 and environmental information 8.

[0042] A thick arrow 611 is represented in FIG. 2 as a dashed line between the processor 12 and the visual stream 10, symbolizing the fact that the processor 12 can modify, in particular by conditioning, the received visual stream 10 based on defined conditions, in particular simulated visual characteristics. These conditions may, for example, relate to the influence of lenses or goggles or some parts of the eye on the light assumed to be received by the eye and transmitted therethrough to the virtual mammal's retina. For example, these conditions may be the presence of a virtual lens on the virtual mammal's eye (or a real lens on the virtual mammal's real eye), the presence of degenerative tissue in the eye, the effects of age on the eye, or the action of an ophthalmologist removing a part of the eye. For this purpose, and to be able to select some of these conditions, a dedicated condition input link (not shown) may be added. These conditions may also be part of the software controlling the processor 12.

[0043] In FIG. 2 , a regular arrow 56 is represented in dashed lines between the resulting head movement 5 and eye movement 6, symbolizing the "reflecting" portion of the eye movement that is attributable to the head movement. Another regular arrow 610 is represented in dashed lines between the resulting eye movement 6 and visual stream 10, symbolizing the fact that when the eyes move relative to the environment, the information contained in visual stream 10 changes because the virtual mammal's eyes are no longer looking in the same direction. If the virtual mammal includes movable eyes that are cameras, the change in information in visual stream 10 is automatic. However, if the virtual mammal's eyes are also simulated, this could be processor 12 modifying the information in the visual stream, which could be done via output 611 discussed above.

[0044] The model-driven regime of the virtual mammal is primarily driven by instructions for locomotion to obtain goal-directed behavior. In the model-driven regime, the virtual mammal can be used to implement new behaviors. This model-driven regime can be used for prediction and monitoring.

[0045] Here, to build a virtual mammal in a model-driven regime, it may not be possible to specify a model from scratch. Moreover, it is often more advantageous to build a virtual mammal from knowledge gained about real mammals (which may be, for example, specific individuals or individuals corresponding to statistical individuals or groups) to obtain a virtual mammal that behaves similarly to real mammals, where the behavior of the virtual mammal is built from the behavior of the real mammals through a learning process in a training phase of what is called a data-driven regime. The data-driven regime corresponds to the machine learning configuration of the disclosed device. Once the virtual mammal has been trained and learned behaviors (i.e., configured to behave like a real mammal through learning), it can be placed in a model-driven regime.

[0046] Thus, with the data-driven regime, as shown in FIG. 7 , the software of the device for simulating a mammal is also configured to learn (represented by arrow 22) stabilization constraints 7 from data 23 representing head and eye training movement sequences, possibly associated with training environment information identified during the training phase of the data-driven regime. To that end, processor 12 receives head and eye training movement sequence data 23 and training visual stream 10′, while the virtual mammal is placed in the identified training environment, which can be stored in memory in the form of training environment information 8′. Database 21 can exchange data with processor 12. The database can store simulation parameters, particularly recorded experimental data, that can be automatically loaded into the device. It will be appreciated that the database can also store many other types of data, including predefined models, to directly configure the device in the model-driven regime without having to execute the data-driven regime and its training phase. They can also store simulation results or resulting models obtained from a training phase for later reuse without having to rerun the training phase.

[0047] In this data-driven regime, depicted in FIG. 7, the training movement sequence data 23 corresponds to applied eye and head movements, which are not and do not correspond to instructions for locomotion behavior as in the model-driven regime of FIG. 2.

[0048] The training movement sequence data 23 representing eye training movement sequences associated with the training environment do not necessarily comprise eye movements, but instead may include relevant input visual streams as observed by a real mammal, an "average" mammal, or a fictitious mammal in the training environment, or other types of input visual streams for training.

[0049] For example, the input visual stream may be determined upstream from actual eye movements tracked by a suitable camera fixed to or external to the mammal and from the surrounding environment, and provided to the device as such, e.g., derived from gaze directions obtained from the tracked eye movements, through ray tracing in a virtual environment corresponding to the surrounding environment, or from moving one or two cameras in a manner that reflects the tracked eye movements in the surrounding environment.

[0050] In another embodiment, the input visual stream is obtained by averaging several visual streams associated with different real people, for example, belonging to the same group (e.g., same gender, similar age, similar eye disease, same height category, etc.) In yet another embodiment, the input visual stream is artificially created, for example, based on a real-world visual stream that is subsequently transformed.

[0051] In any case, having eye movement information as training movement sequence data 23 is in fact a particularly interesting embodiment, with the associated training visual stream 10' being identified by the device from the received data of the training environment, for example by ray tracing associated with the eye posture corresponding to the eye movement information and the virtual environment corresponding to the training environment, as is well known in the art.

[0052] Additionally, the training movement sequence data 23 includes head and eye movement data corresponding to training movement sequences associated with a training environment. Because training movement sequences typically incorporate not only reflex eye movements but also environmental awareness, they typically take voluntary movements into account. In this regard, the environmental awareness process and the stabilization process may be closely intertwined and interdependent.

[0053] Mammalian eye movement stabilization characteristics are derived from a learning process. In this respect, head movement and visual stream, or head and eye movement, together with data related to the training environment, may be sufficient. Information about stabilization constraints can actually be derived from existing instabilities and stabilizing effects in the acquired input visual stream, taking into account head movement and ongoing environmental perception.

[0054] Note that in FIG. 7 (data-driven regime), there is no arrow between eye movements 6 and head movements 5, in contrast to FIG. 2 (model-driven regime), where there is an arrow. Indeed, in the training phase, eye movements and head movements are typically captured and applied simply as received, e.g., in real-world situations, as observed above. They are simply captured together as entries, and the absence of an arrow in FIG. 7 means that the relationship between eye movements and head movements is not an issue when retrieving and applying these data in the training phase, so the relationship between them is typically not considered in this regime.

[0055] In addition to the training phase of the learning process in a data-driven regime, the virtual mammal can be configured to reproduce experimentally recorded behavior to verify that the learned configuration or pre-established model is functioning correctly. Moreover, in this data-driven regime, new data can be obtained that goes beyond the available experimental data. For example, if data on the user's head position in the environment, eye position relative to the head, the user's gaze direction, and field of view can be measured experimentally, the virtual mammal simulation can additionally provide information on the content of the field of view, the visual cues the user is fixating on, and neural activity of, for example, retinal ganglion cells.

[0056] When the stabilization constraints 7 are learned 22, the device can leave the training phase of the data-driven regime and return to its normal operating state in the model-driven regime, or it can remain in the data-driven regime rather than the training phase, e.g., to reproduce experimental behavior data. During the training phase, the learning process is typically performed through a deep learning process and / or any learning process that constitutes a neural network if such an implementation is used. The use of training movement sequences of the head and eyes for learning corresponds to the data-driven regime, in which the virtual mammal is asked to reproduce experimentally recorded behavior.

[0057] In other words, the training phase corresponds to a data-driven regime of the device, in which the device learns to behave in the same way as the real mammal it is simulating. The resulting virtual mammal after the training phase is an image of the mammal that serves as a reference. The reference mammal for training the virtual mammal may be a real mammal, a statistical mammal (e.g., the average of real mammals), or a constructed target mammal. The information addressed in the context of the term "learning" implicitly refers to database information that is learned, and can be correlated using appropriate analytical tools with information resulting from the device's operation (movement sequences, possibly associated locomotion behavior, possibly a visual stream guided in the training environment, etc.) with reference data on the mammal's behavior, and the database may be available, in particular, via a library that is either local (user library) or remote (server library).

[0058] During operation of the device, either in the learning phase or in normal operation, it is possible to access data processed within the device, and also to access the behavior of the virtual mammal from the position of its real parts, such as its movable head and / or eyes. For example, it is possible to measure and / or record the position of the virtual mammal's real head in the environment, the position of its real eyes relative to the head, the gaze direction, and the field of view. Alternatively, it may be possible to obtain and / or calculate the same information from processed data obtained from the device, especially when there are no real parts of the virtual mammal and it is necessary to rely on processed data. Moreover, if the functionality of the visual system is implemented / performed as a simulated neural network, it may also be possible to obtain detailed information about the structure of the simulated / virtual neurons and how they behave resulting from the learning process. It may also be possible to obtain information about how the visual stream is processed and "represented" within the visual system, as well as information about the content of the visual field, the visual cues to which the user is fixated, and the neural activity of, for example, retinal ganglion cells.

[0059] The software controlling the processor 12 has many functions and it is preferable to implement a modular system for different functions, particularly for ease of programming, maintenance, customization and upgrades, including the addition of new functions such as controlling the movement of additional physical bodies of the virtual mammal. Additionally, a modular system makes it easier to create a system that is more functionally similar to the real biological system of the mammal being simulated in the virtual mammal.

[0060] Regarding the upgrade, as far as the visual system is concerned, and since the eyes are located on the head of the virtual mammal, the complete body movements, including walking, can again be followed by the head movements in the space of the environment, simplifying the process. In fact, it should be noted that, as far as the visual system is concerned, considering the head movements in space may be sufficient for the simulation of the whole body, since the head supports the eyes and the vestibular system.

[0061] In an upgraded implementation, the virtual mammal may have a complete body and may walk and / or perform other body movements that may be controlled by specific functional modules, as described below, resulting in head movements that are taken into account by functional modules directly related to the visual system.

[0062] The system can therefore be easily expanded by adding modules that implement additional functions. Each functional module simulates a biological organ or brain region or part thereof (e.g., the eye, retina and visual cortex, hippocampus; cerebellum; musculoskeletal system, and control areas in the brain that control balance and posture of the virtual mammal). Each function can be implemented by several functional modules at different levels of biological detail. To speed up the simulation, some functional modules can be implemented by fast algorithms, while others can be implemented by detailed biological simulations involving neural networks or cell models.

[0063] 3 shows an exemplary modular system with communication means interconnecting different components such as functional modules, processing means, auxiliary tools, and data structures. This modular system with its components is under the control of a control program that is part of the device's software. More precisely, in this example, the server library component 14, the user library and analysis tools 15, the 3D aging virtual mammal and its environment component 16, and three functional modules 17, 18, and 19 are connected to a message-passing middleware 20. The three functional modules represented are: a virtual retina and early visual cortex module 17, a cerebellar VOR control module 18, and a hippocampus module 19.

[0064] The 3D Aging Virtual Mammal and its Environment component 16 mentions the word "aging" because it has been developed to take into account the age of the virtual mammal, the abilities and functions simulated by the device evolving as the virtual mammal ages.

[0065] The Server Library component 14 manages the components (registration, subscription, interface, broadcast). The 3D component 16 is a specific client module that performs the visualization of the virtual mammal, its environment and selected behaviors using a 3D simulation environment. The User Library and Analysis Tools component 15 (including analysis and graphic user interface) performs the control and analysis of the simulation platform and its main functions include: -Configuration of virtual mammal parameters (either manually, or loaded from a file, or learned). -Configuration of virtual mammal environments that can be loaded from a file, for example. - Configuration of virtual mammal behavior in terms of functional modules. The user can, for example, load experimental data that determines the user behavior. In a data-driven regime, where the learning process is performed, the behavior is specified by experimental data, for example head position and gaze direction as a function of time. In a model-driven regime, the behavior is specified by an experimental protocol that describes the behavioral task (for example, in a navigation experiment, the starting position and goal must be given, as well as the number of learning trials). Access to any specific data analysis tools developed to process and visualize the results of the simulation. These analysis tools may relate to specific components of the virtual mammal, the evaluation of its behavioral performance, etc.

[0066] When the functional modules and the virtual environment are known, it is possible to efficiently parameterize the virtual mammal of the device. The parameters of the model can be separated into global parameters (such as age, sex, specific medical conditions, etc.) that affect all or most of the functional modules and components, and local parameters (such as visual acuity or contrast sensitivity, or any other experimentally measured parameter of the mammal) that affect the behavior of a specific functional module or component.

[0067] The message passing middleware 20 is the backbone of the system, allowing each component connected to the system to send messages to other components and invoke functions implemented on other components. From a technical point of view, the implementation of the device can be done via a distributed system that can run on one computer, on several computers, or on a computer cluster, depending on the number of modules and their complexity.

[0068] As mentioned above, other components may be added, which may be functional modules containing computational models of specific brain regions (e.g., primary visual cortex) or biological organs (e.g., eyes). Each functional module must register with the system, be able to subscribe to messages from other modules, and expose its own functionality.

[0069] The message-passing middleware 20 may be implemented using the open-source ROS framework (http: / / www.ros.org). In particular, a development of this framework using the C / C++ language and called cROS (https: / / github.com / rrcarrillo / cros) may be used. This development includes a ROS master and a server module that manages components using a standardized software interface and implements message passing / broadcasting and remote function calls. In a biologically plausible implementation of the functional modules, these modules exchange messages represented by spike trains, i.e., short sequences of events that simulate the electrical action potentials used by neurons in the nervous system to exchange information. For this purpose, special types of messages are compiled into the ROS library so that any module can asynchronously send spike trains to any other module, modeling information transmission in the brain. The system can also be extended by adding other custom message types.

[0070] The 3D Aging Virtual Mammal and Its Environment component 16 may be developed using the "Unity 3D Engine" (https: / / unity3d.com / unity). The "Unity 3D Engine" provides many features that can be used to model realistic 3D environments, including lighting effects, physics, and animations. Full control of the engine can be performed using the programming language C#. This functionality is used to remotely control the 3D model from the User Library and Analysis Tools 15 component. The subcomponent of the User Library that implements the control of the 3D Aging Virtual Mammal and Its Environment 16 component is referred to herein as the Control Program.

[0071] Five general C# scripts are used to control the 3D aging virtual mammal and 16 components of its environment. -RosInterface: This script implements the ROS interface so that the 3D aging virtual mammal and its environmental components can exchange messages with other modules, including a control program, from which the 3D aging virtual mammal and its environmental components can be remotely controlled. -SimulationManager. This script is called by the control program and is responsible for managing the state of the simulation. It is used to create the virtual mammal and its environment, as well as to start, pause, and stop the simulation of behaviors. -SubjectMovement. This script executes the behavior of the virtual mammal as commanded by the control program. This script also receives raw sensory data from the virtual mammal sensors (e.g., receives the flow of visual information in the visual stream experienced by the virtual mammal). This sensory information is returned to the control script or to any other modules that subscribe to receive it via the subscription service of the server library component. -SubjectEye. This script implements the eyes of a virtual mammal, which can be simulated by a video camera attached to the virtual mammal's head. -PALens. This script performs distortion of the visual input with any visual device attached to the virtual mammal's eyes: single vision, lenses / goggles, filters, photochromics, etc. This is what the script is conditioning the visual stream.

[0072] Thus, the 3D aging virtual mammal and its environment component 16 is used to visualize the virtual mammal, its environment, and its behavior. In this embodiment, this component also serves to simulate the visual distortion effects of visual equipment such as lenses / goggles.

[0073] For the visualization of the virtual mammal's environment, any detailed 3D model of the environment created by dedicated software (such as AutoCAD®) can be loaded as the environment model. In another implementation, a 3D model of the actual laboratory can be developed directly in the Unity 3D Engine.

[0074] The visualization of the virtual mammal head and body, and more generally its physical appearance and its movements, if humanoid, can be implemented using MakeHuman software (http: / / www.makehumancommunity.org). The virtual mammal may be a customizable 3D model of a generic human if the simulated mammal is human. In the current implementation, customization of the visual appearance includes age, gender, and outlook. However, any detailed 3D human model (e.g., created by dedicated software) can be loaded.

[0075] The User Libraries and Analysis Tools 15 component allows users to: - Loading virtual mammal parameters from a parameter file or experimental database. - Creating, starting and stopping simulations during normal operating conditions or training phases using control programs written in, for example, Matlab or Python. - Obtaining simulation data generated by any module of the platform, specifically by subscribing to messages from the corresponding module, and analyzing this data to provide performance evaluation, monitoring, and prediction.

[0076] We now describe some of the functional modules that can be used, more particularly those related to the visual system and eye adaptation to head movements. The functional modules considered are those depicted in Figure 3 and implemented in relation to different functions of a real mammalian brain, with respect to its visual system and eye adaptation to head movements.

[0077] The Virtual Retina and Early Visual Cortex Module 17 is a functional module that simulates the activity of retinal ganglion cells in mammalian, particularly human, retinas. This module is based on the elements depicted in Figure 4. This module can also be applied to simulate orientation-sensitive neurons in the V1-V2 regions of the brain. As input, the module receives a visual stream from the eyes of a virtual mammal or from a camera in the virtual mammal's physical eye. As output, the module generates spike trains (sequences of simulated action potentials) emitted by simulated retinal ganglion cells. Retinal processing is simulated in a detailed manner as a cascade of customizable spatiotemporal filters applied to the retinal input visual stream. The parameters of this model, such as the characteristics of the spatiotemporal filters, are either pre-specified or trained and can therefore be adjusted to retinal recording data (usually performed using an extracted retina on a dish with measurement electrodes so that retinal activity can be recorded). In particular, age-related changes in early visual processing can be taken into account in this model by parameterizing the virtual retina model.

[0078] The cerebellar VOR control module 18 is a reflex eye movement that stabilizes the image on the retina during head rotations by contralateral eye movements with the goal of maintaining the image in the center of the visual field. The VOR is mediated by the brain region where adaptation is directly driven by sensorimotor errors, namely the cerebellum. The cerebellar VOR control module 18 may implement a detailed model of VOR control by neural networks in the cerebellum, such as based on the neuronal structure depicted in FIG. 5B. In a more simplified implementation, the cerebellar VOR control module 18 may be implemented based on the controller model depicted in FIG. 5A as a mathematical function.

[0079] The hippocampal module 19 is based on a neuronal model of the hippocampus and is involved in constructing a neural representation of the virtual mammal's environment. This neuronal model is derived from the representation in Figure 6. Vision-based spatial navigation in mammals is controlled by a neural network in a brain region called the hippocampal formation. Several types of neurons in this brain region, particularly place cells and grid cells, interact in a complex manner to construct a neural representation of the surrounding environment based on incoming sensory input. As shown in Figure 6, self-motion and allocentric visual input enter the grid cell array EC, and the processed information is routed through other neuronal arrays, namely, the DG for feedback (sub-alpha), the CA3 with visual place cells and movement-based place cells, and the CA1 with associative place cells. The hippocampal module 19 uses a neural network to simulate the operation of the hippocampal formation during spatial assessment of the environment.

[0080] The hippocampus module 19 receives as input the following pieces of information: - Visual information as output by the virtual retina and early visual cortex module 17 that receives the visual stream. -Other sensory information such as proprioception, vestibular etc.

[0081] The hippocampal module 19 generates as output neuronal activity that encodes the location of the virtual mammal within the environment. These outputs can then be used to drive the goal-directed behavior of the virtual mammal in a model-driven regime.

[0082] The virtual mammal of the device is capable of assessing the current portion of the environment from the visual stream it receives and is also capable of recording information about said current portion in its memory, which recording can be considered a type of learning / training. Thus, the device can be considered to have a dual training scale: one level is directed to configuring the virtual mammal for dynamic eye movement coordination in a data-driven regime with its training phase, and another level is directed to learning the environment in a data-driven as well as a model-driven phase, or else the data-driven phase involves a combined dual training scale.

[0083] Regarding the two possible regimes of the device, the data-driven regime can be implemented after the model-driven regime has already been applied, specifically to reconstruct the virtual mammal (e.g., considering another individual, changing the health state, or aging the same individual, etc.), and in any case it precedes the model-driven regime that is based on it and is preliminary to this extent.

[0084] In a data-driven regime, the movement of a virtual mammal within an environment is performed based on commands issued by a control program.

[0085] In one implementation, the control commands include a set of recorded spatial coordinates of body parts (e.g., provided by motion capture equipment) as a function of time that the virtual mammal must reproduce. To facilitate visualization, the control movements are interpolated to provide the basis for the final virtual mammal animation, which is created by blending a standard set of five base animations.

[0086] With respect to the movement of different parts of the virtual mammal, the control program also provides gaze direction data that is used to control the eyes of the virtual mammal. In one implementation, the spatial coordinates of the gaze target within the 3D environment are provided as a function of time. An inverse kinematics approach is used to match the movement of the virtual mammal's head with the movement of the virtual mammal's eyes. The inverse kinematics equations identify coupled eye-head positions that fit the data and the constraints of the 3D model of the virtual mammal. Additional parameters, such as eye wavefront, pupil diameter, and eyelid movement (obtained by an eye tracker), can be added to improve the accuracy of the simulation. The gaze direction data and additional parameters can also be used in the training phase to learn the stabilization constraints.

[0087] In a model-driven system, a control program provides an experimental protocol for the behavioral task to be performed. In this case, the movement of the virtual mammal is controlled by an additional functional module that implements the brain regions responsible for motor control. In practice, motor control can be implemented in many ways. For example, a specific module for motor control, i.e., a motor control module, can be added to the modular system. In another implementation corresponding to the modular system of FIG. 3, VOR adaptation allows the cerebellar VOR control module 18 to implement eye movement adaptation by directly controlling eye position for the 3D aging virtual mammal component 16.

[0088] In another implementation, an additional functional module, namely a model of an eye muscle module controlled by spiking the output of the cerebellar VOR control module 18, can be added to the modular system, which implements eye control in a highly detailed and physiologically plausible manner. In the same way, but on a larger scale, the hippocampal module 19 in the implementation of FIG. 3 can also directly control the movement of the virtual mammal itself, if the virtual mammal is constructed to move, by simply setting the direction and speed of the virtual mammal's movement. Alternatively, a specific locomotion module can be added to the modular system that implements motor control of the virtual mammal in a physiologically plausible manner.

[0089] The same interpolation algorithm is used to implement natural behavior of virtual mammals.

[0090] An example of a simulation in the model-driven regime is shown below. 1. First, a simulated mammalian subject is recorded during some experimental task to generate recorded data. For example, adaptation to a new visual device is recorded by measuring eye movement dynamics over several trials in a defined environment. 2. The virtual mammal is then simulated in a data-driven regime to learn the model parameters of at least one functional module involved in ocular adaptation, in this example the cerebellar VOR control module with its stabilization constraints, which is based on a neural network. For this purpose, the virtual mammal currently in the training phase is simulated using the previously obtained recording data and the same corresponding environment. When the functional model is trained, especially when the stabilization constraints are set, it represents a model of how this particular subject will adapt to the new visual equipment. 3. After the functional module has been trained in step 2, it can be used to predict how the subject will adapt to a different visual device that has not been tested before, or possibly the same one as a means to check the quality of the model. In this model-driven simulation, the virtual mammal parameters, particularly the stabilization constraints in this example, are therefore fixed, and the new visual device is installed on the virtual mammal. The simulation is then started, and the performance of the virtual mammal is recorded. This performance constitutes a prediction of what the subject will do if this new visual device is fitted to the subject in this particular task.

[0091] More generally, the same sequence of data acquisition (to obtain recorded data from the mammal to be simulated and the associated environment), model training (the learning process during the training phase of the virtual mammal's behavior), and model-based prediction can be used with any functional module. For example, the cerebellar VOR control module 18 can be trained on recorded data to predict how the subject will adapt to new experimental conditions. As another example, the hippocampal module 19 can be trained through a learning process on an actual navigation task in one environment. In a model-driven regime, the virtual mammal can be used to predict what will happen if the same navigation task is performed in a different environment. Any module capable of representing adaptation and learning can be trained and then used for prediction.

[0092] In particular, with regard to the visual system simulated in the virtual mammal, the mammal's eyes are simulated as cylindrical cameras that are either virtual (the virtual mammal does not have a physical part that simulates an eye) or real (the virtual mammal has a physical part that simulates an eye and is equipped with a camera).

[0093] Additionally or alternatively, any detailed model of a mammalian, particularly a human, eye can be used, for example, implemented by a modified SubjectEye C# script to perform the desired calculations directly by the 3D aging virtual mammal module. If the model is highly complex and requires dedicated processing, a new functional module can be implemented, possibly running on a separate computer or computer cluster, to process the visual stream and expose it to other functional modules using message passing middleware.

[0094] Similarly, any visual device can be modeled for the virtual mammal's eye, either by directly implementing it in a C# script in the 3D aging virtual mammal module or by using a dedicated function module. It is possible to simulate visual distortions and / or add several aberrations (on the wavefront) to the eye's lens and calculate the effect on the retinal image. The effect of astigmatism and higher-order aberrations on the virtual mammal's subjective best focus can then be evaluated.

[0095] The implementation of a virtual mammal using a modular system can be carried out in the following manner. 1 - Definition of a virtual mammal based on the simulated mammal. The definition includes parameters related to the mammal's profile. It takes into account demographic information such as age and / or sex and / or other information. As examples of other information, in the context of ophthalmic use, parameters such as prescription, fitting parameters, etc. may be taken into account at least, but also sensory, kinematic, movement and cognitive state information. The content of the definition depends on the purpose and context of the simulation. The definition can be entered in many ways: manually, using a file containing the relevant parameters, or learned during a training phase. In this definition, goals and problems to be solved by the device for simulating the virtual mammal are also set. Goals and problems can be related to any field, industry, medicine, games, and include at least the following: Learning during the training phase to set parameters such as stabilization constraints throughout the learning process. - Playing back / rendering movements with virtual mammals as a result of learning. -Evaluation or monitoring of movement or other factors (e.g. representation of visual information in parts of a modeled (neural network) visual structure), or more generally, data (in particular stabilization constraints, visual quality, etc.) and / or structures (in particular the structure and properties of neural networks) involved in processing in the device. -Predicting movements or other elements. 2- Functional module selection: The selection of the regime (model-driven regime or data-driven regime) and modules is made according to the user's objectives. The output / response of the 3D virtual mammal simulation can be delivered to one specific user or a cluster of users, the output / response can be delivered from different functional modules and / or can be further processed to obtain an assessment of behavior such as navigation, autonomy, and sensory, motor, and / or brain levels. 4- Identifying / selecting / optimizing solutions to problems faced by the virtual mammal. This is a neurocomputation-based method / process for objectively or subjectively identifying or selecting any problem related to simulation, reproduction, or visual device lenses, or for providing personalized recommendations to improve the mammal's health. The effects of visual devices or retraining / restoration can be simulated and tested in a virtual environment that models the mammal's performance. Thanks to software modeling, the behavior of an aging mammal can be simulated or reproduced to predict the effects of potential visual processing (e.g., eye disease), the human brain, behavioral deficiencies, or poor vision (e.g., uncorrected or improper refraction). In the standalone version, the virtual mammal includes an automatic evaluation mode to compare the output / response of the virtual mammal. This standalone version provides a baseline for establishing recommendations regarding the best solution.

[0096] The applications of the disclosed device are numerous.

[0097] In a first application, the device can be used to select new lenses / goggles by evaluating their effect on the locomotion of a simulated mammal. In this first application, the device simulates the distortions caused by lenses / goggles on the visual stream reaching the retina. The movement of a real mammal is recorded using a motion and eye-tracking system while exploring a real environment. The data recorded by the system consists of the spatial coordinates of each sensor and eye-tracking data. These data are loaded into the device during the training phase via a control program to control the movement and gaze direction of the virtual mammal for learning in a data-driven regime. After the training phase, a trained virtual mammal is created wearing the lenses / goggles throughout the simulation, and its distortions due to visual input are modeled using PALens scripts in the 3D Aging Virtual Mammal component 16. The User Library and Analysis Tools component 15 allows visual inputs with and without lenses / goggles to be compared with each other during exploration of a simulated real-world environment. In particular, visual distortions can result in changes in balance and movement patterns. Therefore, by analyzing the changes in visual input caused by visual devices and linking these changes to postural control, balance, sway, and walking speed, it is possible to adjust the characteristics of the visual devices to minimize their impact on performance.

[0098] In this first application of the disclosed device, a database of existing eye drops can be used to select an eye drop that matches the wearer's daily activities, such as mobility and navigation activities. The definition of a human virtual mammal includes a wearer profile (prescription, age, fitting parameters, etc.). For example, the parameters in the virtual mammal definition may be: a 65-year-old individual who needs a prescription change and whose current lenses have a short progressive length; and a commercial environment. The input parameters are an eye drop database containing available visual devices and data records of physical movements recorded during a simplified navigation task in a commercial environment. For functional module and component selection, as the virtual mammal wears different visual devices, their distortions on the visual stream input are modeled using PALens scripts in the 3D Aging Virtual Mammal and Its Environment component. According to the User Libraries and Analysis Tools component 15, the minimum configuration of the involved modules and components is as follows: - For the data-driven regime, a 3D aging virtual mammal and its environment component 16 with its PALens script for the calculation of visual environment distortions, and a cerebellar VOR control module 18. - for a model-driven regime, a 3D aging virtual mammal and its environment and functional modules, including a cerebellar VOR control module 18 and a cerebellar VOR control module 18 for recalculating eye movements based on head movements recorded with the current lens. In fact, the wearer, having adapted to his / her lens distortion, presents an optimally simulated vestibulo-ocular reflex (VOR) to stabilize the image on the retina.

[0099] Regarding the output / response of the virtual mammal, in a data-driven regime, the output / response parameters are distortion amounts calculated from recorded movement patterns, eye movements, and a simulated commercial environment. WO 2017157760A1 can be referenced as a method for selecting visual devices that minimize lens distortion on the retinal visual stream.

[0100] In a model-driven regime, this results in a complementary evaluation of the output / response of the virtual mammal using inverse engineering methods. A wearer with a prescription with an additional change of +2.50 to +3.00 dp faces new visual motion due to the distortion and must adapt their vestibulo-ocular reflex (VOR) response. The sensorimotor error caused by the new visual distortion is calculated using PALens scripts in the functional modules Cerebellar VOR Control Module18 and Virtual Retina and Early Visual Cortex Module17, as well as the 3D Aging Virtual Mammal and its Environment component16.

[0101] To determine / select / optimize a visual device or intervention solution, the device user selects a new lens from available existing designs with an additional +3.00 dp, which produces the smallest sensorimotor error in the VOR response, compared to the wearer's current visual device with a +2.50 dp. Those skilled in the art know that visual device distortion on the temporal side of the lens is stronger with shorter designs, or present "soft" or "hard" lens designs. However, current approaches define lens hardness from optical calculations of the local rate of change of the lens's undesirable astigmatism and associated optical deviation (local prismatic effect). This change can be more or less rapid and complex. However, lens distortion is caused by multiple factors, including fitting parameters, optical design, and the wearer's oculo-head pattern during movement. In this new approach, a model-driven approach allows the device user to have a wearer-oriented approach in evaluating the VOR mechanism generated by each wearer's locomotor and eye movement patterns. The personal selection is made from a database of available lenses by comparing the distortion of the current visual device with all visual devices that have been tested on the wearer.

[0102] In a second application of the disclosed device, new visual instruments can be designed using the vestibulo-ocular reflex (VOR) time adaptation aging effect as a performance criterion during the conception stage, which is the manual or automatic modification of lens parameters such as power and subdivision of unwanted astigmatism, definition of the anterior and posterior surfaces, etc. In this application, we simulate the age effect on the vestibulo-ocular reflex (VOR) adaptation experiment.

[0103] First, a virtual mammal is constructed using standard parameters of a young human mammal. For construction, the virtual mammal device is trained using a learning process during a training phase, and data obtained from the young mammal performs at least a sinusoidal head rotation in the horizontal plane using a specified environment. During experimentation with the virtual mammal in the device, a sinusoidal head rotation in the horizontal plane is induced using the same environment. During this experimentation with the virtual mammal, the cerebellar VOR control module 18 automatically adjusts control of the virtual mammal's "eye muscles" to counteract the sinusoidal head movement with an opposing movement of the virtual mammal's eyes. Vestibulo-ocular reflex (VOR) adaptation based on image slip on the retina is evaluated by analyzing the dynamics of the virtual mammal's eye movements. The amplitude of eye movements due to insufficient stabilization should decrease over time. The time scale of adaptation is a measurement variable.

[0104] The same operation is then performed on an elderly virtual mammal in which age-induced changes in VOR adaptation capacity are established. These age-induced changes result in a slowdown in VOR adaptation dynamics, which can be analyzed using the User Library and Analysis Tools 15 component. Regarding VOR adaptation, if necessary, please refer to "Adaptation to Telescopic Spectacles: Vestibulo-ocular Reflex Plasticity," Demer et al., 1989 IOVS, and "Effect of adaptation to telescopic spectacles on the initial human horizontal vestibulo-ocular reflex," https: / / doi.org / 10.1152 / jn.2000.83.1.38.

[0105] In practice, the choice of regime model is based on available data and wearer parameters. The wearer profile (prescription, age, fitting parameters, etc.) can be a young ametropic patient and a 40-50 year old presbyope, or a 40-50 year old presbyope, or a 60-70 year old presbyope, or a 70-80 year old presbyope. The environment is defined. It can be performed using behavioral data (data-driven regimes) or tasks (model-driven regimes).

[0106] In a model-driven regime, a new visual device is designed based on standard parameters of a juvenile mammal used to construct a virtual mammal through learning in a training phase, while estimating the impact of this new design on a criterion defined by vestibulo-ocular reflex (VOR) time adaptation with age.

[0107] Again, the input data are standard parameters of a young mammal and are used to construct the virtual mammal, for which purpose a sinusoidal rotation of the virtual mammal's head in the horizontal plane is induced.

[0108] A selection of functional modules and components is made. The virtual mammal is fitted with lenses / goggles and their distortion on the visual stream is modelled using the PALens script in the 3D Aging Virtual Mammal and its Environment 16 component. According to the User Libraries and Analysis Tools 15 component, the minimum defined configuration of the involved functional modules and components is as follows: -For data-driven regimes: PALens script, 3D aging virtual mammal and its environment16 components with virtual retina and early visual cortex modules17. -For data-driven regimes: 3D aging virtual mammal and its environment16 components, virtual retina and early visual cortex module17, cerebellar VOR control module18.

[0109] For virtual mammalian outputs / responses, the time scales of residual retinal slip and adaptive output parameters are calculated for all age groups as a result of age-induced changes. The rates of vestibulo-ocular reflex (VOR) adaptation and residual retinal slip are identified for each age group.

[0110] With regard to determining, selecting, optimizing visual equipment or selecting intervention solutions, multifactorial optimization consists of identifying design parameters of visual equipment such as astigmatism, magnification, length progression, etc. that reduce the time-cost adaptation of the vestibulo-ocular reflex (VOR) and residual retinal slip, taking into account the amplitude and velocity of eye movements per age group.

[0111] In a third application of the disclosed device, the influence of visual equipment on a mammal's ability to estimate its own movement trajectory from the visual stream 10 can be evaluated. The movement of a real mammal is recorded using a motion and eye-tracking system during a navigation task (either in a real environment or a virtual environment with a head-mounted display), in which the mammal is required to follow a predetermined trajectory from a start location (e.g., two sides of a triangular trajectory) and then return to an unmarked start location. This task is repeated over several trials. Returning to the start location requires estimating one's own position relative to the start, which is known to depend on neuronal processing in the hippocampus. The accuracy of self-location estimation is determined from the error made by the subject when returning to the start location.

[0112] In the virtual environment, several different types of visual input can be simulated to evaluate lens distortion depending on the statistics of peripheral visual cues. The same task in the absence of visual input allows for estimation of self-location based solely on proprioceptive input (as a vision-independent control condition). Recorded data consist of spatial coordinates of each sensor and eye-tracking data. A virtual mammal is created, including a virtual retina 17, VOR control 18, and hippocampal module 19. The data is loaded into the device during the training phase via a control program to control the movement and gaze direction of the virtual mammal for learning in a data-driven regime. As a result of the training, the neural network in the hippocampal module 19 acquires a representation of position with a precision tuned to that of a real mammal.

[0113] Visual distortions are introduced into the model-driven regime via simulated visual equipment using PALens scripts within the 3D Aging Virtual Mammal module. The impact of lens-introduced visual distortions on task performance is analyzed by the User Libraries and Analysis Tools component 15. Apart from testing the effects of lens-introduced distortions, changes in task performance associated with aging or the progression of visual disease can be tested. In particular, by considering several longitudinal measurements of a healthy or diseased mammal's visual system (e.g., its retinal condition, useful visual field, intraocular light diffusion, etc.), a model of age- or disease-related changes in visual processing over time can be constructed. This model can be loaded into a virtual mammal and then used to predict the state of the mammal's visual system in the near future. Such an aging virtual mammal model can simulate the future behavior of the mammal (and its adaptation to new environmental conditions or new visual equipment) and provide potential rehabilitation solutions or suggest new visual equipment to combat age- or disease-related problems.

[0114] Figure 8, a flow diagram of the disclosed virtual mammal operation process, illustrates two possible ways of operating a virtual mammal: a model-driven regime and a data-driven regime, the latter of which operates in a learning phase to learn at least some information for the stabilization constraints.

[0115] In a data-driven regime, the steps are: receiving training data representing training movement sequences of the head and at least one eye associated with a training environment; learning at least some of the stabilization constraints by: - triggering, in a training environment, a series of head movements corresponding to said training movement sequence; -Evaluating a current portion of a training environment from said training data; - testing whether the current part is not identified in at least one memory; otherwise, recording information about the current part in at least one memory; - determining information about at least a part of the stabilization constraints in function of the training data and the continuous movement of the head, taking into account an evaluation of a current portion of the training environment and recording information about said current portion; recording information relating to at least a portion of said stabilization constraints in at least one memory; and learning by

[0116] At the end of these steps, at least some of the stabilization constraints are recorded and can be used in the model-driven regime, which is symbolized by the discontinuous arrow from the last step of the data-driven regime to the model-driven regime.

[0117] In a model-driven regime where stabilization constraints exist in the system (either learned by a previous data-driven regime or obtained by other means), the steps are: receiving continuous data representative of a posture of at least one eye relative to an environment; receiving an instruction for at least one movement behavior by a virtual mammal in an environment; - estimating a current portion of the environment from the continuous data using information about the environment stored in at least one memory; - testing whether the current part is not identified in at least one memory; otherwise, recording information about the current portion in at least one memory; triggering a continuous movement of the head and at least one eye in function of at least one movement behavior, said continuous data, and stored information about the environment; - controlling the dynamic adjustment of the continuous movement of at least one eye to the continuous movement of the head in function of the continuous data by using a stabilization constraint between said at least one eye movement and the head movement.

Claims

1. An apparatus (11) for simulating physiological behavior of the visual system of a real mammal (1) located in a visual environment (2) by means of a virtual mammal (1), the visual system of the real mammal including movable eyes, the visual environment corresponding to what can be seen by the eyes at the location where the real mammal is located, the eyes having retinas from which an image of a current part of the visual environment is obtained, the current part of the visual environment being defined by a gaze direction and a field of view of the eyes, the visual system of the real mammal including image stabilization means for constraining the movement of the eyes to stabilize the image on the retinas, The virtual mammal (1) is a head (3) that moves (5) relative to the visual environment; at least one eye (4) that moves in rotation (6) relative to the head (3) and includes a virtual retina; The device (11) at least one first input adapted to receive continuous data representative of the pose of said at least one eye (4) relative to said visual environment (2); - at least one second input adapted to receive commands for at least one movement behavior (9) of said virtual mammal (1) in said visual environment (2); at least one memory (13) adapted to store information about the visual environment (8) and image stabilization constraints (7) between the movements (6) of the at least one eye (4) and the movements (5) of the head (3), the image stabilization constraints being related to at least one of the following: vestibulo-ocular reflex, intentional eye fixation, optokinetic reflex, the image stabilization constraints (7) constituting a simulation of the image stabilization means in a vestibulo-ocular reflex control module; at least one processor (12) which, under normal operating conditions, a hippocampal module simulating the operation of the hippocampal formation by means of a neural network, spatially assessing a current portion of the visual environment (2) from the continuous data, and recording spatial information (8) relating to the current portion in the at least one memory (13) if the current portion has not already been recorded in the at least one memory (13); triggering a continuous movement (5, 6) of the head (3) and the at least one eye (4) according to the at least one movement behavior (9), the continuous data, and the stored information about the visual environment (8); and at least one processor (12) configured to perform operations of controlling dynamic adjustment of the continuous movement (6) of the at least one eye (4) to the continuous movement (5) of the head (3) according to the continuous data by using the image stabilization constraint (7), The device (11) further comprises: at least one third input adapted to receive training movement sequence data (23) representing a training movement sequence of said head (3) and said at least one eye (4) associated with a training visual environment during a training phase in which a learning process is carried out; The at least one processor (12) further comprises, during the training stage: - triggering a sequence of movements (5) of the head (3) in the training visual environment corresponding to the training movement sequence data (23); - spatially assessing a current portion of the training visual environment from the training movement sequence data (23) and, if the current portion is not already recorded in the at least one memory (13), recording spatial information about the current portion (8') in the at least one memory (13); - determining information about at least a portion of the image stabilization constraints according to the training movement sequence data and the continuous movement of the head, taking into account spatially assessing the current portion of the training visual environment and recording the spatial information about the current portion, and recording the information about the at least a portion of the image stabilization constraints in the at least one memory; configured to learn at least a portion of the image stabilization constraints (7) by thereby providing a simulated behavior of the virtual mammal (1) to be simulated when, after the training phase, in the normal operating state, the virtual mammal (1) performs the at least one locomotion behavior in the visual environment through the continuously dynamically adjusted current movement of the head (3) and the at least one eye (4) and through spatially evaluating the visual environment (2); the device (11) for simulating the physiological behavior is configured for the operation of the at least one processor (12) as a modular system comprising functional modules (17, 18, 19), processing means, auxiliary tools and data structures interconnected through a message passing middleware, the virtual mammal of the device is implemented as a robot and comprises a substantial part, the head being a material element and the at least one eye being a camera, and the device (11) for simulating physiological behavior is adapted to control an effector that moves or activates the material element; Device (11).

2. 2. The device (11) of claim 1, wherein at least one of the actions of spatially evaluating the current portion of the visual environment (2), recording the spatial information (8, 8'), and triggering the continuous movement (5, 6) is associated with at least one task parameter specific to the mammal to be simulated, and wherein the at least one processor (12) is further configured to learn the at least one task parameter during the training phase according to the training movement sequence data and the continuous movement (5) of the head (3), taking into account the spatial evaluation of the current portion of the training visual environment and the recording of the spatial information related to the current portion.

3. The at least one processor further comprises: - acquiring at least one visual stream (10) corresponding to said continuous data representative of the posture of said at least one eye (4), spatially assessing said current part of said visual environment from said at least one visual stream (10), triggering said continuous movement and in particular controlling said dynamic adjustment according to said at least one visual stream (10); - characterized in that it is configured to acquire, during the training phase, at least one training visual stream (10') corresponding to said training movement sequence data (23) representing said training movement sequence, and to spatially evaluate said current portion of said training visual environment, and in particular to identify said information according to said at least one training visual stream (10'), 3. An apparatus (11) according to claim 1 or 2.

4. 4. The device (11) of claim 3, wherein the at least one processor (12) is further configured to condition the acquired at least one visual stream (10) and the at least one training visual stream (10') in association with at least one conditioning parameter specific to the mammal being simulated, and the at least one processor (12) is further configured to learn the at least one conditioning parameter during the training phase according to the at least one training visual stream (10') and the continuous movement, taking into account a spatial evaluation of the current portion of the training visual environment and recording of the spatial information related to the current portion.

5. The at least one processor (12) further comprises: - deriving successive images from said at least one visual stream (10); - configured to control said dynamic adjustment by stabilizing at least a portion of said derived successive images, 5. An apparatus (11) according to claim 3 or 4.

6. 6. The device (11) according to any one of claims 3 to 5, characterized in that the device (11) comprises at least one fourth input adapted to receive at least one simulated visual characteristic, and the at least one processor (12) is further configured for dynamically conditioning the at least one visual stream (10) according to the at least one simulated visual characteristic.

7. 7. The device (11) according to claim 6, characterized in that the at least one simulated visual characteristic is at least partially related to a physiological behavior associated with the at least one eye (4).

8. 8. Apparatus (11) according to claim 6 or 7, characterized in that said at least one simulated visual property is at least partly related to the visual equipment of said at least one eye (4).

9. 9. The device (11) according to claim 1, further comprising at least one fifth input adapted to receive information relating to a modified value of the image stabilization constraint (7), wherein the at least one processor (12) is configured to replace information relating to a previous value of the image stabilization constraint (7) in the memory (13) with information relating to the modified value.

10. 10. The apparatus (11) of claim 9, wherein the previous and revised values ​​of the image stabilization constraints (7) correspond to at least one of a pair of different mammal ages and a pair of different health states.

11. The device (11) according to any one of claims 1 to 10, characterized in that the at least one processor (12) comprises at least one first neural network (18) configured for controlling the dynamic adjustment by means of a feedback loop.

12. 12. The device according to any one of claims 1 to 11, characterized in that the at least one processor (12) comprises at least one second neural network (19) configured for spatially evaluating the current portion of the visual environment and for recording spatial information relating to the current portion.

13. the at least one processor (12) is configured to perform at least one of the following operations: spatially assessing the current portion of the visual environment (2) by simulated spiking neurons (17), recording the spatial information (8, 8') related to the current portion, and controlling the dynamic adjustment of the continuous movement (6) of the at least one eye (4), An apparatus (11) according to any one of claims 1 to 12.

14. 1. A process for simulating physiological behavior of the visual system of a real mammal located in a visual environment (2) by a virtual mammal (1), the visual system of the real mammal including movable eyes, the visual environment corresponding to what can be seen by the eyes at the location where the real mammal is located, the eyes having retinas from which an image of a current portion of the visual environment is obtained, the current portion of the visual environment being defined by a gaze direction and a field of view of the eyes, the visual system of the real mammal including image stabilization means for constraining the movement of the eyes to stabilize the image on the retina, the virtual mammal (1) a head (3) that is movable (5) relative to said visual environment; and at least one eye (4) that is rotatably movable (6) relative to the head (3), The process, under normal operating conditions, includes the following operations: - receiving continuous data representative of the pose of said at least one eye (4) relative to said visual environment (2); - receiving instructions for at least one movement behavior (9) of said virtual mammal (1) in said visual environment (2); spatially assessing a current portion of the visual environment (2) from the continuous data by at least one processor (12) using information about the visual environment (8) stored in at least one memory (13), the spatial assessment being performed in a hippocampal module simulating the operation of the hippocampal formation by means of a neural network, and recording spatial information (8) about the current portion in the at least one memory (13) if the current portion has not already been recorded in the at least one memory (13); triggering, by the at least one processor, continuous movements (5, 6) of the head (3) and the at least one eye (4) according to the at least one movement behavior (9), the continuous data, and the stored information about the visual environment (8); and controlling, by the at least one processor, a dynamic adjustment of the continuous movement (6) of the at least one eye (4) to the continuous movement (5) of the head (3) according to the continuous data by using an image stabilization constraint (7) between the movement (6) of the at least one eye (4) and the movement (5) of the head (3), wherein the image stabilization constraint (7) is related to at least one of a vestibulo-ocular reflex, intentional eye fixation, and an optokinetic reflex, and the image stabilization constraint (7) constitutes a simulation of the image stabilization means in a vestibulo-ocular reflex control module; The process further comprises, during a training phase in which a learning process is carried out, the following operations: - receiving training movement sequence data (23) representing a training movement sequence of the head (3) and the at least one eye (4) associated with a training visual environment during said training phase; learning, by the at least one processor, during the training phase; - triggering a series of movements (5) of the head (3) in the training visual environment corresponding to the training movement sequence; - spatially assessing a current portion of the training visual environment from the training movement sequence data and, if the current portion has not already been recorded in the at least one memory (13), recording spatial information about the current portion (8') in the at least one memory (13); - determining information about at least a portion of the image stabilization constraints according to the training movement sequence data and the continuous movement of the head, taking into account spatially assessing the current portion of the training visual environment and recording the information about the current portion, and recording the spatial information about the at least a portion of the image stabilization constraints in the at least one memory; and learning at least a portion of the image stabilization constraints (7) by As a result, the virtual mammal (1) provides a simulated behavior of the mammal to be simulated when performing the at least one locomotion behavior in the visual environment through the continuously dynamically adjusted current movement of the head (3) and the at least one eye (4) and through spatially evaluating the visual environment (2). process.

15. A computer program comprising instructions that, when executed by a processor, cause the processor to carry out the process of claim 14.

Images