Head-mounted display device

Abstract

The embodiment of the invention discloses a head-mounted display device. According to one specific embodiment, the head-mounted display device comprises display units, a first display unit and a second display unit are configured to display training content corresponding to amblyopia, the training content comprises a background image layer and a foreground image layer, and the area proportion of elements in the foreground image layer is smaller than a preset proportion; an interaction unit configured to receive an interaction operation; the main control unit is configured to perform synchronous processing on the training contents displayed in the first display unit and the second display unit, and update the training contents displayed in the first display unit and / or the second display unit according to the interactive operation; the main control unit is further configured to superpose the background image layer and the foreground image layer and control the first display unit and / or the second display unit to display the superposed content. According to the embodiment, the training content provided by the device is enriched, the portability is good, and the compliance of teenager users on the head-mounted display device for amblyopia training is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to a head-mounted display device. Background Art

[0002] A amblyopia training device is a tool or instrument for amblyopia users. Currently, commonly used amblyopia training devices include: tools or instruments using occlusion methods, punctate methods, fine visual acuity training methods (such as threading beads or needles), red filters, and grating patterns.

[0003] However, when using the above devices, there are often the following technical problems: the training methods provided by the devices are single, and the portability of the devices is poor (for example, a device using the occlusion method requires an additional occlusion cloth, a device using the punctate method requires a pen and paper respectively, and a device using bead threading requires beads and a thread), resulting in poor compliance of adolescent users with the amblyopia training device.

[0004] The above information disclosed in this background art section is only used to enhance the understanding of the background of the inventive concept, and thus, it may include information that does not form the prior art known to those of ordinary skill in the art in this country. Summary of the Invention

[0005] This content part of the present disclosure is used to briefly introduce concepts, which will be described in detail in the following detailed implementation part. This content part of the present disclosure is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] Some embodiments of the present disclosure propose a head-mounted display device to solve one or more of the technical problems mentioned in the above background art section.

[0007] Some embodiments of the present disclosure provide a head-mounted display device, which includes: a display unit including a first display unit and a second display unit, the first display unit and the second display unit are respectively configured to display training content corresponding to amblyopia, wherein the training content corresponding to amblyopia includes a background layer and a foreground layer, and the area ratio of the elements in the foreground layer is less than a preset ratio; an interaction unit configured to receive the user's interaction operation; a main control unit configured to synchronize the training content displayed in the first display unit and the second display unit, and update the training content displayed in the first display unit and / or the second display unit according to the user's interaction operation; the main control unit is further configured to superimpose the background layer and the foreground layer included in the training content to obtain a superimposed content, and control the first display unit and / or the second display unit to display the superimposed content.

[0008] Optionally, the above-mentioned head-mounted display device further includes a storage unit configured to store the user's interaction operation records.

[0009] Optionally, the above-mentioned head-mounted display device further includes a sound playback unit configured to play an audio corresponding to the training content.

[0010] Optionally, the above-mentioned main control unit is further configured to: determine the training content corresponding to the first display unit as a first picture; determine the training content corresponding to the second display unit as a second picture; control the first display unit to render the first picture according to the first rendering parameter; control the second display unit to render the second picture according to the second rendering parameter.

[0011] Optionally, the above-mentioned main control unit is further configured to update the training content displayed in the first display unit and / or the second display unit according to the user's interaction operation through the following steps: generate training effect information according to the interaction operation record and the training content corresponding to the interaction operation record; determine the training gradient information corresponding to the training effect information according to the training effect information; generate the training content corresponding to the training gradient information as the updated training content according to the training gradient information; control the first display unit and / or the second display unit to display the updated training content.

[0012] Optionally, the above-mentioned main control unit is further configured to: in response to determining that the current training mode meets the first mode condition, control the main body elements included in the training content to randomly appear in the first display unit or the second display unit, where the main body elements are located in the foreground layer; in response to determining that the interaction operation record corresponding to the main body elements meets the preset passing condition, update the training gradient information of the first mode; determine the display switching information according to the updated training gradient information; control the main body elements to randomly appear in the first display unit or the second display unit according to the display switching information.

[0013] Optionally, the above-mentioned main control unit is further configured to: in response to determining that the updated training gradient information meets the preset advanced condition, split the main body elements to obtain a first main body element and a second main body element; control the first display unit to display the first main body element, and control the second display unit to display the second main body element.

[0014] Optionally, the above-mentioned main control unit is further configured to: in response to determining that the current training mode meets the second mode condition, control the above-mentioned first display unit and the above-mentioned second display unit to display the main body element in the foreground layer in a display manner with parallax; determine the first movement parameter information corresponding to the main body element according to the current training gradient information of the second mode, wherein the above-mentioned first movement parameter information includes a movement range; control the above-mentioned first display unit and the above-mentioned second display unit to move the main body element according to the above-mentioned first movement parameter information.

[0015] Optionally, the above-mentioned main control unit is further configured to: in response to determining that the current training mode meets the third mode condition, control the above-mentioned first display unit and the above-mentioned second display unit to display at least one moving element in the foreground layer in a display manner with parallax; determine the second movement parameter information corresponding to each moving element in the above-mentioned at least one moving element according to the current training gradient information of the third mode, wherein the second movement parameter information includes a movement range, a movement speed, and a movement direction; for each moving element in the above-mentioned at least one moving element, control the above-mentioned first display unit and the above-mentioned second display unit to move the moving element according to the second movement parameter information corresponding to the moving element.

[0016] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the head-mounted display device of some embodiments of the present disclosure, the training content provided by the device is enriched, the portability is relatively good, and the compliance of adolescent users with the head-mounted display device for amblyopia training is improved. Specifically, the reasons for the poor compliance of adolescent users with amblyopia training devices are as follows: The training methods that the device can provide are single, and the portability of the device is poor (for example, a device using the occlusion method requires an additional occlusion cloth, a device using the puncture point method requires a pen and paper respectively, and a device using bead threading requires beads and a thread), resulting in poor compliance of adolescent users with amblyopia training devices. Based on this, the head-mounted display device of some embodiments of the present disclosure includes: a display unit, including a first display unit and a second display unit, the first display unit and the second display unit are respectively configured to display training content corresponding to amblyopia, wherein the training content corresponding to amblyopia includes a background layer and a foreground layer, and the area ratio of the elements in the foreground layer is less than a preset ratio; an interaction unit, configured to receive the user's interaction operation; a main control unit, configured to synchronize the training content displayed in the first display unit and the second display unit, and update the training content displayed in the first display unit and / or the second display unit according to the user's interaction operation; the main control unit is further configured to superimpose the background layer and the foreground layer included in the training content to obtain a superimposed content, and control the first display unit and / or the second display unit to display the superimposed content. Thus, the training content for amblyopia training can be directly displayed through the two display units of the head-mounted display device, the portability is relatively good, and the training content is divided into a background layer and a foreground layer, which can provide different types of training methods, enrich the training content provided by the device, and because the area ratio of the elements in the foreground layer is less than the preset ratio, it can enable the user to view both most of the content of the background layer and the elements in the foreground layer, improving the superimposed training effect of the content in the background layer and the foreground layer. In addition, the training content displayed in the display unit can be updated according to the user's interaction operation, further enriching the training content provided by the device. Thus, the training content provided by the device is enriched, the portability is relatively good, and the compliance of adolescent users with the head-mounted display device for amblyopia training is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the various embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the elements and elements are not necessarily drawn to scale.

[0018] Figure 1 is a schematic structural diagram of some embodiments of the head-mounted display device according to the present disclosure; Figure 2 is a schematic diagram of an application scenario of a head-mounted display device according to some embodiments of the present disclosure; Figure 3 is a schematic structural diagram of a head-mounted display device suitable for implementing some embodiments of the present disclosure. Detailed implementation manners

[0019] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0020] In addition, it should be noted that for the sake of convenience of description, only parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence relationship of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "plural" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] Regarding the collection, storage, use and other operations of the user's personal information (such as vision information, user information and habit information) involved in the present disclosure, before performing the corresponding operations, relevant organizations or individuals shall fulfill obligations including conducting a personal information security impact assessment, fulfilling an obligation to inform the personal information subject, and obtaining the prior authorization and consent of the personal information subject.

[0025] The present disclosure will be described in detail below with reference to the drawings and in combination with embodiments.

[0026] Figure 1 Shows a schematic structural diagram of some embodiments of a head-mounted display device according to the present disclosure. The head-mounted display device includes a display unit 101, an interaction unit 102, and a main control unit 103.

[0027] In some embodiments, the display unit 101 may include a first display unit 1011 and a second display unit 1012. The first display unit 1011 and the second display unit 1012 may respectively correspond to the user's left eye and right eye. For example, the first display unit 1011 may be a display module corresponding to the left eye. The second display unit 1012 may be a display module corresponding to the right eye. The display module may include, but is not limited to: a micro display screen, an optical element. The head-mounted display device may include, but is not limited to: an AR glasses, a VR glasses. The above-mentioned first display unit 1011 and the above-mentioned second display unit 1012 may be respectively configured to display training content corresponding to amblyopia. Among them, the training content corresponding to amblyopia includes a background layer and a foreground layer. The background layer may be a layer for displaying the underlying background. The background image displayed in the background layer may fill the entire display range. The background image may include, but is not limited to: a spatial frequency spectrum diagram, an amblyopia training diagram with superimposed red light stimulation. The foreground layer may be a layer for displaying interactive elements. The interactive elements may be displayed, moved or transformed in the foreground layer. The area ratio of the elements in the above-mentioned foreground layer is less than a preset ratio. Here, the area ratio of the elements in the foreground layer may be the ratio of the total area of all interactive elements in the foreground layer to the entire display range. For example, the preset ratio may be 15%. The maximum preset ratio does not exceed 30%, so that the background image displayed in the background layer can effectively play a training role.

[0028] It should be noted that the training content displayed in the above-mentioned first display unit 1011 and the above-mentioned second display unit 1012 may be the same or different. When the training mode of the head-mounted display device is binocular training, the above-mentioned first display unit 1011 and the above-mentioned second display unit 1012 may both display training content. When the training mode of the head-mounted display device is monocular training, only one of the above-mentioned first display unit 1011 and the above-mentioned second display unit 1012 displays training content, and the other does not display a picture or displays a simulated all-black picture. Thus, rendering occlusion can be achieved through the binocular different display method, which simplifies the occlusion method.

[0029] In some embodiments, the interaction unit 102 may be configured to receive the user's interaction operation. The interaction unit 102 may include at least one of the following: an operation handle, a touchpad, a microphone, a movable host. Buttons or touch keys for the user to operate may be configured on the movable host, and it can also provide computing power, power supply, and charging for the head-mounted display device. The operation handle, the touchpad, and the movable host may allow the user to interact in ways of pressing buttons, touching, and remote control (including remote control in terms of direction and speed dimensions). The microphone may allow the user to interact by voice.

[0030] In some embodiments, the main control unit 103 may be a central processing unit. The main control unit 103 may be configured to synchronize the training content displayed on the first display unit 1011 and the second display unit 1012, and update the training content displayed on the first display unit 1011 and / or the second display unit 1012 according to the user's interaction operation. In practice, the main control unit 103 may synchronize the rendered images of the first display unit 1011 and the second display unit 1012 through a synchronization frame synthesizer, and then display the synchronized images on the first display unit 1011 and the second display unit 1012. The time difference between the left-eye and right-eye images pushed by the synchronization frame synthesizer to the left-eye and right-eye display screens may not exceed 34 milliseconds in the standard mode and may not exceed 10 milliseconds in the high-refresh mode. The main control unit 103 may display the next training content on the first display unit 1011 and / or the second display unit 1012 according to a preset training content sequence when the interaction operation meets the switching condition of the current training content. The user's interaction methods for the training content may include, but are not limited to: fine operation interaction methods, skill operation interaction methods, information analysis interaction methods. For example, referring to Figure 2 , the interaction operation may be an operation for the user to select a letter from each letter. When the letter selected by the user is "A", the switching condition is not met. When the letter selected by the user is "M", the switching condition is met, and the next training content is displayed. In addition, Figure 2 In, the black and white vertical stripe figure is the background figure, and each element in the layer superimposed on the background figure is an interactive element in the foreground layer.

[0031] In some embodiments, the main control unit 103 may also be configured to: superimpose the background layer and the foreground layer included in the training content to obtain a superimposed content; control the first display unit and / or the second display unit to display the superimposed content. In practice, the main control unit 103 may superimpose the background layer and the foreground layer included in the training content through a hierarchical rendering method to obtain a superimposed content. Thus, the background layer and the foreground layer can be superimposed and displayed in the same field of view.

[0032] Optionally, the head-mounted display device may further include a storage unit. The storage unit is configured to store the user's interaction operation records. The storage unit may include at least one of the following: ROM (Read Only Memory), RAM (Random Access Memory), UFS (Universal Flash Storage), EMMC (Embedded Multimedia Card). The interaction operation record may be an information record generated by the interaction operation, and may include at least one of the following: operation time, operating user, operation type, element on which the operation acts, operation duration. Thus, the user's interaction process can be recorded.

[0033] Optionally, the above-mentioned head-mounted display device may further include a sound playback unit. The sound playback unit is configured to play an audio corresponding to the training content. The sound playback unit may include a speaker. The audio corresponding to the training content may be an audio for synchronous playback with the training content. Thus, the training content in a multimedia form can be played.

[0034] Optionally, the above-mentioned main control unit may further be configured to: determine the training content corresponding to the first display unit as a first picture; determine the training content corresponding to the second display unit as a second picture; control the first display unit to render the first picture according to a first rendering parameter; control the second display unit to render the second picture according to a second rendering parameter. Wherein, the first rendering parameter may be the parameter-related content preset for rendering the display picture in the first display unit. The second rendering parameter may be the parameter-related content preset for rendering the display picture in the second display unit. The first rendering parameter and the second rendering parameter may include, but are not limited to, at least one of the following: brightness, contrast, sharpness. Further, these rendering parameters may be applied to the entire picture or applied to a part of the picture according to a set coordinate mapping function, or take different values according to the coordinate position difference. It should be noted that the training content corresponding to the first display unit and the training content corresponding to the second display unit may be the same or different. The first rendering parameter and the second rendering parameter may be the same or different. When the first rendering parameter and the second rendering parameter are different, pictures with different effects can be rendered in the first display unit and the second display unit, realizing binocular different display rendering. For example, the brightness included in the first rendering parameter or the second rendering parameter may be 0, and at this time, the corresponding display unit is equivalent to not displaying content.

[0035] Optionally, the above-mentioned main control unit may be further configured to update the training content displayed in the first display unit and / or the second display unit according to the user's interaction operation through the following steps: First step, generate training effect information based on the interaction operation record and the training content corresponding to the interaction operation record. In practice, the operation accuracy rate can be generated as the training effect information based on the interaction operation record and the training content corresponding to the interaction operation record. For example, the interaction operation record may include: the selected letter "A", and the correct letter corresponding to the training content is "M", then this interaction operation record is an incorrect interaction operation record. If the interaction operation record includes: the selected letter "M", then this interaction operation record is a correct interaction operation record. Subsequently, the proportion of correct interaction operation records among all interaction operation records in this training can be determined as the operation accuracy rate. The correct operation duration can also be generated as the training effect information based on the interaction operation record and the training content corresponding to the interaction operation record. For example, the interaction operation record may include: the selected letter "M", the operation duration "10 seconds", and the correct letter corresponding to the training content is "M", then the operation duration included in this interaction operation record is the correct operation duration.

[0036] Second step, determine the training gradient information corresponding to the above training effect information based on the above training effect information. In practice, the training gradient information corresponding to the above training effect information can be determined according to a pre-set effect gradient comparison table. The effect gradient comparison table can be a comparison table of pre-configured training effect information and training gradient information, and the corresponding training gradient information can be found from the effect gradient comparison table through the training effect information. The training gradient information can represent the level of the current user's training. The higher the level, the higher the complexity and fineness of the corresponding training content can be represented. For example, the complexity of the training content can be increased by accelerating the movement speed of the movement elements and reducing the area of the interaction elements, and the fineness of the training content can be increased by increasing the number of interaction elements.

[0037] Third step, generate the training content corresponding to the above training gradient information as the updated training content based on the above training gradient information. The corresponding training content can be pre-configured for each training gradient information. In practice, the pre-configured training content corresponding to the above training gradient information can be determined as the updated training content.

[0038] Fourth step, control the above first display unit and / or the above second display unit to display the above updated training content. Thus, the training content of the corresponding gradient can be displayed according to the differences in the user's amblyopia degree and the interaction operation performance during the training process.

[0039] Optionally, the above main control unit can also be configured to execute the following steps: In the first step, in response to determining that the current training mode meets the first mode condition, control the main elements included in the training content to randomly appear on the first display unit or the second display unit, where the main elements are located in the foreground layer. The training mode can be a mode selected by the user or automatically determined for training amblyopia. The training mode can include, but is not limited to: monocular training mode, binocular training mode. The binocular training mode can include, but is not limited to, at least one of the following: simultaneous vision mode, fusion vision mode, stereoscopic vision mode. The first mode condition can be that the current training mode is the simultaneous vision mode. The main elements can be the main elements that can attract the user's attention and are displayed in the foreground layer in the training content.

[0040] In the second step, in response to determining that the interaction operation record corresponding to the main elements meets the preset passing condition, update the training gradient information of the first mode. The preset passing condition can be that the operation accuracy rate determined according to the interaction operation record is greater than the preset ratio or the correct operation duration is less than the preset duration. The first mode can be the simultaneous vision mode. In practice, the training gradient information of the first mode can be incremented by one step length to update the training gradient information of the first mode. The step length can represent the level of a training gradient.

[0041] In the third step, determine the display switching information according to the updated training gradient information. Among them, the display switching information corresponding to each training gradient information can be pre-configured. The display switching information can be parameter-related information used to configure the switching display of the main elements between the first display unit and the second display unit. The display switching information can include, but is not limited to, at least one of the following: switching frequency, transition time of switching. The higher the training gradient information, the faster the switching frequency and the shorter the transition time of the switching included in the corresponding display switching information.

[0042] In the fourth step, control the main elements to randomly appear on the first display unit or the second display unit according to the above display switching information. In practice, the main elements can be randomly switched and displayed on the first display unit or the second display unit according to the switching frequency and the transition time of switching included in the above display switching information. Thus, through the design of binocular different displays, the main elements can randomly appear on the first display unit or the second display unit, realizing the random target tracking training of a single eye.

[0043] Optionally, the above main control unit can also be configured to execute the following steps: First, in response to determining that the updated training gradient information meets a preset advancement condition, split the above-mentioned main element to obtain a first main element and a second main element. The preset advancement condition may be that the updated training gradient information is greater than or equal to a preset level. In practice, the main element may be split into a first main element and a second main element according to a preset splitting ratio and a preset splitting method. For example, the preset splitting ratio may be 1:1. The preset splitting method may be a top-bottom splitting. It is also possible to split the main element into a first main element and a second main element according to a random splitting ratio and a random splitting method.

[0044] Second, control the above-mentioned first display unit to display the above-mentioned first main element, and control the above-mentioned second display unit to display the above-mentioned second main element. Thus, a part of the split main element can be displayed in the first display unit, and the other part can be displayed in the second display unit at the same time, guiding and enhancing the binocular ability to process the same signal through the simultaneously presented different pictures.

[0045] Optionally, the above-mentioned main control unit may also be configured to perform the following steps: First, in response to determining that the current training mode meets the second mode condition, control the above-mentioned first display unit and the above-mentioned second display unit to display the main element in the foreground layer in a display manner with parallax. The second mode condition may be that the current training mode is a fusion vision mode. Thus, the main element can be presented in the first display unit and the second display unit at the same time according to the effect of presenting a binocular parallax 3D object.

[0046] Second, determine the first movement parameter information corresponding to the main element according to the current training gradient information of the second mode. Among them, the second mode may be a fusion vision mode. The first movement parameter information corresponding to each training gradient information in the second mode may be pre-configured. The first movement parameter information may be parameter-related information for controlling the movement of the main element in the picture in the second mode. The above-mentioned first movement parameter information may include, but is not limited to, a movement range. For example, under one training gradient information, the movement range included in the first movement parameter information may be from infinity to 5 meters away. Under the next training gradient information, the movement range included in the first movement parameter information may be from 10 meters away to 1 meter away. Under the next training gradient information, the movement range included in the first movement parameter information may be from 3 meters away to 30 centimeters away. 30 centimeters may be used as the limit of fusion vision training.

[0047] In the third step, control the first display unit and the second display unit to move the main element according to the first movement parameter information. In practice, the main element can be controlled to move within the movement range included in the first movement parameter information. At the same time, the first display unit and the second display unit display the main element in a parallax manner. Thus, a 3D object can be presented in the order of from far to near in space, gradually improving the binocular fusion degree.

[0048] Optionally, the main control unit can also be configured to perform the following steps: In the first step, in response to determining that the current training mode meets the third mode condition, control the first display unit and the second display unit to display at least one moving element in the foreground layer in a parallax display manner. The third mode condition can be that the current training mode is the stereopsis mode. Thus, the main element can be presented in the first display unit and the second display unit simultaneously with the effect of presenting a 3D object with binocular parallax.

[0049] In the second step, according to the current training gradient information of the third mode, determine the second movement parameter information corresponding to each moving element in the at least one moving element, where the third mode can be the stereopsis mode. The second movement parameter information can be pre-configured for each training gradient information in the third mode. The second movement parameter information can be parameter-related information for controlling the movement of the main element in the picture in the third mode. The second movement parameter information can include, but is not limited to: movement range, movement speed, movement direction. The higher the level corresponding to the training gradient information, the closer the corresponding movement range, the faster the movement speed, and the more movement directions available.

[0050] In the third step, for each moving element in the at least one moving element, control the first display unit and the second display unit to move the moving element according to the second movement parameter information corresponding to the moving element. Thus, through the design of binocular differential display, the main element can be presented in the first display unit and the second display unit simultaneously with the effect of presenting a 3D object with binocular parallax, gradually improving the binocular stereopsis intensity.

[0051] Optionally, the main control unit can also be configured to perform the following steps: First step, obtain the vision information, user information, and habit information of the target user. The vision information may include diopter values and amblyopia levels. The user information may include, but is not limited to, the user's age and gender. The habit information may include, but is not limited to, the usual time and duration of using the head-mounted display device. In practice, the vision information, user information, and habit information of the target user pre-stored in the storage unit can be read, or the target user can be prompted to fill in the vision information, user information, and habit information through a question-and-answer interface. The target user can be the user who needs to undergo amblyopia training currently.

[0052] Second step, based on the above vision information, user information, and habit information, match the initial training content from the pre-constructed training content knowledge base to obtain an initial training content set. The above training content knowledge base can be a pre-constructed knowledge base containing different types of training content. Each training content can correspond to matching vision information, user information, and habit information, and can also correspond to a training type. Different types of training content can be training plans or training tasks configured for the matching vision information, user information, and habit information. The training type can represent the type of interaction method of the training content. The type of interaction method can include, but is not limited to, color recognition, pattern tracking, and letter recognition. And each training content can correspond to priority information, which can represent the order of the training content during the training process. The training content with a higher priority is simpler.

[0053] Third step, according to the priority information corresponding to the above initial training content set, sort each initial training content in the above initial training content set to obtain an initial training content sequence. In practice, each initial training content in the above initial training content set can be sorted in descending order of priority to obtain an initial training content sequence.

[0054] Fourth step, for each initial training content in the above initial training content sequence, perform the following training steps: First sub-step, control the above first display unit and / or the above second display unit to display the above initial training content.

[0055] Second sub-step, in response to detecting an interaction operation corresponding to the above initial training content through the above interaction unit, determine whether the above interaction operation meets the preset passing condition according to the interaction operation record corresponding to the above interaction operation. In practice, when it is determined that the interaction operation record is a correct interaction operation record or the correct operation duration is less than the preset duration, it is determined that the above interaction operation meets the preset passing condition.

[0056] The third sub-step, in response to determining that the above interaction operation does not meet the preset passing condition, controls the above first display unit and / or the above second display unit to display a repeated operation prompt message for the target user to perform an interaction operation on the above initial training content again. The repeated operation prompt message can be pre-set information to prompt the user to operate again. For example, the repeated operation prompt message can be "Failed, please try again."

[0057] The fourth sub-step, in response to determining that the above interaction operation meets the preset passing condition or the display times of the repeated operation prompt message meet the preset times condition, controls the storage unit to store each interaction operation record and each operation result corresponding to the above initial training content, and based on the next initial training content of the above initial training content in the above initial training content sequence, executes the above training step again. The preset times condition can be that the display times are greater than or equal to the preset times.

[0058] The fifth step, for each initial training content in the above initial training content sequence, generates initial training effect information corresponding to the above initial training content according to each interaction operation record and each operation result stored in the storage unit corresponding to the above initial training content. Here, the method of generating the initial training effect information can refer to the method of generating the above training effect information, which will not be elaborated here.

[0059] The sixth step, selects the initial training effect information that meets the preset effect condition from the generated initial training effect information as the training effect information to be improved, and obtains each training effect information to be improved. The preset effect condition can be that the operation correct rate is greater than the preset ratio or the correct operation duration is less than the preset duration.

[0060] The seventh step, determines each initial training content corresponding to the above each training effect information to be improved as each training content to be improved.

[0061] The eighth step, determines each training type corresponding to the above each training content to be improved as each training type to be improved.

[0062] The ninth step, generates updated training content according to the above vision information, the above user information, the above habit information, the above each training type to be improved and a pre-constructed training content generation model. Among them, the above training content generation model can be a pre-constructed lightweight model, and does not rely on a traditional deep neural network, takes the vision information, user information, habit information and each training type to be improved as inputs, and takes the updated training content as the output. The above training content generation model can include a user feature encoding layer, a training strategy matching layer, a training content generator, a behavior prediction layer, a content dynamic regulator and a training content renderer.

[0063] In some alternative implementations of some embodiments, the above-mentioned execution entity may generate a model and updated training content according to the above-mentioned vision information, user information, habit information, each training type to be improved, and pre-constructed training content through the following steps: First step, input the above-mentioned vision information, user information, habit information, and each training type to be improved into the user feature encoding layer included in the pre-constructed training content generation model to obtain a user feature vector. The user feature encoding layer can be used to convert the original information into a standardized form that the system can understand. In practice, the user feature encoding layer can classify the diopter values and segment the user's age. It can be output in the form of a standardized user feature list to obtain a user feature vector. The user feature vector may include standardized vision information, user information, habit information, and each training type to be improved.

[0064] Second step, input the above-mentioned user feature vector into the above-mentioned training strategy matching layer to obtain training content template information and training parameter information. The training strategy matching layer can select the most matching training template from a pre-designed "training strategy knowledge base" according to the user feature vector. Specifically, a decision tree structure or conditional judgment statements can be used for matching. Each training template may include various training contents in sequence. The training content template information may include, but is not limited to: template number. The training parameter information may include, but is not limited to: difficulty level, stimulation intensity, color combination, movement speed.

[0065] Third step, input the above-mentioned training content template information and training parameter information into the above-mentioned training content generator to obtain the first training content. The training content generator can generate specific visual stimulation content according to the template parameters. Specifically, an image generation rule library can be used to generate the background image in the background layer, elements in the foreground layer can be generated through an animation script template, and at the same time, the element attributes such as position, color, size, path, speed, etc. can be controlled through a configuration file to obtain the first training content.

[0066] Fourth step, input the above-mentioned first training content into the above-mentioned behavior prediction layer to obtain a behavior prediction result corresponding to the above-mentioned first training content. The behavior prediction layer can predict whether the user can successfully complete the task according to the interaction operation records of previous similar users and the current training content, and obtain the prediction success rate as the behavior prediction result. Specifically, a lightweight model (such as linear regression, KNN, logistic regression) or an empirical formula can be used for prediction, or a simple rule based on statistics (such as "if the element movement speed is faster than 3 pixels / second, the success rate drops by 10%") can be used for prediction.

[0067] Step 5: Input the above first training content and the above behavior prediction result into the above content dynamic adjuster to obtain the second training content. The content dynamic adjuster can fine-tune the training content according to the prediction result to make it more suitable for the user's ability. For example, if the prediction success rate is too low, the element movement speed can be reduced or the element size can be enlarged. Specifically, the feedback adjustment rule library and the parameter automatic modification mechanism can be used to dynamically adjust the first training content. For example, if the prediction success rate < 60%, the movement speed of the element is slowed down by 20%.

[0068] Step 6: Input the above second training content into the training content renderer to obtain each frame of the training screen as the updated training content. The training content renderer can convert the abstract training content into a real playable game interface, can control the element display order, the animation playback rhythm, and the interaction feedback method (click / drag / voice), and can support the resolution adaptation and interaction method adaptation of different devices (touch screen / mouse / keyboard). Specifically, a game engine or a front-end framework (such as Unity, Cocos2d-x, HTML5 Canvas) can be used for rendering, and an animation script is used to control the element behavior.

[0069] The above steps 1-9 and steps 1-6 are an inventive point of the embodiment of the present disclosure, which solves the technical problem of "using the same set of training programs for all users, being unable to perform personalized adjustment according to multiple aspects of information such as the user's vision level, age, and device usage habits, and being difficult to adjust in real time according to the user's performance during the training process. The fixed training mode may cause the user to feel bored, and the user's compliance is low". The factors that lead to low user compliance are often as follows: using the same set of training programs for all users, being unable to perform personalized adjustment according to multiple aspects of information such as the user's vision level, age, and device usage habits, and being difficult to adjust in real time according to the user's performance during the training process. The fixed training mode may cause the user to feel bored. If the above factors are solved, the effect of improving user compliance can be achieved. To achieve this effect, the present disclosure first locates the weak items of the user during the amblyopia training process through the user's previous interaction operations. Then, through the user information integration of the user feature encoding layer and the training strategy matching of the training strategy matching layer, a training plan that best matches each user is formulated based on individual differences, and the model used is a non-deep learning architecture and does not rely on large-scale data training. After that, through behavior prediction and training content dynamic adjustment, the training content is instantaneously optimized according to the user's performance data, making the training process more interesting and easy to adhere to, improving the overall user experience, and thus improving user compliance. At the same time, because the traditional deep learning model structure is not used, it can be applied to the medical small sample scenario.

[0070] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the head-mounted display device of some embodiments of the present disclosure, the training content provided by the device is enriched, the portability is relatively good, and the compliance of adolescent users with the head-mounted display device for amblyopia training is improved. Specifically, the reasons for the poor compliance of adolescent users with amblyopia training devices are as follows: The training methods that the devices can provide are single, and the portability of the devices is relatively poor (for example, devices using the occlusion method require additional configuration of occlusion cloth, devices using the puncture point method require separate configuration of pens and paper, and devices using bead threading require configuration of beads and threads), resulting in poor compliance of adolescent users with amblyopia training devices. Based on this, the head-mounted display device of some embodiments of the present disclosure includes: a display unit, including a first display unit and a second display unit, the first display unit and the second display unit are respectively configured to display training content corresponding to amblyopia, wherein the training content corresponding to amblyopia includes a background layer and a foreground layer, and the area ratio of the elements in the foreground layer is less than a preset ratio; an interaction unit, configured to receive the user's interaction operation; a main control unit, configured to synchronize the training content displayed in the first display unit and the second display unit, and update the training content displayed in the first display unit and / or the second display unit according to the user's interaction operation; the main control unit is further configured to superimpose the background layer and the foreground layer included in the training content to obtain a superimposed content, and control the first display unit and / or the second display unit to display the superimposed content. Thus, the training content for amblyopia training can be directly displayed through the two display units of the head-mounted display device, the portability is relatively good, and the training content is divided into a background layer and a foreground layer, which can provide different types of training methods, enrich the training content provided by the device, and because the area ratio of the elements in the foreground layer is less than the preset ratio, the user can not only view most of the content of the background layer, but also view the elements in the foreground layer, improving the superimposed training effect of the content in the background layer and the foreground layer. In addition, the training content displayed in the display unit can be updated according to the user's interaction operation, further enriching the training content provided by the device. Thus, the training content provided by the device is enriched, the portability is relatively good, and the compliance of adolescent users with the head-mounted display device for amblyopia training is improved.

[0071] Reference is made below to Figure 3 , which shows a schematic structural diagram of a head-mounted display device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The head-mounted display device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0072] As Figure 3As shown, the head-mounted display device 300 may include a processing device 301 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the head-mounted display device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0073] Generally, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a micro display, a speaker, a vibrator, etc.; and a communication device 309. The communication device 309 may allow the head-mounted display device 300 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 3 a head-mounted display device 300 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices may be implemented or had alternatively. Figure 3 Each block shown in the figure may represent one device or, as needed, multiple devices.

[0074] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such some embodiments, the computer program may be downloaded and installed from a network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above functions defined in the methods of some embodiments of the present disclosure are executed.

[0075] It should be noted that the computer-readable media described in some embodiments of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0076] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed network.

[0077] The above computer-readable medium may be included in the above head-mounted display device; or it may exist independently without being assembled into the head-mounted display device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the head-mounted display device, the head-mounted display device is caused to: synchronize the training content displayed on the above first display unit and the above second display unit, and update the training content displayed on the above first display unit and / or the above second display unit according to the user's interaction operation; the above main control unit is further configured to superimpose the background layer and the foreground layer included in the training content to obtain superimposed content, and control the above first display unit and / or the above second display unit to display the above superimposed content.

[0078] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0080] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0081] The above description is only some preferred embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with (but not limited to) the technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A head-mounted display device, comprising: A display unit, including a first display unit and a second display unit, the first display unit and the second display unit are respectively configured to display training content corresponding to amblyopia, wherein the training content corresponding to amblyopia includes a background layer and a foreground layer, and the area ratio of the elements in the foreground layer is less than a preset ratio; An interaction unit, configured to receive user interaction operations; A main control unit, configured to synchronize the training content displayed in the first display unit and the second display unit, and update the training content displayed in the first display unit and / or the second display unit according to the user's interaction operations; The main control unit is further configured to superimpose the background layer and the foreground layer included in the training content to obtain a superimposed content, and control the first display unit and / or the second display unit to display the superimposed content.

2. The head-mounted display device according to claim 1, wherein, The head-mounted display device further includes a storage unit, and the storage unit is configured to store user interaction operation records.

3. The head-mounted display device according to claim 1, wherein, The head-mounted display device further includes a sound playback unit, and the sound playback unit is configured to play audio corresponding to the training content.

4. The head-mounted display device according to claim 1, wherein, The main control unit is further configured to: Determine the training content corresponding to the first display unit as a first picture; Determine the training content corresponding to the second display unit as a second picture; Control the first display unit to render the first picture according to a first rendering parameter; Control the second display unit to render the second picture according to a second rendering parameter.

5. The head-mounted display device according to claim 2, wherein, The main control unit is further configured to update the training content displayed in the first display unit and / or the second display unit according to the user's interaction operations through the following steps: Generate training effect information according to the interaction operation record and the training content corresponding to the interaction operation record; Determine training gradient information corresponding to the training effect information according to the training effect information; Generate training content corresponding to the training gradient information as updated training content according to the training gradient information; Control the first display unit and / or the second display unit to display the updated training content.

6. The head-mounted display device according to claim 1, wherein, The main control unit is further configured to: In response to determining that the current training mode meets the first mode condition, control the main elements included in the training content to randomly appear in the first display unit or the second display unit, wherein the main elements are located in the foreground layer; In response to determining that the interaction operation record corresponding to the main element meets a preset passing condition, update the training gradient information of the first mode; Determine display switching information according to the updated training gradient information; Control the main element to randomly appear in the first display unit or the second display unit according to the display switching information.

7. The head-mounted display device according to claim 6, wherein, The main control unit is further configured to: In response to determining that the updated training gradient information meets a preset advanced condition, split the main element to obtain a first main element and a second main element; Control the first display unit to display the first main element, and control the second display unit to display the second main element.

8. The head-mounted display device according to claim 1, wherein, The main control unit is further configured to: In response to determining that the current training mode satisfies the second mode condition, control the first display unit and the second display unit to display the main body element in the foreground layer in a display manner with parallax; Determine the first movement parameter information corresponding to the main body element according to the current training gradient information of the second mode, wherein the first movement parameter information includes a movement range; Control the first display unit and the second display unit to move the main body element according to the first movement parameter information.

9. The head-mounted display device according to claim 1, wherein, The main control unit is further configured to: In response to determining that the current training mode satisfies the third mode condition, control the first display unit and the second display unit to display at least one moving element in the foreground layer in a display manner with parallax; Determine the second movement parameter information corresponding to each moving element among the at least one moving element according to the current training gradient information of the third mode, wherein the second movement parameter information includes a movement range, a movement speed, and a movement direction; For each moving element among the at least one moving element, control the first display unit and the second display unit to move the moving element according to the second movement parameter information corresponding to the moving element.

Images