Method for acquiring lens information and near-eye display device

Abstract

The application discloses a method for obtaining lens information and a near-eye display device; the method comprises the following steps: obtaining basic information based on a user wearing a near-eye display device provided with a detachable lens; matching lens information according to the basic information; determining a line-of-sight direction through the lens based on the lens information and an initial eye tracking calculation model; and correcting the eye tracking calculation model of the near-eye display device according to the line-of-sight direction through the lens. The method solves the problem of complex implementation process of obtaining lens information, improves the user experience without the user entering the lens information, and does not change the structure and function of the lens and the near-eye display device, so as to not affect the optical effect of the lens and not increase the design difficulty of the near-eye display device.

Description

Technical Field

[0001] This application relates to the field of eye-tracking technology, and more particularly to a method for acquiring lens information and a near-eye display device. Background Technology

[0002] Near-eye display devices, such as VR, AR, and MR devices, are typically worn on the user's head to display images. Because users have different visual acuity and refractive errors, different users may need to install different corrective lenses when wearing near-eye display devices. Currently, near-eye display devices usually have built-in eye-tracking functionality. Adding a corrective lens between the user's eye and the camera would cause changes in lighting, thus affecting the accuracy of the eye-tracking algorithm.

[0003] Therefore, in order to avoid the impact of vision correction lenses on the accuracy of eye tracking calculations, near-eye display devices need to be able to obtain the parameters of vision correction lenses in a timely manner, so as to use the parameters of vision correction lenses to correct the eye tracking algorithm and ensure the accuracy of eye tracking calculations. Summary of the Invention

[0004] This application provides a method for acquiring lens information and a near-eye display device, so that the near-eye display device can acquire lens information in a timely manner to achieve correction of eye-tracking calculations.

[0005] According to one aspect of this application, a method for obtaining lens information is provided, comprising:

[0006] Based on the user's wear of a near-eye display device with detachable lenses, basic information is obtained;

[0007] Match lens information based on basic information;

[0008] The direction of the line of sight through the lens is determined based on lens information and an initial eye-tracking calculation model;

[0009] The eye-tracking calculation model for near-eye display devices is corrected based on the line of sight through the lens.

[0010] According to another aspect of this application, a near-eye display device is provided, the near-eye display device comprising:

[0011] At least one processor, and a memory communicatively connected to said at least one processor;

[0012] The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the method for acquiring lens information as described in any embodiment of this application.

[0013] The technical solution of this application embodiment obtains basic information based on a user-worn near-eye display device with a detachable lens; matches lens information based on the basic information; determines the viewing direction through the lens based on the lens information and an initial eye-tracking calculation model; and corrects the eye-tracking calculation model of the near-eye display device based on the viewing direction through the lens. This enables the near-eye display device to obtain lens information in a timely manner and use the lens information to correct the eye-tracking calculation of the near-eye display device, avoiding the influence of the lens on the accuracy of the eye-tracking calculation. It also allows for the associated storage of lens information, facilitating the reuse of lens information while ensuring the convenience and speed of obtaining lens information. Lens information can be pre-stored in the storage or service device of the near-eye display device. Basic information and lens information are stored together. When a user adds a lens to the near-eye display device, the lens information is directly obtained and used to correct the eye-tracking calculation of the near-eye display device, thereby improving the accuracy of eye-tracking. The method provided in this application embodiment can directly determine the corresponding lens information based on the basic information, which facilitates the entry of lens information and improves the user experience. Furthermore, it does not require changing or adding to the structure and function of the lens and the near-eye display device itself, does not affect the optical effect of the lens, and does not increase the design difficulty of the near-eye display device.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a method for obtaining lens information according to Embodiment 1 of this application;

[0017] Figure 2 This is a flowchart of another method for obtaining lens information according to Embodiment 1 of this application;

[0018] Figure 3 This is a flowchart of a method for obtaining lens information according to Embodiment 2 of this application;

[0019] Figure 4 This is a flowchart of a method for obtaining lens information according to Embodiment 3 of this application;

[0020] Figure 5This is a flowchart of a method for obtaining lens information according to Embodiment 4 of this application;

[0021] Figure 6 This is a flowchart of a method for obtaining lens information according to Embodiment 5 of this application;

[0022] Figure 7 This is a flowchart of a method for obtaining lens information according to Embodiment Six of this application;

[0023] Figure 8 This is a schematic diagram of a device for acquiring lens information according to Embodiment 7 of this application;

[0024] Figure 9 This is a schematic diagram of a device for acquiring lens information according to Embodiment 8 of this application;

[0025] Figure 10 This is a schematic diagram of a device for acquiring lens information according to Embodiment 9 of this application;

[0026] Figure 11 This is a schematic diagram of a device for acquiring lens information according to Embodiment 10 of this application;

[0027] Figure 12 This is a schematic diagram of the structure of a near-eye display device that implements the method for acquiring lens information according to the embodiments of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Example 1

[0031] This embodiment is applicable to situations where lens information can be acquired quickly and easily. The method can be executed by a device for acquiring lens information, which can be implemented in hardware and / or software and can be configured in a near-eye display device.

[0032] Figure 1 A flowchart of a method for acquiring lens information is provided, such as... Figure 1 As shown, the method includes:

[0033] S10. Obtain basic information based on the user's near-eye display device with a detachable lens.

[0034] In this embodiment, basic information can be understood as identification information of the user's identity, the near-eye display device, or a certain hardware component within the near-eye display device.

[0035] Specifically, basic information includes, but is not limited to, one or more of user identity information, account information, and hardware information. User identity information can be used to identify a user, such as iris information, fingerprint information, and voiceprint information. Account information can be stored in the account during the user's pre-registration process or based on account usage, including but not limited to user identity information, device information, and historical usage records.

[0036] Specifically, hardware information includes, but is not limited to, one or more of the following: device information and lens characteristics, such as the serial number of the near-eye display device and the lens serial number. Furthermore, lens characteristics include, but are not limited to, one or more of the following: lens power, diopter, refractive index, and curvature. For example, the lens power information may be 100 degrees, 200 degrees, or 500 degrees.

[0037] Optionally, obtaining basic information may include at least one or more of the following:

[0038] Basic information is obtained based on user manual input and / or manual selection;

[0039] Basic information is obtained based on the recognition of near-eye display devices.

[0040] In this embodiment, when a user wears a near-eye display device with a detachable lens, they can manually input basic information into the device based on input prompts. The near-eye display device then obtains the basic information used to match the lens information based on the user's manual input. For example, the display on the near-eye display device may show an input information prompt box, prompting the user to enter their account information to log in and retrieve their account information, or prompting the user to enter the lens power information (e.g., the user enters 100 degrees), or prompting the user to enter the lens number, etc.

[0041] As an optional embodiment, when a user wears a near-eye display device with a detachable lens, the display screen of the near-eye display device displays an information selection box, which contains at least one piece of information for the user to select. For example, the display screen of the near-eye display device displays the lens power information, such as 100 degrees, 200 degrees, 300 degrees, etc. Based on the user's selection of 100 degrees, the near-eye display device obtains the basic information that the lens power is 100 degrees.

[0042] As another optional embodiment, basic information can be obtained based on a sensing and recognition device provided in the near-eye display device, which includes, but is not limited to, a camera. For example, when a user wears a near-eye display device with detachable lenses, the camera can actively photograph the user's eyes, identify the user's iris information to determine the user's identity information, and enable the near-eye display device to obtain basic information, namely the user's identity information.

[0043] S20. Match lens information based on basic information.

[0044] In this embodiment, the lens information may be, but is not limited to, spherical power, cylindrical power, axial power, refractive power, refractive index, curvature, etc.

[0045] Lens information for different users can be pre-stored in the storage of the near-eye display device, or the corresponding lens information can be stored in the storage of the near-eye display device after a user has used a set of lens information. The near-eye display device stores lens information for different users and matches it against the basic information in its storage to determine if a match can be found. If a match is found, the lens information corresponding to the basic information is determined; otherwise, the match is determined to have failed.

[0046] Optionally, lens information can be matched based on basic information, including: matching lens information in the storage of the near-eye display device based on basic information.

[0047] In this embodiment, the storage terminal of the near-eye display device includes, but is not limited to, one or more of a memory and information tags, such as QR codes, barcodes, and quick-response codes. Lens information is pre-stored in the storage terminal of the near-eye display device. The lens information can be manually stored by the user or pre-stored by administrators, such as by pre-saving the lens information in an information tag.

[0048] Optionally, lens information can be matched based on the basic information, including: matching lens information from the service device based on the basic information.

[0049] In this embodiment, the service device can be the cloud, a server, a mobile smart device, etc.

[0050] The service device pre-stores lens information for different users. For example, after a user is fitted with lenses, they directly upload their lens information and basic information to the service device for storage. When a user adds lenses to a near-eye display device, the user's basic information is sent to the service device, which then matches the user's lens information.

[0051] Figure 2 A flowchart of another method for obtaining lens information is provided, optionally including matching lens information based on basic information, including:

[0052] S21. Determine whether there is matching lens information in the storage terminal of the near-eye display device for the basic information;

[0053] S22. If so, the lens information match is successful;

[0054] S23. If not, lens information matching fails. Then, lens information is matched from the service device based on the basic information, and the basic information and lens information are associated and stored in the storage terminal of the near-eye display device.

[0055] In this embodiment, the service device pre-stores lens information and basic information for different users. After a user equips their lens, they directly upload the lens information and basic information to the service device for storage. In other words, the service device stores the lens information and basic information for all users. The storage end of the near-eye display device only stores the lens information and basic information for a portion of the users. This lens information and basic information can be manually entered by the user or downloaded and saved from the service device.

[0056] Determine if there is matching lens information in the storage of the near-eye display device based on the basic information. If the matching lens information is successful in the storage of the near-eye display device based on the basic information, the matching lens information in the storage of the device is used to correct the eye-tracking calculation of the near-eye display device.

[0057] The successful matching of lens information with the basic information in the storage of the near-eye display device indicates that the storage of the near-eye display device has pre-stored the lens information associated with the basic information. The near-eye display device can automatically use the matched lens information directly for eye-tracking calculation correction.

[0058] If the lens information matching fails at the storage end of the near-eye display device based on the basic information, the lens information is matched from the service device based on the basic information.

[0059] If the basic information fails to match the lens information in the storage of the near-eye display device, the service device queries for the lens information that matches the basic information. For example, the basic information is sent to the service device, which queries the corresponding lens information based on the basic information and feeds it back to the execution device. The execution device receives the lens information and determines the lens information that matches the basic information.

[0060] Basic information is matched with lens information on the service device. After the relevant lens information is matched, the basic information and lens information are associated and stored on the storage side of the near-eye display device. When the user uses it next time, he / she can directly retrieve the matched lens information from the storage side of the near-eye display device based on the user's identity information.

[0061] Optionally, lens information can be matched in the storage of the near-eye display device based on the basic information, including:

[0062] Based on the basic information, query the local display device of the near eye to determine all lens information corresponding to the basic information;

[0063] If the number of lens information is one, then the lens information is determined to be the lens information corresponding to the lens;

[0064] If there are multiple lens information items, then display each lens information item to the user and determine the corresponding lens information based on the user's selection operation.

[0065] The near-eye display device can locally store basic information and corresponding lens information. It can query the local storage based on the basic information to determine all matching lens information. The number of lens information entries can be one or more. If there are multiple entries, and it's unclear which lens is installed in the near-eye display device, a lens information selection interface can be used to display the information to the user. The user can then determine which lens is installed in the near-eye display device. The user determines the unique lens information through selection operations on the lens information selection interface. For example, by displaying the lens information on the screen, the user can trigger the selection operation by clicking options, buttons, or entering lens information in the input box. The device receives the user's selection operation and determines the corresponding lens information based on the selection.

[0066] Optionally, lens information can be matched from the service device based on the basic information, including:

[0067] Based on the basic information query service equipment, receive all lens information corresponding to the basic information fed back by the service equipment;

[0068] If the number of lens information is one, then the lens information is determined to be the lens information corresponding to the lens;

[0069] If there are multiple lens information items, multiple lens information items will be displayed to the user, and the lens information corresponding to the selected lens will be determined according to the user's selection operation.

[0070] The service device can pre-store basic information and the corresponding lens information. A query is performed on the service device based on the basic information, and the service device returns the lens information matching the basic information to the execution device. The number of lens information entries can be one or more. If there are multiple lens information entries, and it is uncertain which lens is installed in the near-eye display device, the user can be shown the information through a lens information selection interface. The user can trigger a selection operation by clicking options or buttons on the screen, or entering lens information in the input box. The execution device receives the user's selection operation and determines the corresponding lens information based on the selection.

[0071] Optionally, when the service device is a mobile storage device, the system queries the service device based on the basic information and receives all lens information corresponding to the basic information fed back by the service device, including:

[0072] Detect service equipment and establish a connection with the service equipment;

[0073] Send basic information to the device and obtain all lens information corresponding to the basic information sent by the service device; or, obtain one or more basic information and the lens information corresponding to the basic information sent by the service device.

[0074] In this embodiment, the mobile storage terminal can be understood as a portable device with storage capabilities, such as a smartphone, tablet, etc. The system detects whether a service device exists in the vicinity of the execution device. If a service device is detected, a connection is established with the service device. This connection can be wireless, such as Bluetooth or a communication connection, or wired, via a data cable or power cord. The execution device sends basic information to the service device. After receiving the basic information, the service device performs lens information matching and feeds back all lens information matching the basic information to the execution device. The execution device receives all lens information matching the basic information fed back by the service device. Alternatively, the execution device may not send basic information to the service device. After establishing a connection with the service device, the service device proactively sends one or more stored basic information items, along with the lens information corresponding to each basic information item, to the execution device. The execution device then filters the lens information corresponding to all the basic information items to determine all lens information matching the basic information.

[0075] S30. Determine the direction of the line of sight through the lens based on lens information and the initial eye-tracking calculation model.

[0076] In this embodiment, the initial eye-tracking calculation model refers to the eye-tracking calculation model used without considering the installation of a corrective lens. This initial eye-tracking calculation model can be either a calibrated eye-tracking calculation model used when the near-eye display device is not equipped with a corrective lens, or an eye-tracking calculation model used when the near-eye display device is neither equipped with a corrective lens nor calibrated. The gaze direction through the lens refers to the corrected initial eye-tracking model formed after the initial eye-tracking model is corrected based on lens information.

[0077] In this embodiment, after the near-eye display device obtains the lens information of the installed lens, it incorporates the lens information into the initial eye-tracking calculation model. This allows the initial eye-tracking calculation model to consider the impact of changes in the camera's captured image caused by the lens on the eye-tracking calculation model. Based on the lens information, the initial eye-tracking model is corrected. The corrected initial eye-tracking model can represent the direction of the line of sight after passing through the lens, that is, the direction of the line of sight through the lens.

[0078] S40. Correct the eye-tracking calculation model of the near-eye display device according to the line of sight through the lens.

[0079] This application embodiment can calculate the eye's gaze position after passing through the lens based on the obtained gaze direction after passing through the lens, i.e., the corrected initial eye-tracking model. The corrected initial eye-tracking model is then further calibrated to correct the impact of the lens-induced change in the display position of the near-eye display device's screen on the eye-tracking calculation.

[0080] It is understandable that installing a new corrective lens in a near-eye display device, where the camera captures an image of the user's eyes through the lens, will distort this image, affecting the accuracy of eye-tracking calculations. Furthermore, the user's gaze at the display screen through the lens will cause a discrepancy between the displayed position seen by the user and the actual position of the screen, leading to a deviation between the user's gaze position and the actual position displayed on the screen. In other words, the shape, size, and position of the virtual image seen by the user through the corrective lens may differ from the virtual image seen without the corrective lens. Therefore, in addition to correcting the impact of the camera capturing the user's eyes through the lens on eye-tracking calculations, it is also necessary to correct the impact of the difference between the displayed screen position seen by the eyes through the lens and the actual display position on eye-tracking calculations. Thus, after correcting the initial eye-tracking calculation model based on lens information to obtain the gaze direction through the lens—that is, the corrected initial eye-tracking model—further processing is needed to correct the impact of the difference between the displayed screen position seen by the eyes through the lens and the actual display position on eye-tracking calculations, resulting in a more accurate final eye-tracking calculation model.

[0081] Optionally, the eye-tracking calculation model of the near-eye display device is corrected according to the line of sight through the lens, including:

[0082] Step 1: Determine the position of the virtual image based on the lens information and the eyepiece side information of the display system.

[0083] In this embodiment, the eyepiece-side information of the display system refers to various parameter information on the eyepiece side of the near-eye display device, that is, the various lens parameter information near the display screen of the near-eye display device. The eyepiece in the near-eye display device helps the device clearly project the interface information requiring a virtual image onto the user in an appropriate proportion and brightness. The eyepiece-side information parameters of the near-eye display system may include, but are not limited to, the eyepiece's information parameters, and the information parameters of multiple optical structural layers, such as the information parameters of each optical structure in a folded optical path. The eyepiece-side information of the display system can be pre-stored in the near-eye display device or pre-stored on the service device, and the near-eye display device can obtain it when needed.

[0084] In this embodiment, the virtual image position refers to the actual screen position seen by the user's eye after passing through the lens and various optical structures on the eyepiece side of the display system. In other words, the near-eye display device can determine the actual virtual image position seen by the user's eye after passing through the lens and various optical structures on the eyepiece side by acquiring lens information and eyepiece side information of the display system. The virtual image position can be represented as the position of the entire screen; the virtual image may be planar or curved, or it can be represented as the position of a portion of the virtual image, such as the upper left corner or a specific point within the virtual image.

[0085] Step 2: Determine the deviation function based on the virtual image position and the screen position.

[0086] In this embodiment, the screen position refers to the original display position in space where the near-eye display device projects the screen. Specifically, due to the influence of the lens, the position of the virtual image seen by the user will deviate from the original display position in space where the near-eye display device projects the screen. Based on this positional deviation, a deviation function between the virtual image position and the screen position can be fitted. In other words, the virtual image position can be corrected to the screen position based on this deviation function.

[0087] Step 3: Correct the line of sight through the lens based on the deviation function to obtain the corrected eye-tracking model.

[0088] In this embodiment, the corrected eye-tracking model can be the final eye-tracking model, used for subsequent eye-tracking calculations. Specifically, the gaze direction through the lens has already corrected for the influence of the lens on the eye image captured by the camera, and the deviation function is used to correct the influence of the difference between the position of the virtual image viewed through the lens and the screen position on the eye-tracking calculation. Therefore, based on the gaze direction through the lens corrected by the deviation function, the obtained corrected eye-tracking model can simultaneously correct the influence of changes in the eye image captured by the camera and changes in the screen position on the display screen on the eye-tracking calculation, enabling the final eye-tracking calculation model to more accurately calculate the user's eye gaze position.

[0089] As an optional embodiment, the corrected eye-tracking model can also be a preliminary correction of the gaze direction. Further corrections can be made to the preliminary correction of the gaze direction to obtain the final eye-tracking calculation model.

[0090] The viewing direction through the lens is corrected based on the deviation function to obtain a preliminary corrected viewing direction;

[0091] Based on the initially corrected gaze direction, calculate several sets of gaze positions with corresponding true values;

[0092] Determine the mapping relationship between several sets of gaze positions and their corresponding truth values;

[0093] The preliminary correction of the gaze direction is based on the mapping relationship to obtain the final eye-tracking calculation model.

[0094] In this embodiment, the gaze position refers to the user's gaze position calculated based on the initially corrected gaze direction when the user gazes at a certain location. The true value refers to the coordinate position of the calibration point during the calibration process. When the user gazes at a calibration point, the initially corrected gaze direction calculates the user's gaze position when gazing at that calibration point, and the calibration point has the correct coordinate position as a reference for whether the initially corrected gaze direction calculation is accurate.

[0095] Specifically, the viewing direction through the lens is corrected based on the deviation function to obtain a preliminary corrected viewing direction. This preliminary corrected viewing direction can then be further corrected based on the true value of the calibration point.

[0096] During the eye-tracking calibration process, the display interface of the near-eye display device will show multiple calibration points with different positions. The coordinate position of each calibration point represents the true value of that calibration point. When the user gazes at each calibration point, the user's gaze position is calculated based on the corrected initial eye-tracking model. The eye-tracking calculation model is then recalibrated based on the user's gaze position calculated based on the initially corrected gaze direction and the true value of the calibration point to obtain the final eye-tracking calculation model, ensuring the accuracy of eye-tracking calculation after the user's gaze passes through the lens.

[0097] The user's gaze position is calculated using a pre-corrected gaze direction. Each calculated gaze position should have a corresponding calibration point coordinate position, which is the true value. Several mapping relationships between gaze positions and their corresponding calibration point coordinate positions are determined; that is, this mapping relationship represents the relationship between the true value and the gaze position calculated based on the pre-corrected gaze direction. The pre-corrected gaze direction is then corrected based on this mapping relationship to obtain the final eye-tracking calculation model, which is used for eye-tracking calculations. A secondary correction of the gaze direction through the lens is performed using a bias function and true value correction to improve the calculation accuracy of the final eye-tracking calculation model.

[0098] Optionally, the eye-tracking calculation model of the near-eye display device is corrected according to the line of sight through the lens, including:

[0099] Based on the line of sight through the lens, calculate several sets of gaze positions with corresponding true values.

[0100] Determine the mapping relationship between several sets of gaze positions and their corresponding truth values.

[0101] The eye-tracking calculation model is obtained by correcting the viewing direction through the lens based on the mapping relationship.

[0102] In this embodiment, the gaze position refers to the user's gaze position calculated based on the direction of the line of sight through the lens when the user gazes at a certain position. The true value refers to the coordinate position of the calibration point during the calibration process.

[0103] This embodiment calculates the eye's gaze position after passing through the lens using the viewing direction through the lens. Each calculated gaze position should have a corresponding calibration point coordinate position, which is the true value. Several mapping relationships between gaze positions and their corresponding calibration point coordinate positions are determined; that is, this mapping relationship represents the relationship between the true value and the gaze position calculated based on the viewing direction through the lens. Based on this mapping relationship, the viewing direction through the lens is corrected to obtain the final eye-tracking calculation model for eye-tracking calculation. In this embodiment, it is not necessary to consider the eyepiece side information of the display system. The mapping relationship formed between the true value and the gaze position calculated based on the viewing direction through the lens directly corrects the impact of the difference between the virtual image position and the screen position caused by the lens on the eye-tracking calculation, ensuring the accuracy of the eye-tracking calculation after adding the lens.

[0104] Optionally, the method further includes: based on user identity information and lens characteristics, matching defects in lens information stored in the near-eye display device, generating a first prompt message to prompt the user, the first prompt message including prompting the user whether the lens is installed incorrectly or prompting the user whether to add lens information.

[0105] In this embodiment, the first prompt information can be understood as information used to prompt the user, which can be text, images, voice, etc., or a combination of multiple methods. For example, playing an animation while simultaneously playing a voice prompt and displaying the corresponding text. The prompting method of the first prompt information can be preset, or multiple prompting methods can be set for the user to choose from. For example, for users with hearing impairments, they can choose images, animations, text, etc., and the user can determine the content of the prompt by viewing the images, animations, text, etc.; for users who cannot read, such as young children, users with poor eyesight, or users with low literacy levels, they can choose voice prompts; for normal users, or users who want to increase the fun during use, they can also choose to use text, animation, and voice prompts simultaneously, and so on.

[0106] The system matches user identity information and lens characteristics in the storage of the near-eye display device. If there is a mismatch in the lens information, it may be due to an incorrect lens installation or the addition of a new lens. The system generates a first prompt message to alert the user. The first prompt message can be used to alert the user whether the lens is installed incorrectly or whether new lens information has been added.

[0107] Optionally, matching defects include: only matching user identity information or lens features in the storage of the near-eye display device, or matching user identity information and lens features in different groups of associated information in the storage of the near-eye display device.

[0108] If only user identity information or only lens features are matched in the storage of the near-eye display device, the lens information matching is considered flawed. If both user identity information and lens features are matched in the storage of the near-eye display device, but they are in different associated information groups, the lens information matching is also considered flawed. For example, if the near-eye display device stores associated information group 1 and associated information group 2, where associated information group 1 includes: user identity information 1, lens feature 1, and lens information 1; and associated information group 2 includes: user identity information 2, lens feature 2, and lens information 2; when user identity information is 1 and lens feature is 2, although user identity information and lens features can be matched, they are in different associated information groups.

[0109] Optionally, the method further includes:

[0110] Based on the user's selection to add lens information, the user's identity information and / or lens features are sent to the service device; the service device is received to provide lens information that matches the user's identity information and / or lens features, and the lens information corresponding to the lens is determined.

[0111] When the initial prompt asks the user whether to add lens information, the user decides whether to add lens information based on whether a new lens has been installed. For example, if the installed lens is the user's new lens, new lens information will be added accordingly. The user can select to add lens information by clicking, swiping, or typing. When the user selects to add lens information, the user's identity information and / or lens characteristics are sent to the service device. The service device queries the lens information that matches the user's identity information and / or lens characteristics based on pre-stored association information and sends it back to this execution device. This execution device receives the lens information that matches the user's identity information and / or lens characteristics from the service device and identifies this lens information as the lens information corresponding to the lens.

[0112] Example 2

[0113] Figure 3 This is a flowchart of a method for acquiring lens information provided in Embodiment 2 of this application. This embodiment is applicable to situations where lens information can be acquired quickly and easily. The method can be executed by a device for acquiring lens information, which can be implemented in hardware and / or software. The device for acquiring lens information can be configured in a near-eye display device.

[0114] like Figure 3As shown, taking user identity information as the basic information as an example, the method includes:

[0115] S101. Obtain user identity information based on the user wearing a near-eye display device with a detachable lens.

[0116] In this embodiment, user identity information can be understood as information used to uniquely identify a user.

[0117] This application embodiment can identify the near-eye display device when the user is wearing it and determine whether a detachable lens is installed; or, when the user is wearing a near-eye display device with a detachable lens installed, the user can manually trigger a button or trigger it in other ways to prompt the execution device that a detachable lens is installed on the near-eye display device.

[0118] When a user wears a near-eye display device with detachable lenses, user identity information can be obtained through one or more methods such as image recognition, fingerprint recognition, and iris recognition. For example, when using one method, the user's identity information is directly determined based on the recognition result. When using multiple methods, the user's identity information can be determined by combining multiple recognition results. For instance, if multiple recognition results correspond to the same user, the user's identity information can be confirmed; if multiple recognition results correspond to different users, the user's identity can be considered to have failed to be recognized. Alternatively, the user whose identity information appears most frequently among the different users corresponding to multiple recognition results can be identified as the recognized user's identity information. Image recognition can utilize the image acquisition device used for eye tracking on the near-eye display device, eliminating the need for an additional image acquisition device.

[0119] Of course, user identity information can also be stored in the user's account information. Account information can be understood as an account that stores relevant user information. The account includes, but is not limited to, the user's identity information, the device information matched with the user, and the relevant lens information. Users can log in to the near-eye display device by manually entering their account information, and the user's account information will be matched to the near-eye display device.

[0120] Optionally, whether a near-eye display device has a removable lens can be detected by at least one of the following methods:

[0121] The distortion in the image captured by the image acquisition device was identified, confirming that a lens was installed;

[0122] The lens is connected to or near the near-eye display device via a mechanical sensor to detect whether a lens is installed.

[0123] The installation lens is determined by sensing changes in light transmittance using a light transmittance sensor.

[0124] The built-in optometry sensor detects the lens thickness or other lens parameters to determine whether to install a lens.

[0125] By illuminating the light source and detecting the reflected light signal, it can be determined that a custom lens has been installed. The edge design of the custom lens can detect materials.

[0126] Optionally, obtaining user identity information includes at least one of the following methods:

[0127] Based on the recognition of the user's iris information, obtain the user's iris information;

[0128] Based on the recognition of user fingerprint information, obtain the user's fingerprint information;

[0129] Based on the recognition of user voice information, obtain the user's voice information;

[0130] Based on the identification of the user's corneal information, obtain the user's corneal information;

[0131] Based on the identification of user capillary information, obtain the user's capillary information;

[0132] The user's identity information is obtained based on the user's manually entered authentication information;

[0133] Establish a connection with the smart device, obtain the identity information in the smart device, and use the identity information as the user's identity information. The identity information includes at least one identity identifier, which may include the user's identity ID information, the user's iris information, the user's fingerprint information, the user's voice information, the user's corneal information, the user's capillary information, etc.

[0134] S102. Match lens information in the storage terminal of the near-eye display device according to the user's identity information.

[0135] In this embodiment, the lens information can be spherical power, cylindrical power, axial power, etc.

[0136] Lens information for different users can be pre-stored in the storage of the near-eye display device, or the corresponding lens information can be stored in the storage of the near-eye display device after a user has used a set of lens information. The near-eye display device stores lens information for different users and matches it against the user's identity information in its storage. If a match is found, the lens information corresponding to the user's identity information is determined; otherwise, the match is considered a failure.

[0137] S103. If the lens information matching fails at the storage end of the near-eye display device based on the user's identity information, the lens information is matched from the service device based on the user's identity information.

[0138] In this embodiment, the service device can be the cloud, a server, a mobile smart device, etc.

[0139] Lens information for different users is pre-stored on the service device. For example, after a user is fitted with lenses, the lens information and user identity information are directly uploaded to the service device for storage. If the user identity information fails to match the lens information in the storage of the near-eye display device, the service device queries for lens information that matches the user identity information. For example, the user identity information is sent to the service device, which queries the corresponding lens information based on the user identity information and sends the results back to this execution device. This execution device receives the lens information and determines the lens information that matches the user identity information.

[0140] In this embodiment, the lens information is determined based on the identification and matching of the user's identity information. Therefore, there is no need to add a separate identification tag or function to the near-eye display device, which can reduce the weight or design difficulty of the near-eye display device.

[0141] S104. Use the lens information to correct the eye-tracking calculation of the near-eye display device, and associate the user identity information and lens information and store them in the storage terminal of the near-eye display device.

[0142] Lens information is used to correct eye-tracking calculations in near-eye display devices, avoiding the impact of lenses on the accuracy of eye-tracking calculations. Introducing lens information into the eye-tracking algorithm corrects the algorithm and ensures the accuracy of eye-tracking calculations. User identity information and lens information are associated and stored in the near-eye display device's storage. The next time the user uses the device, the matching lens information can be directly retrieved from the near-eye display device's storage based on the user's identity information.

[0143] It should be noted that the lens in the embodiments of this application is a vision correction lens.

[0144] This application provides a method for obtaining lens information, solving problems such as the complexity of the lens information acquisition process. The lens information can be pre-stored in the storage of the near-eye display device or in a service device. When a user wears a near-eye display device with a detachable lens, the lens information is matched with the storage of the near-eye display device using the user's identity information. If the matching fails, the lens information is matched with the service device based on the user's identity information, and the user's identity information and the lens information are associated and stored in the storage of the near-eye display device. This allows the lens information to be automatically retrieved from the storage of the near-eye display device the next time it is used. The lens information is then used to correct the eye-tracking calculation of the near-eye display device, improving the accuracy of eye-tracking. The method provided in this application can directly determine the corresponding lens information based on the user's identity information, eliminating the need for the user to input lens information, thus improving the user experience. Furthermore, it does not require changes to the structure and function of the lens and the near-eye display device, does not affect the optical effect of the lens, and does not increase the design difficulty of the near-eye display device.

[0145] Example 3

[0146] Figure 4 This is a flowchart illustrating a method for acquiring lens information according to Embodiment 3 of this application. This embodiment is a refinement based on the above embodiments. Figure 4 As shown, the method includes:

[0147] S201. Obtain user identity information based on a near-eye display device worn by the user and equipped with a detachable lens.

[0148] S202. Match lens information in the storage terminal of the near-eye display device according to the user's identity information.

[0149] After successful or unsuccessful matching of lens information at the storage end of the near-eye display device, different processes are executed respectively: if the matching is successful, S203 is executed; if the matching fails, S204 is executed.

[0150] S203. Based on the user's identity information, the lens information is successfully matched in the storage terminal of the near-eye display device. The matched lens information is used to correct the eye-tracking calculation of the near-eye display device.

[0151] The system queries the storage of the near-eye display device to see if there is matching identity information based on the user's identity information. If so, the lens information corresponding to the matching identity information is used as the matched lens information. At this point, the lens information matching is successful, and the matched lens information is used to correct the eye-tracking calculation of the near-eye display device.

[0152] Optionally, lens information can be matched in the storage of the near-eye display device based on the basic information, including:

[0153] A1. Query the local information of the near-eye display device based on the user's identity information to determine all lens information corresponding to the user's identity information.

[0154] The near-eye display device can locally store identity information and the corresponding lens information. Based on the user's identity information, a query is performed locally on the near-eye display device to determine the identity information that matches the user's identity information. All lens information corresponding to this matching identity information is then identified as the lens information corresponding to the user's identity information, thus obtaining all lens information corresponding to the user's identity information. The number of lens information entries can be one or more.

[0155] A2. If the number of lens information is one, then the lens information is determined as the lens information corresponding to the lens.

[0156] A3. If there are multiple lens information items, display each lens information item to the user and determine the corresponding lens information based on the user's selection operation.

[0157] If there are multiple lens information entries, and it's uncertain which lens is installed in the near-eye display device, the lens information selection interface can be used to display the information to the user. The user can then determine which lens is installed in the near-eye display device. The user determines the unique lens information by making a selection operation on the lens information selection interface. For example, by displaying the lens information on the screen, the user can trigger the selection operation by clicking on options or buttons on the screen, or entering the lens information in the input box. This execution device receives the user's selection operation and determines the lens information corresponding to the lens based on the selection operation. This lens information can be used to correct the eye-tracking calculation of the near-eye display device.

[0158] S204. If the lens information matching in the storage terminal of the near-eye display device fails based on the user's identity information, query the service device based on the user's identity information and receive all lens information corresponding to the user's identity information from the service device.

[0159] The service device queries the user's identity information to determine the lens information and then sends this information back to the execution device. The execution device then uses this information to determine all lens information corresponding to the user's identity information. The service device can either query directly and return lens information matching the user's identity information, or it can return lens information corresponding to all identity information, including the lens information matching the user's identity information.

[0160] When the service device is a mobile storage device, as an optional embodiment, this optional embodiment will further query the service device based on the user's identity information and receive all lens information corresponding to the user's identity information fed back by the service device, including:

[0161] B1. Detect service equipment and establish a connection with the service equipment.

[0162] In this embodiment, a mobile storage device can be understood as a portable device with storage capabilities, such as a smartphone or tablet. When the service device is a mobile storage device, the lens information is obtained through steps B1-B2: detecting whether a service device exists around the execution device; if a service device is detected, establishing a connection with the service device, which can be a wireless connection, such as a Bluetooth connection or a communication connection, or a wired connection via a data cable or power cord.

[0163] B2. Send user identity information to the service device and obtain all lens information corresponding to the user identity information sent by the service device; or, obtain one or more identity information and lens information corresponding to the identity information sent by the service device, and determine all lens information corresponding to the user identity information based on the identity information that matches the user identity information.

[0164] This execution device sends user identity information to the service device. Upon receiving the user identity information, the service device performs lens information matching and sends back all lens information matching the user identity information to this execution device. This execution device then receives all lens information matching the user identity information from the service device. Alternatively, this execution device may not send user identity information to the service device. After establishing a connection with the service device, the service device proactively sends one or more stored identity information entries, along with the corresponding lens information for each identity, to this execution device. This execution device then filters all the received identity information based on the user identity information, determines the identity information matching the user identity, and identifies all lens information corresponding to this matching identity as the lens information corresponding to the user's identity.

[0165] When the service device is in the cloud or on a server, this execution device can communicate directly with the service device. This execution device communicates directly with the service device and obtains all lens information corresponding to the user's identity information through step B2.

[0166] S205. If the number of lens information is one, then the lens information is determined to be the lens information corresponding to the lens.

[0167] S206. If there are multiple lens information items, multiple lens information items will be displayed to the user, and the lens information corresponding to the lens will be determined according to the user's selection operation.

[0168] If there are multiple lens information items, and it is unclear which lens is being used, the user can be shown all the lens information. The user can then select the corresponding lens information from among the available options, such as by displaying the lens information on the screen and the user triggering the selection by clicking on options or buttons on the screen or entering lens information in the input box. The execution device receives the user's selection and determines the corresponding lens information based on the selection.

[0169] S207. Use the lens information to correct the eye-tracking calculation of the near-eye display device, and associate the user identity information and lens information and store them in the storage terminal of the near-eye display device.

[0170] This application provides a method for obtaining lens information, solving problems such as the complexity of the lens information acquisition process. The lens information can be pre-stored in the storage of the near-eye display device or in a service device. When a user wears a near-eye display device with a detachable lens, the lens information is matched against the storage of the near-eye display device using the user's identity information. If the match is successful, the lens information is directly obtained from the storage of the near-eye display device; if the match fails, the lens information is matched against the service device based on the user's identity information, and the user's identity information and lens information are associated and stored in the storage of the near-eye display device. This allows for direct retrieval of the lens information from the storage of the near-eye display device during subsequent use. The method provided in this application can directly determine the corresponding lens information based on the user's identity information, eliminating the need for the user to input lens information, thus improving the user experience. This application can also save multiple lens information entries and maintain multiple lens information entries for a single user. When a user switches between different lenses, the lens information can be quickly matched without repeated information input. Furthermore, it does not require changes to the structure and function of the lens and the near-eye display device, does not affect the optical effect of the lens, and does not increase the design complexity of the near-eye display device.

[0171] This embodiment provides an exemplary method for matching lens information in different scenarios.

[0172] Scenario 1: User 1 installs a lens on a near-eye display device for the first time.

[0173] The near-eye display device detected that a lens was installed inside it;

[0174] The near-eye display device prompts the user to check if their lens is installed correctly. (To prompt the user to check if the installed lens is correct and if it is their own lens, the user needs to confirm that the lens is their own. The device will then assume that the lens is associated with the user.) (Note: The step of prompting the user to confirm whether the lens is installed correctly in subsequent scenarios is optional. Even without prompting information, the user will generally know whether they are wearing the correct lens, so the above prompt process does not need to be set up.)

[0175] The near-eye display device identifies user 1's identity information;

[0176] The near-eye display device cannot match the lens information in the storage of the near-eye display device based on the user 1's identity information;

[0177] The near-eye display device will prompt whether lens information matching is required;

[0178] The near-eye display device performs lens information matching based on user 1's confirmation and sends user 1's identity information to the service device;

[0179] The service device matches lens information based on user 1's identity information and sends the lens information that matches user 1's identity information back to the near-eye display device.

[0180] The lens information matched with user 1's identity information is used as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device;

[0181] At the same time, the near-eye display device associates and stores the user 1's identity information with the lens information matched by the service device in the storage of the near-eye display device.

[0182] If the service device matches multiple lens information based on user 1's identity information (i.e., user 1 has multiple lenses), it will send all the lens information matching user 1's identity information back to the near-eye display device.

[0183] The near-eye display device generates a selection prompt box based on multiple sets of lens information to help the user determine which lens is being used.

[0184] Based on User 1's lens selection, the selected lens information is used as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device;

[0185] At the same time, the near-eye display device associates and stores the user 1's identity information and all lens information matched with the service device in the near-eye display device's storage.

[0186] Scenario 2: User 1 is installing a lens on a near-eye display device for the first time.

[0187] The near-eye display device detected that a lens was installed inside it;

[0188] The near-eye display device prompts user 1 whether they have installed their own lenses;

[0189] The near-eye display device identifies user 1's identity information;

[0190] The near-eye display device matches the lens information in the storage of the near-eye display device based on the user 1's identity information;

[0191] The near-eye display device will match the lens information stored in the near-eye display device as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device.

[0192] Based on user 1's identity information, the near-eye display device matches multiple sets of lens information in the near-eye display device's storage.

[0193] The near-eye display device generates a selection prompt box based on multiple sets of lens information (the prompt box can also be pre-stored after the first generation) to help the user determine which lens is being used.

[0194] Based on user 1's lens selection, the selected lens information is used as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device.

[0195] The following details the different methods of identity verification:

[0196] Scenario 3: When User 1 installs a lens on a near-eye display device for the first time, the user's identity information is identified by matching the proximity of the smart device and the near-eye display device.

[0197] The near-eye display device detected that a lens was installed inside it;

[0198] The near-eye display device prompts user 1 whether they have installed their own lenses;

[0199] The near-eye display device determines that there is no user identity information or associated lens information in the storage of the near-eye display device.

[0200] The near-eye display device prompts the user to match lens information; (for example, by playing an animation showing the user shaking their phone around the near-eye display device).

[0201] The near-eye display device identifies surrounding mobile storage devices; (the user moves the mobile storage device closer to the near-eye display device).

[0202] The near-eye display device has detected a mobile storage device. Confirm the connection?

[0203] A connection is established between the near-eye display device and the mobile storage device based on user confirmation;

[0204] Retrieving lens information from a mobile storage device can be done in the following two ways:

[0205] Method 1: After recognizing the installed lens and the connection with the mobile storage device, the near-eye display device sends information and instructions to the mobile storage device.

[0206] The mobile storage device sends user 1's identity information and related lens information to the near-eye display device; at this time, the mobile storage device acts as the storage carrier (i.e., the service device) for the lens information.

[0207] Method 2: After the near-eye display device and the mobile storage device establish a connection, the mobile storage device receives a signal that a lens is installed inside the near-eye display device. The mobile storage device then actively sends the user 1's identity information and the associated lens information to the near-eye display device.

[0208] After receiving the user 1's identity information and associated lens information, the near-eye display device uses the lens information as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device.

[0209] At the same time, the near-eye display device stores user 1's identity information and associated lens information in the near-eye display device's storage terminal;

[0210] If the near-eye display device receives user 1's identity information and matches multiple sets of lens information, a selection page will be generated.

[0211] The near-eye display device uses the lens information selected by the user as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device.

[0212] Meanwhile, the near-eye display device stores user 1's identity information, associated lens information, and the generated selection interface in its storage.

[0213] Scenario 4: User 1 has installed the lens on the near-eye display device for the first time. User identity information is identified by matching the proximity of the smart device and the near-eye display device.

[0214] The near-eye display device detected that a lens was installed inside it;

[0215] The near-eye display device prompts user 1 whether they have installed their own lenses;

[0216] The near-eye display device determines that its storage contains the user's identity information and associated lens information;

[0217] The near-eye display device generates a selection interface that associates user identity information with lens information; (this selection interface may include user 2 and their lens information, as well as user 1 and their lens information, that is, all information stored in the near-eye display device's storage will be displayed, whether it is the lens information of user 1 or user 2, because there is no user identity recognition process at this time)

[0218] Based on the user's selection of user identity information and lens information displayed on the selection interface, the near-eye display device uses the selected lens information as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device.

[0219] Scenario 5: When User 2 installs the lens on the near-eye display device for the first time, the user's identity information is identified by matching the proximity of the smart device and the near-eye display device.

[0220] The difference between this scenario and scenario 4 above is that;

[0221] The near-eye display device can determine that its storage contains the user's identity information and associated lens information; (at this time, the stored information is the lens information of user 1).

[0222] After the selection interface was displayed on the near-eye display device, User 2 did not find their identity and selected the negative option (e.g., no matching information);

[0223] The near-eye display device prompts user 2 whether they have installed their own lenses based on user 2's negative selection.

[0224] The near-eye display device prompts the user to match lens information; (for example, by playing an animation showing the user shaking their phone around the near-eye display device).

[0225] The near-eye display device identifies surrounding mobile storage devices; (the user moves the mobile storage device closer to the near-eye display device).

[0226] The near-eye display device has detected a mobile storage device. Confirm the connection?

[0227] The next step is the matching process for the user's first lens installation, see scenario 3.

[0228] Scenario 6: User 1 installs a lens on a near-eye display device for the first time. Identification is achieved through automatic recognition such as iris scanning, and lens information is obtained from the service device.

[0229] The near-eye display device detected that a lens was installed inside it;

[0230] The near-eye display device displays a prompt box asking user 1 whether their own lens has been installed;

[0231] The near-eye display device uses iris recognition to identify user 1's identity information;

[0232] The near-eye display device cannot match the lens information in the storage of the near-eye display device based on the user 1's identity information;

[0233] The near-eye display device prompts the user to match lens information; (for example, by playing an animation showing the user shaking their phone around the near-eye display device).

[0234] The near-eye display device identifies surrounding mobile storage devices; (the user moves the mobile storage device closer to the near-eye display device).

[0235] The near-eye display device has detected a mobile storage device. Confirm the connection?

[0236] A connection is established between the near-eye display device and the mobile storage device based on user confirmation;

[0237] The user 1 identity information identified by the near-eye display device is matched with the user identity information on the smart device. The smart device then sends the lens information associated with the user 1 identity information to the near-eye display device.

[0238] The near-eye display device uses lens information as an auxiliary parameter to correct the eye-tracking calculations of the near-eye display device.

[0239] Furthermore, the near-eye display device associates and stores user 1's identity information with the lens information;

[0240] If the near-eye display device receives multiple sets of lens information from user 1, it generates a selection interface for the user to choose from, and uses the selected lens information as auxiliary parameters to correct the eye-tracking calculation of the near-eye display device.

[0241] Furthermore, the near-eye display device associates and stores user 1's identity information with multiple sets of lens information.

[0242] Scenario 7: User 1 has installed the lens on a near-eye display device before. Identification is done through automatic recognition such as iris scanning, and the lens information is also obtained locally.

[0243] The near-eye display device detected that a lens was installed inside it;

[0244] The near-eye display device displays a prompt box asking user 1 whether their own lens has been installed;

[0245] The near-eye display device uses iris recognition to identify user 1's identity information;

[0246] Based on user 1's identity information, the near-eye display device can match lens information in the near-eye display device's storage.

[0247] The near-eye display device uses the lens information matched by the storage terminal of the near-eye display device as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device;

[0248] The near-eye display device matches multiple sets of lens information for user 1 based on the storage of the near-eye display device, and determines the lens information based on the user's selection.

[0249] If User 2 enters at this time, and the lens information cannot be matched with User 2's identity information, then refer to Scenario 6, rematch the lens information for User 2 on the smart device and store it on the near-eye display device's storage.

[0250] In Scenario 7, since the near-eye display device has obtained User 1's identity and lens information before, this information is stored in the near-eye display device's storage. Therefore, it is unnecessary to match the identity information and retrieve the lens information from the service device. In Scenario 7, the service device is not required. In other words, in Scenario 6 and Scenario 7, identity recognition is achieved through the near-eye display device's built-in identity recognition function. The lens information obtained initially needs to be stored on the service device, which only handles identity matching and storing and sending the lens information.

[0251] The solution provided in this embodiment requires the user to ensure that the lens installed on the near-eye display device is their correct lens. This is because the near-eye display device only recognizes that a new lens has been added, but cannot obtain the specific information of the lens. Therefore, each time the near-eye display device recognizes that a lens has been installed, the user needs to be prompted whether the correct lens has been installed. In this way, the lens information matched based solely on the user's identity information will be directly defaulted to the lens information of that lens, and the lens information will be used as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device.

[0252] Of course, it is not necessary to prompt the user to confirm whether the lens is correct every time. If the user puts the wrong lens into the near-eye display device without realizing it and uses the incorrect lens information as an auxiliary parameter to correct the eye tracking calculation of the near-eye display device, then during the subsequent calibration process, when the algorithm finds that the error between the gaze point and the calibration point is too large or abnormal, it is considered that the user may have used the wrong lens information. At this time, the user is prompted to check whether the lens is wrong.

[0253] Example 4

[0254] Figure 5This is a flowchart of a method for acquiring lens information provided in Embodiment 4 of this application. This embodiment is applicable to situations requiring simple and rapid acquisition of lens information. The method can be executed by a device for acquiring lens information, which can be implemented in hardware and / or software. This device can be configured in a near-eye display device. Figure 5 As shown, taking hardware information as the basic information as an example, the method includes:

[0255] S301. Obtain hardware information based on the user's wear of a near-eye display device with a detachable lens.

[0256] In this embodiment, hardware information includes, but is not limited to, one or more of device information, lens features, and account information. Device information can be understood as the serial number of the near-eye display device itself or the serial number of a component within the near-eye display device, such as the serial number of the lens itself. This serial number can be obtained by the near-eye display device based on manual input by the user, or it can be identified by the device. The serial number can be a magnetic signature, optical or infrared mark, such as a quick-response code, barcode, QR code, or alphanumeric code. Lens features can be understood as the characteristics generated by the lens that can be used to compare the lens with other lenses. For example, a distortion image of the lens is obtained through an image acquisition device; different lenses have different distortion images, and the distortion image is a lens feature. The transmittance of the installed lens is sensed by a transmittance sensor; different lenses have different transmittances, and the transmittance is a lens feature. Certain parameters of the lens are sensed by an optometry sensor; different lenses have different parameters, and these parameters are lens features.

[0257] The method for detecting whether a near-eye display device has a detachable lens can be referred to in the above embodiments. Alternatively, determining whether a near-eye display device has a detachable lens can also be based on a user-triggered lens matching information command. In other words, after the user voluntarily installs a detachable lens, the installation of the lens in the near-eye display device can be confirmed based on the user-triggered lens matching information operation. When the user is wearing a near-eye display device with a detachable lens, hardware information is obtained through a pre-determined algorithm and pre-installed equipment.

[0258] S302. Match lens information in the storage terminal of the near-eye display device according to hardware information.

[0259] Lens information and hardware information for different lenses can be pre-stored in the storage of the near-eye display device. Alternatively, after a user has used a set of lens information, the corresponding hardware information and lens information can be stored in the storage of the near-eye display device. The storage of hardware information and lens information for different lenses in the near-eye display device allows for matching based on the hardware information. If a match is found, the corresponding lens information is determined; otherwise, the matching is considered a failure.

[0260] S303, Matching lens information in the storage of the near-eye display device based on hardware information fails. Match lens information from the service device based on hardware information.

[0261] The service device pre-stores hardware and lens information for different lenses. For example, after a user equips a lens, they directly upload the lens and hardware information to the service device for storage. If the hardware information fails to match the lens information in the storage of the near-eye display device, the service device queries for matching lens information. For example, the service device sends the hardware information to the service device, queries the corresponding lens information based on the hardware information, and sends the information back to the execution device. The execution device receives the lens information and determines the lens information that matches the hardware information.

[0262] This application eliminates the need to add a separate identification tag or function to the near-eye display device when determining lens information, thereby reducing the weight or design complexity of the near-eye display device.

[0263] S304. Use the lens information to correct the eye-tracking calculation of the near-eye display device, and associate the hardware information and lens information and store them in the storage of the near-eye display device.

[0264] Lens information is incorporated into the eye-tracking algorithm to correct it and ensure the accuracy of eye-tracking calculations. Hardware and lens information are linked and stored in the near-eye display device's storage. The next time the user uses the device, they can directly retrieve the matching lens information from the near-eye display device's storage based on the hardware information.

[0265] This application provides a method for obtaining lens information, solving problems such as the complexity of the lens information acquisition process. The lens information can be pre-stored in the storage of the near-eye display device or in a service device. When the user wears a near-eye display device with a detachable lens, the lens information is matched with the storage of the near-eye display device based on hardware information. If the matching fails, the lens information is matched with the service device based on the hardware information, and the hardware information and lens information are associated and stored in the storage of the near-eye display device. This allows the lens information to be automatically retrieved from the storage of the near-eye display device the next time it is used. The lens information is then used to correct the eye-tracking calculation of the near-eye display device, improving the accuracy of eye-tracking. The method provided in this application can directly determine the corresponding lens information based on the hardware information, eliminating the need for the user to input lens information, thus improving the user experience. Furthermore, it does not require changes to the structure and function of the lens and the near-eye display device, does not affect the optical effect of the lens, and does not increase the design difficulty of the near-eye display device.

[0266] Optionally, obtaining hardware information includes:

[0267] Hardware information is acquired based on an information recognition device and / or based on manual user input; the information recognition device includes at least one of the following:

[0268] The device includes an image recognition device, a mechanical sensor, a transmittance sensor, an optometry sensor, and a photoelectric sensor. In this embodiment, the information recognition device can be understood as a device that identifies hardware information, and can be one or more of the following: an image acquisition device, a mechanical sensor, a transmittance sensor, an optometry sensor, and a photoelectric sensor. The image recognition device can identify unique features, QR code information, serial numbers, distorted images transmitted through the lens, etc., on the near-eye display device, lens, or other components, and use the identified information as hardware information. The mechanical sensor identifies the lens's serial number, markings, etc., and uses the identified information as hardware information. The transmittance sensor senses the lens's transmittance and uses the transmittance as hardware information. The optometry sensor identifies the lens's thickness, power, etc., and uses the identified information as hardware information. The photoelectric sensor detects the light signal reflected by the lens and uses the position, angle, etc., of the reflected light signal as hardware information. The near-eye display device can also obtain hardware information based on manual user input. For example, based on the user inputting the lens power information 100 degrees, the near-eye display device obtains the lens power information.

[0269] As an optional embodiment, this optional embodiment further includes: successfully matching lens information at the storage end of the near-eye display device based on hardware information, and using the matched lens information to correct the eye-tracking calculation of the near-eye display device.

[0270] Based on the hardware information, the system queries the storage of the near-eye display device to see if there is matching hardware information. If so, the lens information corresponding to the matching hardware information is used as the matched lens information. At this point, the lens information matching is successful, and the matched lens information is used to correct the eye-tracking calculation of the near-eye display device.

[0271] When the service device is a mobile storage device, as an optional embodiment, this optional embodiment further matches lens information from the service device based on hardware information, including:

[0272] C1. Detect service equipment and establish a connection with the service equipment.

[0273] The device detects whether there are any service devices around it. If a service device is detected, it establishes a connection with the service device. This connection can be wireless, such as Bluetooth or a communication connection, or it can be wired via a data cable or power cord.

[0274] C2. Send hardware information to the service device and obtain lens information that matches the hardware information sent by the service device; or, obtain one or more alternative hardware information and lens information corresponding to the alternative hardware information sent by the service device, and determine the lens information of the alternative hardware information that matches the hardware information as the lens information corresponding to the hardware information.

[0275] This execution device sends hardware information to the service device. Upon receiving the hardware information, the service device performs lens information matching and sends the matching lens information back to this execution device. This execution device then receives the matching lens information from the service device. Alternatively, this execution device may not send hardware information to the service device. After establishing a connection, the service device proactively sends one or more stored candidate hardware information items, along with the corresponding lens information for each candidate hardware information item, to this execution device. This execution device then filters the received candidate hardware information based on the hardware information, determines the candidate hardware information that matches the hardware information, and identifies the lens information corresponding to this candidate hardware information as the lens information corresponding to the hardware information.

[0276] When the service device is a cloud-based or server-based device, this execution device can communicate directly with the service device. This execution device communicates directly with the service device and obtains the lens information corresponding to the hardware information through step B2.

[0277] In this embodiment, the hardware information and lens information have a one-to-one correspondence, meaning that one piece of hardware information uniquely corresponds to one piece of lens information. The corresponding lens information can be directly matched based on the hardware information.

[0278] This application provides a method for acquiring lens information, solving problems such as the complexity of the lens information acquisition process. The lens information can be pre-stored in the storage of the near-eye display device or in a service device. When the user wears a near-eye display device with a detachable lens, the lens information is matched with the storage of the near-eye display device through hardware information. If the match is successful, the corresponding lens information is directly obtained from the storage of the near-eye display device. If the match fails, the lens information is matched with the service device based on the hardware information, and the hardware information and lens information are associated and stored in the storage of the near-eye display device. This allows the lens information to be directly obtained from the storage of the near-eye display device for the next use. The lens information is used to correct the eye-tracking calculation of the near-eye display device, improving the accuracy of eye-tracking. The unique lens information can be determined based on the hardware information, eliminating the need for the user to input the lens information. The entire process requires no user operation and can be completed seamlessly, improving the user experience. Furthermore, it does not require changes to the structure and function of the lens and the near-eye display device, does not affect the optical effect of the lens, and does not increase the design difficulty of the near-eye display device.

[0279] Example 5

[0280] Figure 6 This is a flowchart of a method for acquiring lens information provided in Embodiment 5 of this application. This embodiment is applicable to situations requiring simple and rapid acquisition of lens information. The method can be executed by a device for acquiring lens information, which can be implemented in hardware and / or software. This device can be configured in a near-eye display device. Figure 6 As shown, the method includes:

[0281] S401. Based on a near-eye display device worn by a user and equipped with a detachable lens, obtain user identity information and lens characteristics.

[0282] The method for determining whether a near-eye display device has a detachable lens can refer to the above embodiments. When a user wears a near-eye display device with a detachable lens, user identity information is obtained through one or more methods such as image recognition, fingerprint recognition, and iris recognition. Lens features are then obtained through a pre-determined algorithm and pre-installed equipment. Specific user identity information recognition and lens feature recognition methods can be found in the recognition methods described in the above embodiments.

[0283] S402. Match lens information in the storage terminal of the near-eye display device based on user identity information and lens characteristics.

[0284] A set of data containing user identity information, lens characteristics, and lens information can be pre-stored in the storage terminal of the near-eye display device. Alternatively, after a user has used a set of lens information, a corresponding set of data containing user identity information, lens characteristics, and lens information can be stored in the storage terminal of the near-eye display device. Different sets of user identity information, lens characteristics, and lens information are stored in the storage terminal of the near-eye display device. A match is then performed based on the user identity information and lens characteristics in the storage terminal of the near-eye display device to determine if a successful match can be achieved.

[0285] S403. The lens information matching at the storage end of the near-eye display device failed based on the user identity information and lens characteristics. The lens information is matched from the service device based on the user identity information and lens characteristics.

[0286] The service device pre-stores user identity information, lens features, and lens information corresponding to different lenses. For example, after a user is equipped with lenses, they directly upload their identity information, lens information, and lens features to the service device for storage. If the user identity information and lens features fail to match the lens information in the storage of the near-eye display device, the service device queries for lens information that matches the user identity information and lens features. For example, the service device sends the user identity information and lens features to the service device, which retrieves the corresponding lens information based on the user identity information and lens features and sends it back to the execution device. The execution device receives the lens information and determines the lens information that matches the user identity information and lens features.

[0287] This application eliminates the need to add a separate identification tag device or function to the near-eye display device when determining lens information, thereby reducing the weight or design complexity of the near-eye display device. It also eliminates the need for users to manually operate and control the near-eye display device to identify lens information, reducing the complexity of user operation and improving the user experience.

[0288] S404. Use the lens information to correct the eye-tracking calculation of the near-eye display device, and associate the user identity information and lens features with the lens information and store them in the storage terminal of the near-eye display device.

[0289] Lens information is incorporated into the eye-tracking algorithm to correct it and ensure the accuracy of eye-tracking calculations. User identity information and lens features are associated and stored in the near-eye display device's storage. The next time the user uses the device, they can directly retrieve the matching lens information from the near-eye display device's storage based on their identity information and lens features.

[0290] This application provides a method for obtaining lens information, solving problems such as the complexity of the lens information acquisition process. The lens information can be pre-stored in the storage of the near-eye display device or in the service device. When a user wears a near-eye display device with a detachable lens, the lens information is matched with the user's identity information and lens characteristics in the storage of the near-eye display device. If the matching fails, the lens information is matched with the user's identity information and lens characteristics from the service device. The user's identity information, lens characteristics, and lens information are then stored together in the storage of the near-eye display device so that the lens information can be automatically retrieved from the storage of the near-eye display device the next time it is used. The lens information is then used to correct the eye-tracking calculation of the near-eye display device, improving the accuracy of eye-tracking. The method provided in this application can directly determine the lens information of the lens corresponding to the user based on the user's identity information and lens characteristics, without requiring the user to enter lens information, thus improving the user experience. Furthermore, it does not require changes to the structure and function of the lens and the near-eye display device, does not affect the optical effect of the lens, and does not increase the design difficulty of the near-eye display device. Furthermore, the near-eye display can simultaneously store the user's identity information and lens characteristics. When a user wears a near-eye display device with detachable lenses, the device automatically identifies whether the installed lens matches the user by matching the user's identity information and lens characteristics with the corresponding lens information. In other words, even if the user accidentally installs the wrong lens or fails to remove a lens that is not theirs in time, the near-eye display device can automatically identify the mismatch between the lens and the user, improving the intelligence of the near-eye display device in matching lens information for the user.

[0291] Example 6

[0292] Figure 7 This is a flowchart illustrating a method for acquiring lens information according to Embodiment Six of this application. This embodiment is a refinement based on the above embodiments. Figure 7 As shown, taking the acquisition of user identity information and lens features as basic information as an example, the method includes:

[0293] S501. Based on a near-eye display device worn by a user and equipped with a detachable lens, obtain user identity information and lens characteristics.

[0294] Optionally, obtaining lens features includes:

[0295] Lens features are obtained based on information recognition devices and / or based on manual user input;

[0296] Information identification devices include at least one of the following:

[0297] Image recognition devices, mechanical sensors, light transmittance sensors, optometry sensors, and photoelectric sensors.

[0298] For details on how to obtain lens features using information recognition devices, please refer to the relevant descriptions in the above embodiments. Near-eye display devices can also obtain lens features based on manual user input. For example, based on user input of a lens power of 100 degrees, the near-eye display device can obtain the lens power information.

[0299] S502. Match lens information in the storage terminal of the near-eye display device based on user identity information and lens characteristics.

[0300] S503. Based on user identity information and lens characteristics, the lens information is successfully matched at the storage end of the near-eye display device. The matched lens information is used to correct the eye-tracking calculation of the near-eye display device.

[0301] Among them, successful matching of lens information means that the user's identity information and the lens features are in a set of related information and can be matched simultaneously.

[0302] Based on user identity information and lens features, lens information is matched in the storage of the near-eye display device. The storage of the near-eye display device typically stores different sets of associated information, each set including identity information, lens features, and lens information. The system queries the associated information stored in the near-eye display device's storage based on user identity information and lens features. If both user identity information and lens features are matched and within the same set of associated information, the lens information matching is considered successful. The lens information associated with the user identity information and lens features is then identified as the matched lens information and used to correct the eye-tracking calculations of the near-eye display device.

[0303] S504. If the lens information matching fails at the storage end of the near-eye display device based on the user identity information and lens characteristics, query the service device according to the user identity information and lens characteristics, and receive the lens information that matches the user identity information and lens characteristics from the service device.

[0304] Among them, failure to match lens information means that no user identity information or lens features can be matched in each set of associated information.

[0305] When querying the storage of a near-eye display device, if neither the user's identity information nor the lens characteristics are matched, the lens information matching is deemed a failure. User identity information, lens characteristics, and lens information are pre-stored on the service device side. The service device is queried based on the user's identity information and lens characteristics. The service device then feeds back the lens information matching the user's identity information and lens characteristics to this execution device. This execution device receives the lens information matching the user's identity information and lens characteristics fed back by the service device.

[0306] When the service device is a mobile storage device, as an optional embodiment, this optional embodiment further queries the service device based on the user's identity information and lens features, and receives lens information from the service device that matches the user's identity information and lens features, including:

[0307] D1. Detect service equipment and establish a connection with the service equipment.

[0308] The device detects whether there are any service devices around it. If a service device is detected, it establishes a connection with the service device. This connection can be wireless, such as Bluetooth or a communication connection, or it can be wired via a data cable or power cord.

[0309] D2. Send user identity information and lens features to the service device, and obtain lens information corresponding to the user identity information and lens features sent by the service device; or, obtain one or more sets of association information sent by the service device, determine the target association information that matches the user identity information and lens information, and determine the lens information in the target association information as the lens information corresponding to the user identity information and lens features.

[0310] In this embodiment, the target association information can be understood as the association information obtained by filtering from one or more sets of association information.

[0311] The execution device sends user identity information and lens features to the service device. Upon receiving the user identity information and lens features, the service device performs lens information matching and sends back the lens information that matches both the user identity information and lens features and falls within a set of associated information. The execution device then receives the lens information corresponding to the user identity information and lens features from the service device. Alternatively, the execution device may not send the user identity information and lens features to the service device. After establishing a connection with the service device, the service device proactively sends one or more sets of associated information to the execution device. The execution device matches the received sets of associated information based on the user identity information and lens features, determines the associated information that matches the user identity information and lens features, uses this associated information as the target associated information, and identifies the lens information within the target associated information. This lens information is the lens information corresponding to the user identity information and lens features.

[0312] When the service device is in the cloud or on a server, this execution device can communicate directly with the service device. This execution device communicates directly with the service device and obtains all lens information corresponding to the user's identity information through step D2.

[0313] S505. Use the lens information to correct the eye-tracking calculation of the near-eye display device, and associate the user identity information and lens features with the lens information and store them in the storage terminal of the near-eye display device.

[0314] S506. Based on the user's identity information and lens characteristics, if there is a defect in the lens information matching at the storage end of the near-eye display device, generate a first prompt message to prompt the user. The first prompt message includes prompting the user whether the lens is installed incorrectly or prompting the user whether to add lens information.

[0315] In this embodiment, the first prompt information can be understood as information used to prompt the user, which can be text, images, voice, etc., or a combination of multiple methods. For example, playing an animation while simultaneously playing a voice prompt and displaying the corresponding text. The prompting method of the first prompt information can be preset, or multiple prompting methods can be set for the user to choose from. For example, for users with hearing impairments, they can choose images, animations, text, etc., and the user can determine the content of the prompt by viewing the images, animations, text, etc.; for users who cannot read, such as young children, users with poor eyesight, or users with low literacy levels, they can choose voice prompts; for normal users, or users who want to increase the fun during use, they can also choose to use text, animation, and voice prompts simultaneously, and so on.

[0316] The system matches user identity information and lens characteristics in the storage of the near-eye display device. If there is a mismatch in the lens information, it may be due to an incorrect lens installation or the addition of a new lens. The system generates a first prompt message to alert the user. The first prompt message can be used to alert the user whether the lens is installed incorrectly or whether new lens information has been added.

[0317] Optionally, matching defects include: only matching user identity information or lens features in the storage of the near-eye display device, or matching user identity information and lens features in different groups of associated information in the storage of the near-eye display device.

[0318] If only user identity information or only lens features are matched in the storage of the near-eye display device, the lens information matching is considered flawed. If both user identity information and lens features are matched in the storage of the near-eye display device, but they are in different associated information groups, the lens information matching is also considered flawed. For example, if the near-eye display device stores associated information group 1 and associated information group 2, where associated information group 1 includes: user identity information 1, lens feature 1, and lens information 1; and associated information group 2 includes: user identity information 2, lens feature 2, and lens information 2; when user identity information is 1 and lens feature is 2, although user identity information and lens features can be matched, they are in different associated information groups.

[0319] As an optional embodiment, this optional embodiment further includes: sending user identity information and / or lens features to the service device based on the user's selection to add lens information; receiving lens information that matches the user identity information and / or lens features from the service device, and determining the lens information corresponding to the lens.

[0320] When the initial prompt asks the user whether to add lens information, the user decides whether to add lens information based on whether a new lens has been installed. For example, if the installed lens is the user's new lens, new lens information will be added accordingly. The user can select to add lens information by clicking, swiping, or typing. When the user selects to add lens information, the user's identity information and / or lens characteristics are sent to the service device. The service device queries the lens information that matches the user's identity information and / or lens characteristics based on pre-stored association information and sends it back to this execution device. This execution device receives the lens information that matches the user's identity information and / or lens characteristics from the service device and identifies this lens information as the lens information corresponding to the lens.

[0321] This application provides a method for acquiring lens information, solving problems such as the complexity of the lens information acquisition process. Lens information and user identity information can be pre-stored in the storage of the near-eye display device or in a service device. When a user wears a near-eye display device with a detachable lens, the lens information is matched against the storage of the near-eye display device using lens features and user identity information. If the match is successful, the corresponding lens information is directly retrieved from the storage of the near-eye display device. If the match fails, the user's lens information is directly matched from the service device based on the lens features and user identity information. The user identity information, lens features, and lens information are then associated and stored in the storage of the near-eye display device. This allows for direct retrieval of lens information from the storage of the near-eye display device during subsequent use. The lens information is used to correct the eye-tracking calculations of the near-eye display device, improving eye-tracking accuracy. The lens features and user identity information help the user uniquely identify the corresponding lens information, eliminating the need for user input. The entire process requires no user intervention, allowing for seamless acquisition of lens information and improving user experience. Furthermore, it does not alter the structure or function of the lens or the near-eye display device, does not affect the optical performance of the lens, and does not increase the design complexity of the near-eye display device.

[0322] This embodiment provides an exemplary method for matching lens information in different scenarios.

[0323] Scenario 1: User 1 installs lens 1 on a near-eye display device for the first time, and the lens information is obtained based on the service device.

[0324] The near-eye display device detected that a lens was installed inside it;

[0325] The near-eye display device identifies the lens characteristics of lens 1;

[0326] The near-eye display device identifies user 1's identity information;

[0327] The near-eye display device cannot match user 1's identity information or lens features of lens 1 in the device's storage.

[0328] The near-eye display device prompts the user to confirm whether they are installing or matching their lenses.

[0329] Based on User 1's selection, the near-eye display device sends User 1's identity information and lens characteristics to the service device to match its lens information;

[0330] The near-eye display device receives lens information sent by the service device;

[0331] The near-eye display device uses this set of lens information as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device.

[0332] At the same time, the near-eye display device stores the user 1's identity information, the lens information of lens 1, and the lens features of lens 1 in the near-eye display device's storage as a set of associated information.

[0333] Scenario 2: User 1 is installing lens 1 on a near-eye display device for the first time.

[0334] The near-eye display device detected that a lens was installed inside it;

[0335] The near-eye display device identifies the lens characteristics of lens 1;

[0336] The near-eye display device identifies user 1's identity information;

[0337] The near-eye display device matches user 1's identity information and lens 1's lens features in the same group of associated information in the device's storage.

[0338] The near-eye display device uses the lens information of lens 1 within this set of associated information as an auxiliary parameter to correct the eye-tracking calculation of the near-eye display device;

[0339] Scenario 3: User 1 installs lens 2 on a near-eye display device for the first time, and the lens information is obtained based on the service device.

[0340] The near-eye display device detected that a lens was installed inside it;

[0341] The near-eye display device identifies the lens characteristics of lens 2;

[0342] The near-eye display device identifies user 1's identity information;

[0343] The near-eye display device only matched the identity information of user 1 in the device storage, but did not match the lens feature information of lens 2;

[0344] Generate the first prompt message to indicate to the user whether they need to add a new lens or if the lens is installed incorrectly.

[0345] Based on the user's choice to add their own new lens information, the near-eye display device sends the user's identity information to the service device to match the lens information;

[0346] The near-eye display device receives lens information (at this time, there should be two sets of lens information: one set of information for lens 1 and one set of information for the newly added lens 2), and generates a selection interface. Based on the lens information selected by the user, it determines the lens information as lens 2. (Of course, the near-eye display device can also automatically use the elimination method here, because user 1 already has one set of lens information for lens 1, and by comparison, it can be known that the other set is the lens information for lens 2. If more than two sets of lens information are sent back, the user needs to make a selection manually.)

[0347] The lens information of lens 2 is used as an auxiliary parameter to correct the eye tracking calculation of the near-eye display device.

[0348] At the same time, the near-eye display device stores the user 1's identity information, the lens information of lens 2, and the lens features of lens 2 in the near-eye display device's storage as a set of associated information.

[0349] Specific

[0350] Scenario 4: When User 1 installs Lens 1 on a near-eye display device for the first time, identity recognition is achieved through automatic identification such as iris scanning, and lens information is obtained based on the service device.

[0351] The near-eye display device detected that a lens was installed inside it;

[0352] The near-eye display device identifies the lens characteristics of lens 1;

[0353] The near-eye display device identifies user 1's identity information;

[0354] The near-eye display device cannot match user 1's identity information or lens features of lens 1 in the device's storage.

[0355] The near-eye display device prompts the user to confirm whether they are installing or matching their lenses.

[0356] Based on User 1's selection to match a new lens, the near-eye display device prompts User 1 to match lens information; (for example, playing an animation showing the phone being shaken around the near-eye display device).

[0357] The near-eye display device identifies surrounding moving storage devices; (User 1 moves the smart device close to the near-eye display device).

[0358] The near-eye display device has detected a mobile storage device. Confirm the connection?

[0359] Based on user 1's confirmation, a connection is established between the near-eye display device and the mobile storage device;

[0360] The user 1 identity information identified by the near-eye display device is matched with the user 1 identity information on the mobile storage device, and the mobile storage device sends the lens information associated with the user 1 identity information to the near-eye display device.

[0361] The near-eye display device uses lens information as an auxiliary parameter to correct the eye-tracking calculations of the near-eye display device.

[0362] Furthermore, the near-eye display device stores the user 1's identity information, lens 1's information, and lens 1's lens features together in the near-eye display device's storage as a set of associated information.

[0363] If the near-eye display device receives multiple sets of lens information from user 1, it generates a selection interface for the user to choose from, and uses the selected lens information as auxiliary parameters to correct the eye-tracking calculation of the near-eye display device.

[0364] Furthermore, the near-eye display device stores the user 1's identity information, lens information of lens 1, and lens features of lens 1 together as a set of associated information. (Even if multiple sets of lens information are sent, only the set selected by the user as the information for lens 1 is stored, because the other set of lens information does not yet have matching lens features. Of course, in another case, if the service device has already stored the user's identity information, lens information, and lens features, it can also associate and store the other set of lens features and lens information with the user's identity information.)

[0365] Scenario 5: User 1 installs lens 2 on a near-eye display device for the first time. Identification is achieved through automatic recognition such as iris scanning, and lens information is obtained from the service device.

[0366] The near-eye display device detected that a lens was installed inside it;

[0367] The near-eye display device identifies the lens characteristics of lens 2;

[0368] The near-eye display device automatically identifies user 1's identity information (iris);

[0369] The near-eye display device only matched the identity information of user 1 in the device storage, but did not match the lens features of lens 2;

[0370] Generate the first prompt message to indicate to the user whether they need to add a new lens or if the lens is installed incorrectly.

[0371] Based on the user's need to add their own new lens selection, the near-eye display device prompts the user to match lens information; (for example, playing an animation in which the phone is shaken around the near-eye display device);

[0372] The subsequent operations are the same as those after matching the mobile storage terminal in scenario 4. Finally, the near-eye display device will store the user 1's identity information, the lens information of lens 2, and the lens features of lens 2 in the near-eye display device's storage terminal as a set of associated information.

[0373] The advantage of this embodiment lies in its ability to record the lens characteristics of each installed lens. Only during the initial information matching between the user and the lens does the user need to accurately identify that the lens belongs to them. After the lens characteristics are recorded, the near-eye display device can help the user remember whether the lens belongs to the currently used user, eliminating the need to prompt the user to confirm the lens's ownership each time. If the user mixes up the lenses and incorrectly identifies a lens as their own, the near-eye display device can only match lens information based on the user's identity information, which can easily result in the matched lens information not being the correct one. This solution effectively solves this problem. It helps the user determine whether the current lens belongs to them and promptly prompts the user to replace the lens if it might not be theirs.

[0374] Example 7

[0375] Figure 8 This is a schematic diagram of a device for acquiring lens information provided in Embodiment 7 of this application. Figure 8 As shown, the device includes: a basic information acquisition module 61, a basic information matching module 62, a line-of-sight direction calculation module 63, and a model correction module 64.

[0376] The basic information acquisition module 61 is used to acquire basic information based on the near-eye display device worn by the user and equipped with a detachable lens;

[0377] The basic information matching module 62 is used to match lens information based on the basic information;

[0378] The gaze direction calculation module 63 is used to determine the gaze direction through the lens based on the lens information and the initial eye-tracking calculation model;

[0379] The model correction module 64 is used to correct the eye-tracking calculation model of the near-eye display device according to the line of sight through the lens.

[0380] The device for acquiring lens information provided in this application embodiment enables near-eye display devices to acquire lens information in a timely manner. This lens information is then used to correct the eye-tracking calculations of the near-eye display device, avoiding any impact of the lens on the accuracy of the eye-tracking calculations. The device can store lens information in association, facilitating its reuse while ensuring convenient and quick acquisition. Lens information can be pre-stored in the storage or service device of the near-eye display device. Basic information and lens information are stored together. When a user adds a lens to the near-eye display, the lens information is directly acquired and used to correct the eye-tracking calculations of the near-eye display device, improving eye-tracking accuracy. The method provided in this application embodiment can directly determine the corresponding lens information based on the basic information, and the lens information is automatically entered, improving the user experience. Furthermore, it does not require changing or adding to the structure and function of the lens or the near-eye display device itself, does not affect the optical effect of the lens, and does not increase the design complexity of the near-eye display device.

[0381] Optionally, the basic information acquisition module 61 includes:

[0382] The first basic information acquisition unit is used to acquire basic information based on user manual input and / or manual selection.

[0383] The second basic information acquisition unit is used to acquire basic information based on the recognition of near-eye display devices.

[0384] The basic information includes one or more of user identity information, account information, and hardware information. The hardware information includes one or more of device information and lens characteristics. The lens characteristics include one or more of lens power information, diopter, refractive index, and curvature.

[0385] Optionally, the basic information matching module 62 is specifically used to: match lens information in the storage terminal of the near-eye display device according to the basic information.

[0386] Optionally, the basic information matching module 62 is specifically used to: match lens information from the service device based on the basic information.

[0387] Optionally, the basic information matching module 62 is specifically used to: determine whether there is matching lens information in the storage terminal of the near-eye display device; if so, the lens information matching is successful; if not, the lens information matching fails, then match the lens information from the service device according to the basic information, and associate the basic information and the lens information in the storage terminal of the near-eye display device.

[0388] Optionally, matching lens information in the storage of the near-eye display device based on the basic information includes: querying the local storage of the near-eye display device based on the basic information to determine all lens information corresponding to the basic information; if the number of lens information is one, then determining the lens information as the lens information corresponding to the lens; if the number of lens information is multiple, then displaying each of the lens information to the user, and determining the lens information corresponding to the lens based on the user's selection operation.

[0389] Optionally, matching lens information from the service device based on the basic information includes: querying the service device based on the basic information, receiving all lens information corresponding to the user identity information fed back by the service device; if the number of lens information is one, then the lens information is determined as the lens information corresponding to the lens; if the number of lens information is multiple, then multiple lens information will be displayed to the user, and the lens information corresponding to the lens will be determined according to the user's selection operation.

[0390] Optionally, when the service device is a mobile storage device, the step of querying the service device based on the basic information and receiving all lens information corresponding to the user identity information fed back by the service device includes: detecting the service device and establishing a connection with the service device; sending the basic information to the service device and obtaining all lens information corresponding to the basic information sent by the service device; or, obtaining one or more pieces of basic information sent by the service device and the lens information corresponding to the basic information.

[0391] Optionally, the model correction module 64 is specifically used for: determining the virtual image position based on the lens information and the eyepiece side information of the display system; determining a deviation function based on the virtual image position and the screen position; and correcting the line of sight through the lens based on the deviation function to obtain a corrected eye-tracking calculation model.

[0392] Optionally, the model correction module 64 is specifically used for: calculating several sets of gaze positions with corresponding true values ​​based on the gaze direction through the lens; determining the mapping relationship between the several sets of gaze directions and the corresponding true values; and correcting the gaze direction through the lens based on the mapping relationship to obtain the final eye-tracking calculation model.

[0393] Optionally, when the basic information is user identity information and lens features, a successful match of lens information means that the user identity information and lens features can be matched simultaneously within the same set of associated information, while a failure to match lens information means that neither the user identity information nor the lens features can be matched within each set of associated information.

[0394] Optionally, the device may also include:

[0395] The prompting module is used to generate a first prompting message to the user based on the user's identity information and lens characteristics to match lens information defects in the storage terminal of the near-eye display device. The first prompting message includes prompting the user whether the lens is installed incorrectly or prompting the user whether to add lens information.

[0396] Optionally, matching defects include: only matching user identity information or lens features in the storage of the near-eye display device, or matching user identity information and lens features in different groups of associated information in the storage of the near-eye display device.

[0397] Optionally, the device may also include:

[0398] The first information sending module is used to send the user identity information and / or lens features to the service device based on the user's selection to add lens information;

[0399] The first lens information receiving module is used to receive lens information fed back by the service device that matches the user identity information and / or lens features, and to determine the lens information corresponding to the lens.

[0400] The apparatus for acquiring lens information provided in this application can execute the method for acquiring lens information provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method.

[0401] Example 8

[0402] Figure 9 This is a schematic diagram of a device for acquiring lens information provided in Embodiment 8 of this application. Figure 9 As shown, the device includes: a user identification module 71, a first lens matching module 72, a first information matching failure module 73, and a first storage module 74.

[0403] User identification module 71 is used to obtain user identity information based on the user wearing a near-eye display device with a detachable lens;

[0404] The first lens matching module 72 is used to match lens information in the storage terminal of the near-eye display device according to the user identity information;

[0405] The first information matching failure module 73 is used to match lens information from the service device based on the user identity information if the lens information matching fails at the storage end of the near-eye display device.

[0406] The first storage module 74 is used to use the lens information to correct the eye-tracking calculation of the near-eye display device, and to associate the user identity information and the lens information in the storage terminal of the near-eye display device.

[0407] This application provides a device for acquiring lens information, solving problems such as the complexity of the lens information acquisition process. The lens information can be pre-stored in the storage terminal of the near-eye display device or in a service device. When a user wears a near-eye display device with a detachable lens, the lens information is matched with the storage terminal of the near-eye display device using the user's identity information. If the matching fails, the lens information is matched with the service device based on the user's identity information, and the user's identity information and the lens information are associated and stored in the storage terminal of the near-eye display device. This allows the lens information to be automatically retrieved from the storage terminal of the near-eye display device the next time it is used. The lens information is then used to correct the eye-tracking calculation of the near-eye display device, improving the accuracy of eye-tracking. The method provided in this application can directly determine the corresponding lens information based on the user's identity information, eliminating the need for the user to input lens information, thus improving the user experience. It also eliminates the need to set physical information storage tags on the lens or other locations, avoiding the loss or damage of physical information tags. Furthermore, it does not require changes to the structure and function of the lens and the near-eye display device, does not affect the optical effect of the lens, and does not increase the design difficulty of the near-eye display device.

[0408] Optionally, the device further includes:

[0409] The first information matching success module is used to successfully match lens information in the storage terminal of the near-eye display device based on the user's identity information, and to use the matched lens information to correct the eye-tracking calculation of the near-eye display device.

[0410] Optionally, the first information matching success module is specifically used to: query the local storage of the near-eye display device based on the user identity information, and determine all lens information corresponding to the user identity information; if the number of lens information is one, then use the lens information to correct the eye-tracking calculation of the near-eye display device; if the number of lens information is multiple, then display each of the lens information to the user, determine the lens information corresponding to the lens based on the user's selection operation, and use the lens information corresponding to the lens to correct the eye-tracking calculation of the near-eye display device.

[0411] Optionally, the first information matching failure module includes:

[0412] The service device query unit is used to query the service device based on the user identity information and receive all lens information corresponding to the user identity information fed back by the service device;

[0413] The lens information determination unit is used to determine the lens information as the lens information corresponding to the lens if the number of lens information is one; if the number of lens information is multiple, multiple lens information will be displayed to the user, and the lens information corresponding to the lens will be determined according to the user's selection operation.

[0414] Optionally, when the service device is a mobile storage terminal, the service device query unit is specifically used for: detecting the service device and establishing a connection with the service device; sending the user identity information to the service device and obtaining all lens information corresponding to the user identity information sent by the service device; or, obtaining one or more identity information sent by the service device and the lens information corresponding to the identity information, and determining all lens information corresponding to the user identity information based on the identity information matching the user identity information.

[0415] The apparatus for acquiring lens information provided in this application can execute the method for acquiring lens information provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method.

[0416] Example 9

[0417] Figure 10 This is a schematic diagram of a device for acquiring lens information provided in Embodiment 9 of this application. Figure 10 As shown, the device includes: a lens feature acquisition module 81, a second lens matching module 82, a second information matching failure module 83, and a second storage module 84.

[0418] The lens feature acquisition module 81 is used to acquire lens features based on a near-eye display device worn by a user and equipped with a detachable lens;

[0419] The second lens matching module 82 is used to match lens information in the storage terminal of the near-eye display device according to the lens characteristics;

[0420] The second information matching failure module 83 is used to match lens information from the service device when the matching of lens information fails at the storage end of the near-eye display device based on the lens characteristics.

[0421] The second storage module 84 is used to use the lens information to correct the eye-tracking calculation of the near-eye display device, and to associate the lens features and the lens information in the storage terminal of the near-eye display device.

[0422] This application provides a device for acquiring lens information, solving problems such as the complexity of the lens information acquisition process. The lens information can be pre-stored in the storage of the near-eye display device or in a service device. When a user wears a near-eye display device with a detachable lens, the lens information is matched with the storage of the near-eye display device based on the lens features. If the matching fails, the lens information is matched with the service device based on the lens features, and the lens features and lens information are associated and stored in the storage of the near-eye display device. This allows the lens information to be automatically retrieved from the storage of the near-eye display device the next time it is used. The lens information is then used to correct the eye-tracking calculation of the near-eye display device, improving the accuracy of eye-tracking. The method provided in this application can directly determine the corresponding lens information based on the lens features, eliminating the need for the user to input lens information, thus improving the user experience. It also eliminates the need to set physical information storage tags on the lens or other locations, avoiding the loss or damage of physical information tags. Furthermore, it does not require changes to the structure and function of the lens and the near-eye display device, does not affect the optical effect of the lens, and does not increase the design difficulty of the near-eye display device.

[0423] Optionally, the lens feature acquisition module 81 is specifically used for: acquiring lens features based on an information recognition device and / or acquiring lens features based on manual user input;

[0424] The information identification device includes at least one of the following:

[0425] Image recognition devices, mechanical sensors, light transmittance sensors, optometry sensors, and photoelectric sensors.

[0426] Optionally, the device may also include:

[0427] The second information matching success module is used to successfully match lens information in the storage terminal of the near-eye display device based on lens characteristics, and to use the matched lens information to correct the eye-tracking calculation of the near-eye display device.

[0428] Optionally, when the service device is a mobile storage device, the second information matching failure module 83 is specifically used to: detect the service device and establish a connection with the service device; send the lens feature to the service device and obtain lens information sent by the service device that matches the lens feature; or, obtain one or more alternative lens features sent by the service device and the lens information corresponding to the alternative lens features, and determine the lens information of the alternative lens feature that matches the lens feature as the lens information corresponding to the lens feature.

[0429] The apparatus for acquiring lens information provided in this application can execute the method for acquiring lens information provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method.

[0430] Example 10

[0431] Figure 11 This is a schematic diagram of a device for acquiring lens information provided in Embodiment 10 of this application. Figure 11 As shown, the device includes: a user information and feature acquisition module 91, a third lens matching module 92, a third information matching failure module 93, and a third storage module 94.

[0432] User information and feature acquisition module 91 is used to acquire user identity information and lens features based on the user wearing a near-eye display device with a detachable lens installed;

[0433] The third lens matching module 92 is used to match lens information in the storage terminal of the near-eye display device according to the user identity information and lens characteristics.

[0434] The third information matching failure module 93 is used to match lens information from the service device based on the user identity information and lens characteristics when the lens information matching fails at the storage end of the near-eye display device.

[0435] The third storage module 94 is used to use the lens information to correct the eye-tracking calculation of the near-eye display device, and to associate the user identity information and lens features with the lens information and store them in the storage terminal of the near-eye display device.

[0436] This application provides a device for acquiring lens information, solving problems such as the complexity of the lens information acquisition process. The lens information can be pre-stored in the storage of the near-eye display device or in a service device. When a user wears a near-eye display device with a detachable lens, the lens information is matched with the user's identity information and lens characteristics in the storage of the near-eye display device. If the matching fails, the lens information is matched with the user's identity information and lens characteristics from the service device. The user's identity information, lens characteristics, and lens information are then associated and stored in the storage of the near-eye display device so that the lens information can be automatically retrieved from the storage of the near-eye display device the next time it is used. The lens information is used to correct the eye-tracking calculation of the near-eye display device, improving the accuracy of eye-tracking. The method provided in this application can directly determine the lens information of the lens corresponding to the user based on the user's identity information and lens characteristics, without requiring the user to enter the lens information, thus improving the user experience. It also eliminates the need to set physical information storage tags on the lens or other locations, avoiding the loss or damage of physical information tags. Furthermore, it does not change the structure and function of the lens and the near-eye display device, does not affect the optical effect of the lens, and does not increase the design difficulty of the near-eye display device.

[0437] Optionally, the user information and feature acquisition module 91 is specifically used for: acquiring lens features based on the information recognition device;

[0438] The information identification device includes at least one of the following:

[0439] Image recognition devices, mechanical sensors, light transmittance sensors, optometry sensors, and photoelectric sensors.

[0440] The user manually inputs lens features and user identity information.

[0441] Optionally, the device may also include:

[0442] The third information matching success module is used to successfully match lens information in the storage terminal of the near-eye display device based on user identity information and lens characteristics, and use the matched lens information to correct the eye tracking calculation of the near-eye display device;

[0443] Among them, successful matching of lens information means that the user identity information and lens features are in a set of related information and can be matched simultaneously.

[0444] Optionally, the third information matching failure module 93 is specifically used to: query the service device based on the user identity information and lens features, and receive lens information from the service device that matches the user identity information and lens features.

[0445] Optionally, when the service device is a mobile storage device, the step of querying the service device based on the user identity information and lens features, and receiving lens information from the service device that matches the user identity information and lens features, includes: detecting the service device and establishing a connection with the service device; sending the user identity information and lens features to the service device, and obtaining lens information sent by the service device that corresponds to the user identity information and lens features; or, obtaining one or more sets of association information sent by the service device, determining target association information that matches the user identity information and lens features, and determining the lens information in the target association information as the lens information corresponding to the user identity information and lens features.

[0446] Optionally, the device may also include:

[0447] The prompting module is used to generate a first prompting message to the user based on the user's identity information and lens characteristics to match lens information defects in the storage terminal of the near-eye display device. The first prompting message includes prompting the user whether the lens is installed incorrectly or prompting the user whether to add lens information.

[0448] Optionally, matching defects include: only matching user identity information or lens features in the storage of the near-eye display device, or matching user identity information and lens features in different groups of associated information in the storage of the near-eye display device.

[0449] Optionally, the device may also include:

[0450] The information sending module is used to send the user identity information and / or lens features to the service device based on the user's selection to add lens information;

[0451] The lens information receiving module is used to receive lens information fed back by the service device that matches the user identity information and / or lens features, and to determine the lens information corresponding to the lens.

[0452] The apparatus for acquiring lens information provided in this application can execute the method for acquiring lens information provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method.

[0453] Example 11

[0454] Figure 12 A schematic diagram of the structure of a near-eye display device 100 that can be used to implement embodiments of this application is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of this application described and / or claimed herein.

[0455] like Figure 12 As shown, the near-eye display device 100 includes at least one processor 101 and a memory, such as a read-only memory (ROM) 102 and a random access memory (RAM) 103, communicatively connected to the at least one processor 101. The memory stores computer programs executable by the at least one processor. The processor 101 can perform various appropriate actions and processes based on the computer program stored in the ROM 102 or loaded from storage unit 108 into the RAM 103. The RAM 103 may also store various programs and data required for the operation of the near-eye display device 100. The processor 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0456] Multiple components in the near-eye display device 100 are connected to the I / O interface 105, including: an input unit 106, such as a keyboard, mouse, etc.; an output unit 107, such as various types of displays, speakers, etc.; a storage unit 108, such as a disk, optical disk, etc.; and a communication unit 109, such as a network card, modem, wireless transceiver, etc. The communication unit 109 allows the near-eye display device 100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0457] Processor 101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 101 performs the various methods and processes described above, such as methods for acquiring lens information.

[0458] In some embodiments, the method for acquiring lens information may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 108. In some embodiments, part or all of the computer program may be loaded and / or mounted on the near-eye display device 100 via ROM 102 and / or communication unit 109. When the computer program is loaded into RAM 103 and executed by processor 101, one or more steps of the method for acquiring lens information described above may be performed. Alternatively, in other embodiments, processor 101 may be configured to perform the method for acquiring lens information by any other suitable means (e.g., by means of firmware).

[0459] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0460] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0461] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0462] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0463] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0464] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0465] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0466] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for acquiring lens information, characterized in that, include: Based on the user's wear of a near-eye display device with detachable lenses, basic information is obtained; Match lens information based on the aforementioned basic information; The direction of the line of sight through the lens is determined based on the lens information and the initial eye-tracking calculation model; The eye-tracking calculation model of the near-eye display device is corrected according to the line of sight through the lens; The basic information includes one or more of the following: user identity information, account information, and hardware information. The step of matching lens information based on the basic information includes: Determine whether the basic information has matching lens information in the storage terminal of the near-eye display device; If so, the lens information is successfully matched; If not, the lens information matching fails. Then, the lens information is matched from the service device based on the basic information, and the basic information and the lens information are associated and stored in the storage terminal of the near-eye display device. The step of correcting the eye-tracking calculation model of the near-eye display device based on the line of sight through the lens includes: The virtual image position is determined based on the lens information and the eyepiece side information of the display system; The deviation function is determined based on the virtual image position and the screen position; The eye-tracking calculation model is obtained by correcting the viewing direction through the lens based on the deviation function.

2. The method according to claim 1, characterized in that, The acquisition of basic information includes one or more of the following: Basic information is obtained based on user manual input and / or manual selection; Basic information is obtained based on the recognition of near-eye display devices.

3. The method according to claim 1, characterized in that, Hardware information includes one or more of the following: device information and lens characteristics. Lens characteristics include one or more of the following: lens power, diopter, refractive index, and curvature.

4. The method according to claim 1, characterized in that, The step of matching lens information based on the basic information includes: Based on the aforementioned basic information, lens information is matched at the storage end of the near-eye display device.

5. The method according to claim 1, characterized in that, The step of matching lens information based on the basic information includes: Lens information is matched from the service device based on the aforementioned basic information.

6. The method according to claim 4, characterized in that, The step of matching lens information to the storage terminal of the near-eye display device based on the basic information includes: Based on the basic information, query the local area of ​​the near-eye display device to determine all lens information corresponding to the basic information; If the number of lens information items is one, then the lens information item is determined to be the lens information corresponding to the lens. If there are multiple lens information items, then each lens information item is displayed to the user, and the lens information corresponding to the lens is determined according to the user's selection operation.

7. The method according to claim 1 or 5, characterized in that, The step of matching lens information from the service device based on the basic information includes: The service device is queried based on the basic information, and all lens information corresponding to the basic information is received from the service device. If the number of lens information items is one, then the lens information item is determined to be the lens information corresponding to the lens. If there are multiple lens information items, multiple lens information items will be displayed to the user, and the lens information corresponding to the lens will be determined according to the user's selection operation.

8. The method according to claim 7, characterized in that, When the service device is a mobile storage device, the step of querying the service device based on the basic information and receiving all lens information corresponding to the basic information fed back by the service device includes: Detect service equipment and establish a connection with the service equipment; Send the basic information to the service device to obtain all lens information corresponding to the basic information sent by the service device; or, obtain one or more basic information sent by the service device and the lens information corresponding to the basic information.

9. The method according to claim 1, characterized in that, Correcting the eye-tracking calculation model of the near-eye display device based on the line-of-sight direction through the lens includes: Based on the line of sight through the lens, calculate several sets of gaze positions with corresponding true values; Determine the mapping relationship between several sets of the aforementioned line-of-sight directions and their corresponding truth values; The eye-tracking direction is corrected based on the mapping relationship to obtain the final eye-tracking calculation model.

10. The method according to claim 1, characterized in that, When the basic information is user identity information and lens features, a successful match of lens information means that the user identity information and lens features can be matched simultaneously within the same set of associated information. A failure to match lens information means that neither the user identity information nor the lens features can be matched within any set of associated information.

11. The method according to claim 10, characterized in that, Also includes: Based on user identity information and lens characteristics, the system matches lens information in the storage terminal of the near-eye display device to identify defects and generates a first prompt message to alert the user. The first prompt message includes prompting the user whether the lens is installed incorrectly or whether the user needs to add new lens information.

12. The method according to claim 11, characterized in that, Matching defects include: only matching user identity information or lens features in the storage of the near-eye display device, or matching user identity information and lens features in different groups of associated information in the storage of the near-eye display device.

13. The method according to claim 11, characterized in that, Also includes: Based on the user's selection to add lens information, the user's identity information and / or lens features are sent to the service device; Receive lens information from the service device that matches the user's identity information and / or lens features, and determine the lens information corresponding to the lens.

14. A near-eye display device, characterized in that, The near-eye display device includes: At least one processor, and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method for acquiring lens information as described in any one of claims 1-13.

Images