Methods, devices, electronic equipment and storage media for eyeglass prescription recognition

Abstract

This disclosure relates to a method, apparatus, electronic device, and storage medium for identifying eyeglass prescriptions. The method includes: acquiring an image to be processed, the image including optical features generated when a light source emits light onto the lenses of target eyeglasses; and inputting input data containing the image to be processed into an eyeglass prescription recognition model to obtain the eyeglass prescription of the target eyeglasses. Since the image to be processed includes optical features generated when a light source emits light onto the lenses of the target eyeglasses, and these optical features can reflect the eyeglass prescription of the target eyeglasses, the eyeglass prescription recognition model can effectively identify the eyeglass prescription of the target eyeglasses in the image to be processed.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to methods, devices, electronic devices, and storage media for recognizing eyeglass prescriptions. Background Technology

[0002] Since glasses with different prescriptions produce completely different refractive effects, eye-tracking interaction algorithms need to obtain the glasses prescription to make corresponding corrections to the algorithm results, making the recognition of glasses prescription very important for eye-tracking interaction.

[0003] However, if the user's eyeglass prescription cannot be obtained directly, specialized optical instruments are generally required to determine the prescription, resulting in high costs associated with measuring eyeglass prescriptions. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and storage medium for recognizing eyeglass prescriptions.

[0005] According to a first aspect of this disclosure, a method for identifying eyeglass prescription is provided, the method comprising:

[0006] Acquire an image to be processed; the image to be processed is obtained by image acquisition of the lens of the target glasses under the illumination of a light source, and the image to be processed includes the optical features generated when the light source emits light into the lens of the target glasses;

[0007] Input data containing the image to be processed is fed into the glasses prescription recognition model to obtain the prescription of the target glasses.

[0008] According to a second aspect of this disclosure, a spectacle prescription recognition device is provided, the device comprising:

[0009] An image acquisition module is used to acquire an image to be processed; the image to be processed is obtained by image acquisition of the lens of the target glasses under the illumination of a light source, and the image to be processed includes the optical features generated when the light source emits light to the lens of the target glasses;

[0010] The eyeglasses prescription recognition module is used to input input data containing the image to be processed into the eyeglasses prescription recognition model to obtain the eyeglasses prescription of the target eyeglasses.

[0011] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0012] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described above.

[0013] The eyeglass prescription recognition method, apparatus, electronic device, and storage medium provided in this disclosure obtain the eyeglass prescription of a target eyeglass by acquiring an image to be processed and using input data containing the image to be processed as input to an eyeglass prescription recognition model. Since the image to be processed is an image captured from the lens of the target eyeglass under illumination by a light source, it contains optical features generated when light is emitted from the light source onto the lens of the target eyeglass. These optical features reflect the prescription of the eyeglasses. Therefore, by using the eyeglass prescription recognition model on the image to be processed, the prescription of the target eyeglasses in the image can be accurately identified. Attached Figure Description

[0014] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0015] Figure 1 A schematic diagram of a VR device provided as an exemplary embodiment of this disclosure;

[0016] Figure 2A A first lens effect image obtained by image acquisition of target glasses, provided as an exemplary embodiment of this disclosure;

[0017] Figure 2B A second lens effect image obtained by image acquisition of target glasses, provided as an exemplary embodiment of this disclosure;

[0018] Figure 3 A flowchart illustrating an exemplary embodiment of this disclosure of a method for recognizing eyeglass prescriptions;

[0019] Figure 4 Another flowchart of a method for recognizing eyeglass prescriptions provided as an exemplary embodiment of this disclosure;

[0020] Figure 5 Another flowchart of a method for recognizing eyeglass prescriptions provided as an exemplary embodiment of this disclosure;

[0021] Figure 6 A schematic diagram of an eyeglass prescription recognition device provided as an exemplary embodiment of the present disclosure;

[0022] Figure 7 A structural block diagram of an electronic device provided as an exemplary embodiment of this disclosure;

[0023] Figure 8 A block diagram of a computer system provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0024] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0025] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0026] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0027] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0028] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0029] Measuring eyeglass prescriptions typically requires users to visit specialized testing institutions and use specialized optical instruments, making the measurement process costly. Since different prescriptions result in completely different refractive effects, eyeglass prescriptions are necessary to correct the algorithm's results when using eye-tracking interaction algorithms. Therefore, accurate eyeglass prescription recognition is crucial for eye-tracking interaction.

[0030] When using eye-tracking interaction algorithms, the results need to be corrected in two ways using eyeglass prescription: First, adding a spectacle lens between the camera and the human eye causes a slight deviation between the observed gaze direction and the actual gaze direction. The higher the prescription, the greater the deviation. Myopia and hyperopia have different directional deviation effects. Through extensive data collection and optical simulation, the gaze estimation results can be corrected in the gaze estimation neural network and in post-processing. Second, adding a spectacle lens between the camera and the human eye also causes a difference between the observed interpupillary distance and the actual interpupillary distance. This difference is related to the spectacle prescription and the position of the spectacle. Using the spectacle prescription can provide some systematic and coarse corrections.

[0031] Eye-tracking interaction algorithms are widely used in VR (Virtual Reality) devices. By accurately and quickly identifying the prescription of the user's glasses and making corrections accordingly, eye-tracking interaction algorithms can run effectively on VR devices, resulting in a better user experience.

[0032] Therefore, in order to accurately, quickly and effectively identify the prescription of the user's glasses, in the embodiments provided in this disclosure, such as Figure 1 As shown, at least one light source 11 and an image acquisition device 12 are respectively installed on the two lens barrels of the VR device 10. The image acquisition device can be a camera, which can be an infrared camera, and the light source can be an infrared light source, such as a light-emitting diode (LED). In this embodiment, an LED light source is used as an example. The LED light involved in this embodiment can be an infrared LED light, but it can also be a regular LED light; the specific choice can be made according to needs.

[0033] It should be noted that the operating wavelength of the image acquisition device in this embodiment should match the wavelength of the light emitted by the light source. For example, the light source in the embodiment can be an infrared light source. Since infrared light is imperceptible to the human eye, it is unlikely to cause harm to the human eye and will not affect the normal display of the VR device. In addition, the image acquisition device can also be a device that performs imaging based on infrared light.

[0034] When users wear the glasses and use the VR device, they can activate multiple LED lights on each of the two lens barrels. These LED lights will emit light towards the area where the glasses are located, and the light will create reflected spots and lens textures on the lenses. Figure 2A and Figure 2B As shown, Figure 2A The first lens effect image of the target glasses. Figure 2B This is an image showing the effect of the second lens on the target glasses.

[0035] Therefore, in this embodiment of the present disclosure, when a user is detected wearing a VR device, the LED lights on both lens barrels can be turned on, and the cameras on both lens barrels can be turned on simultaneously to acquire images. Since the light emitted by the LED lights will produce optical features on the lenses of the user's glasses, such as reflected light spots and lens textures, the images acquired by the image acquisition device will generally also contain these light spots and lens textures. In addition, the light emitted by the LED lights will produce diffuse reflection on the sides of the lenses, and the image acquisition device may also capture the thickness of the lenses.

[0036] In this embodiment, the lens prescriptions include those for nearsightedness and farsightedness. Since the lens structures for different prescriptions differ, the optical characteristics produced by the light emitted from the light source on lenses of different prescriptions will also differ. That is, the optical characteristics produced by the light emitted from the light source on the lens can reflect the lens prescription, and the lens prescription can be identified through these optical characteristics. These optical characteristics can include one or more of the following: light spot, lens texture, and lens thickness information.

[0037] Specifically, under normal circumstances, LED lights create two distinct sets of reflected light spots on the lenses of eyeglasses: one set formed by specular reflection from the outer surface of the lens, and the other set formed on the inner surface. The difference in position and size between the inner and outer surface spots is determined by the curvature of these two surfaces, which in turn determines the prescription of the eyeglasses. Therefore, the light spots on the lenses can reflect the prescription of the eyeglasses.

[0038] In this embodiment, when light is emitted onto the lenses of the glasses via an LED light, the light typically undergoes multiple reflections and refractions within the lens, resulting in circular patterns, i.e., lens texture. Therefore, the texture on the lens can also reflect the glasses' prescription. Generally, the higher the prescription, the more pronounced the circular texture will appear near the edge of the lens when the camera captures the glasses; these textures can be used to estimate the prescription. It should be noted that the prescription of the glasses in this embodiment can be an approximate prescription or a farsighted prescription, etc. Of course, the glasses can also be plane mirrors, such as non-prescription blue light blocking glasses that users wear regularly.

[0039] In addition, generally speaking, the higher the prescription of eyeglasses, the thicker the rims. When a camera photographs eyeglasses, it also captures the thickness information of the rims. Therefore, the thickness information on the lenses can also reflect the prescription of the eyeglasses.

[0040] It should be noted that cameras typically capture better images of the lens texture and thickness when shooting from the side, thus requiring a certain non-perpendicular angle between the camera and the lens. However, since most eyeglass lenses are curved, the camera on the VR device's lens barrel will also form a non-perpendicular angle with the lens when capturing images, without affecting the captured information such as lens light spots, texture, and thickness.

[0041] Therefore, in this embodiment of the present disclosure, the cameras on the two lenses of the glasses can respectively capture images of the two lenses of the glasses. The captured image information can be input into the trained glasses diopter recognition model to identify the diopter of the glasses and obtain the glasses diopter, which includes the diopter of the two lenses of the glasses.

[0042] In this embodiment, cameras on the two lenses of the glasses, namely the first lens image and the second lens image, can be captured separately. In some examples, the first and second lens images can be used as images to be processed and as input to the glasses prescription recognition model. In other examples, the two lens images can be stitched together horizontally, and the stitched image can be used as input to the glasses prescription recognition model. After inputting the first and second lens images as images to be processed into the glasses prescription recognition model, the prescription of the first and second lenses can be identified separately. When the two lens images are stitched together horizontally and the stitched image is used as input to the glasses prescription recognition model, the prescription of both lenses can be obtained simultaneously. Since there is generally a correlation between the prescriptions of the left and right eyes, stitching together the images of the first and second lenses and using the stitched image as input to the eyeglass prescription recognition model can achieve a more accurate recognition result. Correspondingly, during the training process of the eyeglass prescription recognition model, the stitched image can be used as a training sample, allowing the model to learn the correlation between the images of the two lenses during training, thus enabling the trained model to better identify the prescription of the glasses.

[0043] In the embodiments provided in this disclosure, when a VR device is worn on a user's eyes, the image content included in the captured images varies depending on the pose and field of view of the image acquisition device itself. For example, in some examples, the image content in the first and second lens images captured by the image acquisition device may only include the lens area. In other examples, the image content in the first and second lens images captured by the image acquisition device may simultaneously include the lens area and the area outside the lens, where the area outside the lens can be other facial areas of the user (e.g., nose, forehead, etc.). Since the image acquisition device emits light towards the area where the target glasses are located during the acquisition process, the lens area in the first and second lens images captured by the image acquisition device will include optical features, such as light spot information. Therefore, to enable the eyeglasses prescription recognition model to focus more on eyeglasses-related information in the input data, thereby extracting effective information for eyeglasses prescription recognition more quickly and ensuring recognition efficiency, the embodiment can also use an image segmentation model to segment the first lens image and the second lens image, obtaining the lens image and spot image of the first lens image, and the lens image and spot image of the second lens image, respectively. The lens images segmented from the first lens image and the lens images segmented from the second lens image may contain only the lens region. Then, the lens image and spot image segmented from the first lens image are stitched together with the first lens image in the channel dimension to form the first lens stitched image; simultaneously, the lens image and spot image segmented from the second lens image are stitched together with the second lens image in the channel dimension to form the second lens stitched image. The first lens stitched image and the second lens stitched image can be used as inputs to the eyeglasses prescription recognition model to identify the eyeglasses prescription of the first lens and the second lens, respectively. Alternatively, the stitched images of the first and second lenses can be stitched together horizontally and used as input to a glasses prescription recognition model to identify the prescription of the first and second lenses. The glasses prescription recognition model in this embodiment can be a trained deep regression model. For example, if the first lens image is an RGB color image, it may contain three channels: R, G, and B. Stitching the segmented lens image, the spot image, and the first lens image along the channel dimension will result in a stitched image of the first lens with five channels. If the first lens image is acquired through an infrared camera, it has only one channel. Stitching the segmented lens image, the spot image, and the first lens image along the channel dimension will result in a stitched image of the first lens with three channels.

[0044] It should be noted that since the light spot, lens texture, and lens thickness information on the lens can all reflect the prescription of the glasses, in this embodiment, when identifying the prescription of the glasses, at least one of the light spot, lens texture, and lens thickness information can be used as input to the glasses prescription recognition model simultaneously, thus enabling the identification of the glasses prescription. Alternatively, an image containing light spot, lens texture, and lens thickness information can be used as input to the glasses prescription recognition model to identify the glasses prescription, which can achieve more accurate identification of the glasses prescription. During the training process of the glasses prescription recognition model, the training samples used can be sample images containing at least one of the light spot, lens texture, and lens thickness information. These sample images will be labeled with the light spot area, lens area, and / or lens thickness information, and will also carry the corresponding glasses prescription.

[0045] In this embodiment, an image segmentation model can be used to segment the light spot from the first lens image and the second lens image. This image segmentation model can be obtained by training a preset model with training samples. These training samples can include images captured by an image acquisition device on a VR device when the user wears the glasses. Furthermore, to obtain sufficient samples and reduce the cost of acquiring training samples, multiple training samples can be obtained through image synthesis. Lens thickness information can be detected by identifying the lens edges; this will not be elaborated upon here.

[0046] Specifically, in this embodiment, various types of glasses can be collected, and the subject can wear multiple different pairs of glasses. Images of the subject's eyes are then captured using a camera on a VR device. For each pair of glasses, multiple different wearing positions need to be captured, and the lens area and light spot area need to be labeled. For example, the light spot area can be labeled manually, or a threshold algorithm can be used to binarize the image, with areas where the pixel value is greater than a threshold being designated as the light spot area.

[0047] In this embodiment, a large number of different images containing the texture of the glasses can be obtained by capturing images of pure glasses using a camera on the VR device, and the lens area and reflected light spot area can be labeled. By collecting a large amount of eye image data of subjects without glasses, an image synthesis method is used to arbitrarily combine the subject's eye image data and glasses image to generate batch synthetic data with glasses. This synthetic data is then used as training samples for the image segmentation model, thus obtaining sufficient training samples to train the model and greatly reducing the cost of sample collection. In addition, this embodiment can also use non-synthetic sample data to fine-tune the trained image segmentation model. This way, while using synthetic data as training samples reduces the cost of training sample collection, fine-tuning the trained image segmentation model using non-synthetic data ensures the model's processing effect on real data. Here, the non-synthetic data is data directly collected by the image acquisition device set on the VR device after the subject wears different types of glasses and wears the VR device.

[0048] In conjunction with the above embodiments, in another embodiment provided in this disclosure, a method for recognizing eyeglass prescriptions is also provided, such as... Figure 3 As shown, the method may include the following steps:

[0049] In step S310, the image to be processed is acquired.

[0050] The image to be processed is obtained by acquiring an image of the lens of the target glasses under illumination by a light source. The image to be processed includes the optical features generated when the light source emits light onto the lens of the target glasses.

[0051] The above can be combined Figure 1 In a corresponding embodiment, the VR device provided in this disclosure captures images of a user wearing glasses. When capturing images through the camera on the VR device, the LED light on the VR device is turned on. The LED light emits light towards the lens of the target glasses, thus capturing an image containing the optical features generated by the light on the lens of the glasses. These optical features may include features such as the light spot formed by the light on the lens of the target glasses, lens texture, and lens thickness information.

[0052] In this embodiment, the VR device includes a first lens barrel and a second lens barrel. The first and second lens barrels are respectively equipped with an image acquisition device and at least one light source; the image acquisition device can specifically be a camera.

[0053] In step S320, input data containing the image to be processed is input to the glasses prescription recognition model to obtain the glasses prescription of the target glasses.

[0054] The first preset model is trained using multiple training samples of the first type to obtain a glasses prescription recognition model. The first type of training samples includes lens images of preset glasses, each containing optical features generated when light is emitted from a light source onto the lens of the preset glasses. Each training sample carries a corresponding glasses prescription. The preset glasses can include various styles and prescriptions of glasses.

[0055] In this embodiment, the image to be processed can be an image corresponding to one lens of the target glasses, so that the glasses prescription can be obtained through the glasses prescription recognition model. Alternatively, the image to be processed can be an image corresponding to both lenses of the target glasses, namely, a first lens image and a second lens image. The first lens image and the second lens image can be input into the glasses prescription recognition model separately to obtain the glasses prescription of each lens of the target glasses. Or, the image to be processed can be a stitched image of the first lens image and the second lens image of the target glasses. Inputting this stitched image into the glasses prescription recognition model can simultaneously obtain the glasses prescription of both lenses of the target glasses. It should be noted that in this embodiment, the stitched image can be directly used as input to the glasses prescription recognition model, or input data containing at least the stitched image can be used as input to the glasses prescription recognition model.

[0056] As described in the embodiments above, the optical features produced by the light emitted from the light source on the lens can reflect the prescription of the glasses. For example, the optical features may include one or more of the following: light spot, lens texture, and lens thickness information can all be used as the basis for identifying the prescription of the glasses. Therefore, the image to be processed may contain one or more of the following features: light spot, lens texture, and lens thickness information. In order to more accurately identify the prescription of the glasses, the image to be processed in the embodiments may contain both light spot and lens texture, or the image to be processed may contain both light spot, lens texture, and lens thickness information.

[0057] Furthermore, in the process of training the first preset model with the first type of training samples to obtain the glasses prescription recognition model, the training samples may contain one or more features such as light spots, lens texture, and lens thickness information, and each training sample carries the corresponding glasses prescription. The first preset model can be a neural network model or similar model. Training the first preset model with the training samples yields the glasses prescription recognition model upon completion.

[0058] The glasses prescription recognition method provided in this disclosure obtains the prescription of the target glasses by acquiring an image to be processed and using input data containing the image to be processed as input to a glasses prescription recognition model. Since the image to be processed is an image acquired from the lens of the target glasses under illumination by a light source, it contains optical features generated when light is emitted from the light source onto the lens of the target glasses. These optical features reflect the prescription of the glasses. Therefore, when the image to be processed is used to recognize the prescription of the glasses through the glasses prescription recognition model, the model is trained using training samples containing the optical features generated when light is emitted from the light source onto the lens of a preset pair of glasses and carrying corresponding prescriptions. This allows the glasses prescription recognition model to accurately identify the prescription of the target glasses in the image to be processed.

[0059] In the embodiments provided in this disclosure, based on the above embodiments, the eyeglass prescription recognition method provided in this disclosure is as follows: Figure 4 As shown, it may also include the following steps:

[0060] In step S330, the image to be processed is segmented to obtain a feature image including optical features, and the feature image is used as at least part of the input data.

[0061] In this embodiment, since the optical features include one or more of light spots, lens texture, and lens thickness information, for example, when the feature image includes a lens image and a light spot image, the image to be processed is input into the image segmentation model to obtain the lens image and the light spot image. The image segmentation model is obtained by training a second preset model using a second type of training samples. This second preset model can be a neural network model or similar model. The second type of training samples includes images synthesized from different user images and different preset feature images. The preset feature images include pre-labeled light spot regions and pre-labeled lens regions. The user image refers to a facial image that at least includes the user's eyes.

[0062] In this way, inputting the feature image and the image to be processed together into the glasses prescription recognition model is equivalent to inputting the original image of the target glasses together with the feature image of the original image into the glasses prescription recognition model. This can improve the accuracy of the glasses prescription recognition model in recognizing the prescription of the target glasses and make it more accurate.

[0063] It should be noted that the training process of the image segmentation model and the method of synthesizing training samples can be found in the description of the above embodiments, and will not be repeated here.

[0064] In the embodiments provided in this disclosure, if the feature image includes a lens image and a spot image, and the lens image contains lens texture, the image to be processed, the lens image, and the spot image can be stitched together along the channel dimension, and the stitched image can be used as input to the glasses prescription recognition model. Since glasses texture is not easily segmented separately, the spot image can be segmented from the image to be processed, resulting in a lens image carrying the glasses texture. By stitching the image to be processed, the lens image, and the spot image along the channel dimension, they are associated to form a unified image, which facilitates more accurate recognition of the target glasses prescription by the glasses prescription recognition model.

[0065] In the embodiments provided in this disclosure, when the VR device provided in the above embodiments is used to acquire images of the target glasses, images of the two lenses of the target glasses can be obtained, namely, the first lens image and the second lens image of the target glasses. In the embodiments, the first lens image and the second lens image can be stitched together in the horizontal direction, and the stitched image is used as the image to be processed. In this way, when the glasses prescription recognition model is used to identify the image to be processed to obtain the glasses prescription of the target glasses, the glasses prescription of the first lens and the second lens of the target glasses can be identified simultaneously, resulting in better recognition. Since there is generally a certain correlation between the glasses prescription of the left and right eyes of a person, when the first lens image and the second lens image are stitched together and the stitched image is used as the input of the glasses prescription recognition model, a more accurate recognition effect can be achieved.

[0066] In the embodiments provided in this disclosure, the image to be processed can be acquired by an image acquisition device located on a VR device. The image acquisition device includes at least a first image acquisition device and a second image acquisition device. The first image acquisition device is configured to acquire an image of a first lens in the target glasses, and the second image acquisition device is configured to acquire an image of a second lens in the target glasses. At least one light source is provided on the VR device. The VR device in the embodiments of this disclosure performs image acquisition on the target glasses with the user's authorization. When it is detected that the target user is wearing the VR device, it can be determined whether the target user is wearing glasses. If the target user is not wearing glasses, image acquisition of the target user's eyes can be stopped. If the target user is wearing glasses, light is emitted from the light source on the VR device towards the lenses of the target glasses, and the image acquisition device on the VR device acquires an image of the lenses of the target glasses. The glasses can be the target glasses in the embodiments. By determining whether the user is wearing glasses before image acquisition, the efficiency of recognition can be improved, avoiding the poor user experience caused by emitting light and acquiring images from the user even when they are not wearing glasses, and also improving the efficiency of glasses prescription recognition.

[0067] Based on the above embodiments, in another embodiment provided in this disclosure, such as Figure 5 As shown, the method may further include the following steps:

[0068] In step S510, multiple images of the target glasses to be processed are acquired.

[0069] In step S520, based on multiple images to be processed, multiple sets of glasses diopters for the target glasses are obtained through a glasses diopters recognition model.

[0070] Each set of glasses prescriptions includes the prescription of the first lens and the prescription of the second lens of the target glasses.

[0071] In step S530, the target eyeglass prescription is obtained based on multiple sets of eyeglass prescriptions.

[0072] In this embodiment, to improve the accuracy of the target glasses' prescription, multiple images of the target glasses can be continuously acquired. These images are then processed using the methods described in the above embodiment and input into the glasses prescription recognition model to obtain the glasses prescription corresponding to each processed image. This yields multiple sets of glasses prescriptions for the target glasses. The prescription can be determined by averaging these sets, resulting in a more accurate prescription. Furthermore, when obtaining the target glasses prescription based on multiple sets of prescriptions, outliers can be first excluded, and then the remaining sets of prescriptions can be averaged to obtain the final, more accurate result.

[0073] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0074] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as electronic devices (including wearable devices like VR devices), applications, servers, or storage media, that perform the operations of this disclosed technical solution, based on the prompt message.

[0075] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to an electronic device (such as a VR device).

[0076] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0077] This disclosure provides a glasses prescription recognition device, which can be a VR device. Figure 6 This is a schematic block diagram illustrating the functional modules of an eyeglass prescription recognition device provided as an exemplary embodiment of this disclosure. Figure 6 As shown, the glasses prescription recognition device includes:

[0078] Image acquisition module 61 is used to acquire an image to be processed; the image to be processed is obtained by image acquisition of the lens of the target glasses under the illumination of a light source, and the image to be processed includes the optical features generated when the light source emits light to the lens of the target glasses;

[0079] The glasses prescription recognition module 62 is used to input input data containing the image to be processed into the glasses prescription recognition model to obtain the glasses prescription of the target glasses.

[0080] In yet another embodiment provided in this disclosure, the apparatus further includes:

[0081] An image segmentation module is used to segment the image to be processed to obtain a feature image including the optical features, and to use the feature image as at least a part of the input data.

[0082] In another embodiment provided in this disclosure, the optical features include one or more combinations of the following: light spot, lens texture, and lens thickness information.

[0083] In yet another embodiment provided in this disclosure, the apparatus further includes:

[0084] The training module is used to train a first preset model using multiple training samples of the first type to obtain the glasses prescription recognition model. The first type of training samples includes lens images of preset glasses. The lens images contain optical features generated when a light source emits light onto the lens of the preset glasses. The training samples carry the corresponding glasses prescription.

[0085] In another embodiment provided in this disclosure, the feature image includes a lens image and a light spot image, and the image segmentation module is specifically used for:

[0086] The image to be processed is input into the image segmentation model to obtain the lens image and the spot image; wherein, the image segmentation model is obtained by training the second preset model with the second type of training samples, and the second type of training samples includes images obtained by synthesizing different user images with different preset feature images, and the preset feature images include the pre-labeled spot region and the pre-labeled lens region.

[0087] In another embodiment provided in this disclosure, the feature image includes a lens image and a light spot image, the lens image includes a lens texture, and the eyeglass prescription recognition module is specifically used for:

[0088] The image to be processed, the lens image, and the spot image are stitched together along the channel dimension, and the stitched image is used as the input to the glasses prescription recognition model.

[0089] The lens image includes a first lens image and a second lens image, and the device further includes:

[0090] The stitching processing module is used to stitch the images of the first lens and the second lens in the channel dimension and in the horizontal direction, and to use the stitched image as the input of the glasses diopter recognition model.

[0091] In yet another embodiment provided in this disclosure, the apparatus further includes:

[0092] The lens image acquisition module acquires the first lens image and the second lens image of the target glasses;

[0093] The image stitching module is used to stitch the first lens image and the second lens image in the horizontal direction, and use the stitched image as the image to be processed.

[0094] In another embodiment provided in this disclosure, the image to be processed is acquired by an image acquisition device located on a VR device; wherein, the image acquisition device includes at least a first image acquisition device and a second image acquisition device, the first image acquisition device is configured to acquire an image of a first lens in the target glasses, the second image acquisition device is configured to acquire an image of a second lens in the target glasses, and at least one light source is provided on the VR device.

[0095] In yet another embodiment provided in this disclosure, the apparatus further includes:

[0096] The glasses wearing determination module is used to determine whether the target user is wearing the target glasses when the target user is detected to be wearing a VR device;

[0097] An image acquisition module is used to emit light onto the lenses of the target glasses through a light source on the VR device when the target user is wearing the target glasses, and to acquire images of the lenses of the target glasses through an image acquisition device on the VR device.

[0098] In yet another embodiment provided in this disclosure, the apparatus further includes:

[0099] The image acquisition module is used to acquire multiple images of the target glasses.

[0100] The image recognition module is used to obtain multiple sets of glasses prescriptions for the target glasses based on the multiple images to be processed, using the glasses prescription recognition model; wherein each set of glasses prescriptions includes the first lens prescription and the second lens prescription of the target glasses.

[0101] The target glasses prescription acquisition module is used to obtain the target glasses prescription based on the multiple sets of glasses prescriptions.

[0102] Since this device corresponds to the method described above, its specific description can be found in the description of the method embodiments described above, and will not be repeated here.

[0103] The eyeglass prescription recognition device provided in this embodiment acquires an image to be processed and uses input data containing the image to be processed as input to an eyeglass prescription recognition model to obtain the prescription of the target eyeglasses. Since the image to be processed is an image captured from the lens of the target eyeglasses under illumination by a light source, it contains optical features generated when light is emitted from the light source onto the lens of the target eyeglasses. These optical features reflect the prescription of the eyeglasses. Therefore, when the image to be processed is used to recognize the prescription of the eyeglasses through the eyeglass prescription recognition model, the model is trained using training samples containing the optical features generated when light is emitted from the light source onto the lens of a preset eyeglasses and carrying corresponding prescriptions. This allows the eyeglass prescription recognition model to accurately identify the prescription of the target eyeglasses in the image to be processed.

[0104] This disclosure also provides an electronic device, which may be the VR device described above, comprising: at least one processor; and a memory for storing executable instructions of the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the method disclosed in this disclosure.

[0105] Figure 7This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 7 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0106] The processor 1801 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 1801 or by software instructions. The processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1802, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1801 reads information from the memory 1802 and, in conjunction with its hardware, completes the steps of the method described above.

[0107] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 8 The computer system 1900 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including those described above. Figure 8 A block diagram of a computer system provided for an exemplary embodiment of this disclosure.

[0108] Computer System 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0109] like Figure 8 As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. The RAM 1903 may also store various programs and data required for the operation of the computer system 1900. The computing unit 1901, ROM 1902, and RAM 1903 are interconnected via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0110] Multiple components in computer system 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information into computer system 1900. Input unit 1906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, hard disks and optical disks. Communication unit 1909 allows computer system 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0111] The computing unit 1901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1900 via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0112] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0113] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0114] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0115] This disclosure also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this disclosure.

[0116] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0117] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0118] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0119] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0120] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0121] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A method of identifying the power of a spectacle lens, characterized in that The method includes: Acquire an image to be processed; the image to be processed is obtained by image acquisition of the lens of the target glasses under the illumination of a light source, and the image to be processed includes the optical features generated when the light source emits light into the lens of the target glasses; Input data containing the image to be processed is fed into the glasses prescription recognition model to obtain the glasses prescription of the target glasses. The method further includes: The first preset model is trained by multiple training samples of the first type to obtain the glasses prescription recognition model. The first type of training samples includes lens images of preset glasses. The lens images contain optical features generated when a light source emits light onto the lens of the preset glasses. The training samples carry the corresponding glasses prescription.

2. The method of claim 1, wherein, The method further includes: The image to be processed is segmented to obtain a feature image including the optical features, and the feature image is used as at least a part of the input data.

3. The method of claim 1, wherein, The optical features include one or a combination of the following: light spot, lens texture, and lens thickness information.

4. The method according to claim 2, characterized in that, The feature image includes a lens image and a light spot image. The step of segmenting the image to be processed includes: The image to be processed is input into the image segmentation model to obtain the lens image and the spot image; wherein, the image segmentation model is obtained by training the second preset model with the second type of training samples, and the second type of training samples includes images obtained by synthesizing different user images with different preset feature images, and the preset feature images include the pre-labeled spot region and the pre-labeled lens region.

5. The method according to claim 2, characterized in that, The feature image includes a lens image and a light spot image. The lens image contains lens texture. The input includes input data of the image to be processed to the eyeglass prescription recognition model, including: The image to be processed, the lens image, and the spot image are stitched together along the channel dimension, and the stitched image is used as the input to the glasses prescription recognition model.

6. The method according to claim 1, characterized in that, The method further includes: Acquire the first lens image and the second lens image of the target glasses; The first lens image and the second lens image are stitched together in the horizontal direction, and the stitched image is used as the image to be processed.

7. The method according to claim 1, characterized in that, The image to be processed is acquired by an image acquisition device located on the VR device; wherein the image acquisition device includes at least a first image acquisition device and a second image acquisition device, the first image acquisition device is configured to acquire an image of a first lens in the target glasses, the second image acquisition device is configured to acquire an image of a second lens in the target glasses, and at least one light source is provided on the VR device.

8. The method according to claim 7, characterized in that, The method further includes: If the VR device is detected to be worn by the target user, it is determined whether the target user is wearing the target glasses. When the target user is wearing the target glasses, light is emitted from the light source on the VR device to the lenses of the target glasses, and the image acquisition device on the VR device acquires an image of the lenses of the target glasses.

9. The method according to claim 1, characterized in that, The method further includes: Acquire multiple images of the target glasses to be processed; Based on the multiple images to be processed, the glasses prescription recognition model is used to obtain multiple sets of glasses prescriptions for the target glasses; wherein, each set of glasses prescriptions includes the prescription of the first lens and the prescription of the second lens of the target glasses; Based on the multiple sets of eyeglass prescriptions, the target eyeglass prescription is obtained.

10. A device for recognizing eyeglass prescriptions, characterized in that, The device includes: An image acquisition module is used to acquire an image to be processed; the image to be processed is obtained by image acquisition of the lens of the target glasses under the illumination of a light source, and the image to be processed includes the optical features generated when the light source emits light to the lens of the target glasses; The glasses prescription recognition module is used to input input data containing the image to be processed into the glasses prescription recognition model to obtain the glasses prescription of the target glasses; The training module is used to train a first preset model using multiple training samples of the first type to obtain the glasses prescription recognition model. The first type of training samples includes lens images of preset glasses. The lens images contain optical features generated when a light source emits light onto the lens of the preset glasses. The training samples carry the corresponding glasses prescription.

11. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-9.

Images