Federal learning in service environment

Abstract

The invention relates to federated learning in a service environment. The present disclosure relates to a computing device for an application in a retail environment, the computing device comprising: an interface module configured to transmit and receive signals between a local neural network and a global neural network, wherein: the local neural network implemented by the computing device is configured to process and analyze in-store data relating to a wearer or potential wearer of a head-mounted device for predictive and personalized service provision, and the interface module enables the local neural network to participate in a federated learning process with the global neural network through the transmitted and received signals.

Description

Technical Field

[0001] The present invention relates to data science, and more particularly, to a computing device, a corresponding method, and a corresponding computer-readable storage medium for deploying applications, for example, in a retail environment. Background Art

[0002] In a retail environment, there is an increasing demand for advanced innovations and algorithms to provide personalized services to customers. However, the sharing of sensitive customer data and the need to comply with data protection regulations pose significant challenges. Optometrists and other eye care professionals (ECPs) need to access algorithms and predictive models without compromising data privacy or violating confidentiality requirements.

[0003] Traditional methods involve centralizing and sharing large amounts of data to improve predictive models and algorithms. However, this approach is often hindered by regulatory restrictions, data protection regulations, and the proprietary nature of customer information. There is a need for a novel solution that overcomes these limitations and enables retailers and individual operators to fully utilize advanced algorithms while maintaining data privacy and confidentiality.

[0004] Existing methods in the field of predictive modeling and machine learning have several drawbacks. Strengthened regulations on sensitive data and the emphasis on customer data protection limit data sharing with various organizations, thus restricting ECPs' access to advanced technologies that can enhance service quality.

[0005] In addition, the lack of common experience among different stores hinders the effectiveness of predictive models of individual retailers, which heavily rely on the experience or datasets of their own stores.

[0006] Furthermore, the practical difficulties associated with collecting large amounts of data, such as complying with data protection regulations and data storage limitations, pose an obstacle to improving machine learning models.

[0007] There is a need for devices and methods that can overcome the above limitations. Summary of the Invention

[0008] The present invention is defined by the appended independent claims. Additional features and advantages of the concepts disclosed herein are set forth in the following description.

[0009] The present disclosure aims to improve the situation.

[0010] To this end, the present disclosure describes a computing device comprising:

[0011] an interface module configured to send and receive signals between a local neural network and a global neural network,

[0012] wherein:

[0013] The local neural network implemented by the computing device is configured to process and analyze data related to the wearer or potential wearer of the head-mounted device for predictive and personalized service provision, and

[0014] the interface module enables the local neural network to participate in the federated learning process with the global neural network via the transmitted and received signals.

[0015] In other words:

[0016] The computing device is configured to use the local neural network to process and analyze data related to the wearer or potential wearer of the head-mounted device for predictive and personalized service provision, and

[0017] the interface module is configured to send and receive these signals between the local neural network and the global neural network, enabling the local neural network to participate in the federated learning process with the global neural network via the transmitted and received signals.

[0018] For example, the computing device can be a computing device for an application in a retail environment, the computing device including:

[0019] an interface module configured to send and receive signals between the local neural network and the global neural network,

[0020] wherein:

[0021] the local neural network implemented by the computing device is configured to process and analyze in-store data related to the wearer or potential wearer of the head-mounted device for predictive and personalized service provision, and

[0022] the interface module enables the local neural network to participate in the federated learning process with the global neural network via the transmitted and received signals.

[0023] Federated learning offers a promising solution to these challenges. By leveraging distributed and decentralized data science techniques, federated learning enables different entities to jointly train machine learning models without directly sharing data. This approach limits data flow, addresses ecological and security issues, and allows for the use of more efficient and general algorithms.

[0024] The present disclosure also describes a method, the method including:

[0025] collecting data related to the wearer or potential wearer of the head-mounted device via an input module of a computing device,

[0026] Process the in-store data through a local neural network within the computing device for providing at least one predictive and personalized service.

[0027] Send and receive signals between the local neural network and the global neural network through an interface module within the computing device.

[0028] Wherein, the local neural network participates in a federated learning process with the global neural network, and the federated learning process is facilitated by the sending and receiving of the signals.

[0029] For example, the method can be a method for deploying an application in a retail environment, and the method includes:

[0030] Collect in-store data related to the wearer or potential wearer of a head-mounted device through an input module of the computing device.

[0031] Process the in-store data through a local neural network within the computing device for providing at least one predictive and personalized service.

[0032] Send and receive signals between the local neural network and the global neural network through an interface module within the computing device.

[0033] Wherein, the local neural network participates in a federated learning process with the global neural network, and the federated learning process is facilitated by the sending and receiving of the signals.

[0034] The present disclosure also describes a computer-readable storage medium, optionally a non-transitory computer-readable storage medium, having stored thereon a computer program including instructions that, when executed by a processor, cause the processor to execute the methods described herein.

[0035] The storage medium can include a hard disk drive, a solid-state drive, a CD, a USB drive, etc. In this context, it is used to refer to any medium that is storing the computer program for running the method.

[0036] The present disclosure also describes a computer program that includes instructions accessible by a processor and that, when executed by the processor, cause the processor to execute the methods described herein.

[0037] The present disclosure also describes a device equipped with a processor that is operably connected to a memory and a communication interface, and the device is configured to execute any of the methods described herein.

[0038] In one example, the data (or in-store data) includes images captured by an imaging device that depict a wearer or potential wearer during an actual fitting of a head-wearable (or head-mounted) device, and processing the data (or in-store data) includes determining boxing points corresponding to the contour of the head-mounted device in these images, wherein the federated learning process enhances the determination of these boxing points.

[0039] Processing images (or in-store images) to determine boxing points has practical applications in ensuring the optimal fit of a head-wearable (or head-mounted) device. This approach enables precise customization, thereby enhancing the comfort and satisfaction of the wearer. Using federated learning in this context includes improving the accuracy of measurements, resulting in a better-fitting product. This can be particularly useful in eyewear fitting, where even minor differences can affect the comfort of the wearer.

[0040] In one example, the data (or in-store data) includes real-time or static facial data of a potential wearer, and processing the data (or in-store data) includes generating a simulated appearance of the potential wearer wearing the head-wearable (or head-mounted) device based on the facial data, implementing a virtual fitting process, wherein the federated learning process enhances the rendering of the simulated appearance.

[0041] Generating a simulated appearance using real-time or static facial data provides significant advantages for virtual fitting. It enables customers to visually experience how different head-wearable (or head-mounted) devices would look on them without actually trying them on, thus saving time and enhancing the shopping experience. In this context, federated learning enables a more attractive and accurate presentation of products, which is particularly beneficial in the fashion and eyewear retail sectors.

[0042] In one example, the data (or in-store data) further includes feedback on the perceived comfort of the virtual fitting collected by the input module from the potential wearer, and processing the data (or in-store data) further includes adjusting the rendering of the simulated appearance by the local neural network based on the feedback in subsequent virtual fittings.

[0043] Incorporating feedback into the virtual fitting process enables iterative improvement, thereby enhancing user satisfaction. By adjusting the rendering based on the wearer's feedback, the model can provide a more personalized and comfortable experience. This approach is particularly effective in customizing virtual fittings according to individual preferences, thus increasing customer engagement and improving product selection.

[0044] In one example, the data (or in-store data) includes one or more measurements or characteristics of at least one eye of a potential wearer, and processing the data (or in-store data) includes analyzing the one or more measurements or characteristics to determine a prescription for the potential wearer, wherein the federated learning process enhances the accuracy of the prescription.

[0045] Analyzing eye measurements or characteristics to determine a prescription can greatly contribute to the recommendation process for personalized eye care. Through the enhancement of federated learning, this method provides higher accuracy in prescription determination, reduces errors, and ultimately provides a more customized optical solution for the individual.

[0046] In one example, the federated learning process includes: combining or aggregating the weights of a specific model from the local neural network with the weights of other specific models of other local neural networks to create a general model that fully utilizes specific characteristics from all specific models, and updating the local neural network based on the weights of the general model, thereby enhancing the capabilities of the local neural network.

[0047] The combination or aggregation of model weights in the federated learning process promotes a more robust and adaptable neural network. By fully leveraging the strengths of individual local models, the resulting general model exhibits enhanced predictive capabilities and generality. This is particularly advantageous in diverse retail environments where the ability to meet a wide range of customer needs and preferences is crucial.

[0048] In one example, the local neural network facilitates continuous learning based on corrections or feedback associated with previous outputs of the local neural network.

[0049] The ability of the local neural network to continuously learn through the influence of feedback and corrections ensures continuous improvement and relevance. The resulting system is not only able to adapt to current trends and preferences but also improves in accuracy over time. This feature is particularly advantageous in dynamic retail environments where customer preferences and product offerings often change.

[0050] In one example, the data (or in-store data) further includes wearer-specific information such as personal data, Internet of Things data, wearer preferences, and the local neural network fully utilizes the wearer-specific information to enhance the accuracy and personalization of service provision.

[0051] Fully utilizing wearer-specific information (such as personal data and preferences) enhances the personalization and accuracy of service provision. This approach enables a more customized shopping experience, directly addressing the unique needs and preferences of each wearer. In environments such as customized eyewear or personalized health device retail, this ability can significantly enhance customer satisfaction.

[0052] In one example, the data (or in-store data) includes characteristics of a vision impairment control solution used by the wearer or potential wearer, and the predictive and personalized service offering includes determining future values of the visual characteristics of the wearer or potential wearer.

[0053] In one example, the vision impairment is myopia.

[0054] In one example, the data (or in-store data) includes characteristics of a hearing impairment control solution used by the wearer or potential wearer, and the predictive and personalized service offering includes determining future values of the auditory characteristics of the wearer or potential wearer. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Depicts an overview of a system suitable for facilitating the deployment of applications in a retail environment of a head-mounted device in an exemplary embodiment.

[0056] Figure 2 Depicts the general structure of an artificial neural network in an exemplary embodiment.

[0057] Figure 3 Depicts in an exemplary embodiment Figure 1 The communication scheme between the elements shown.

[0058] Figures 4 to 6 Depicts various scenarios in which the treatment efficacy of myopia control can be simulated using federated learning in an exemplary embodiment. DETAILED DESCRIPTION

[0059] The present disclosure focuses on methods and systems for facilitating the deployment of applications in a retail environment of a head-mounted device.

[0060] "Retail environment" can include not only physical retail environments such as stores, experience centers, trade shows, expos, service and support centers, but also virtual or augmented reality spaces, online stores, and mobile commerce platforms.

[0061] "Head-wearable device" can be any device wearable on the head. In the context of the present disclosure, this can include, but is not limited to, glasses, head-mounted headphones, helmets, virtual reality devices, augmented reality devices, holographic displays, direct neural interfaces, etc. The head-wearable device can include one or more optical lenses. An optical lens is a transparent substance with curved sides for focusing or dispersing light.

[0062] "Retail environment of a head-mounted device" should be understood as the retail environment in which the head-mounted device is located or can be sold, displayed, or experienced by customers.

[0063] "Application" refers to a computer program, service, or, in a broader sense, a software that performs one or more functions for an end user or another application. The term "application" shall also include software suites, mobile applications, web-based platforms, and any future form of software technology yet to be developed. It can be standalone or part of a larger system.

[0064] "Deployment" shall be understood to include the initial setup and / or ongoing operations, maintenance, and upgrades of an application. This can cover cloud-based services, in-store servers, or distributed computing environments. Deployment can involve integration with existing retail management systems, point-of-sale systems, or e-commerce platforms. This may include virtual try-on applications, customer relationship management (CRM) systems, or inventory tracking applications.

[0065] The main element disclosed therein relates to a computing device or system suitable for participating in the deployment of an application in a retail environment.

[0066] The designation "computing device or system" encompasses a wide range of computing hardware and can also refer to virtual computing machines.

[0067] The computing device or system includes at least an interface module configured to support two-way communication between a local neural network and a global neural network as both a signal transmitter and a signal receiver through any kind of data transmission technology.

[0068] These signals encapsulate data related to facilitating the co-operation of the local neural network and the global neural network and / or facilitating the co-operation of the global neural network and the local neural network. More specifically, the interface module is configured to enable the local neural network to participate in the federated learning process with the global neural network through the transmitted and received signals.

[0069] The "local neural network" is associated with a local area of the network, whether from a geographical or topological perspective, while the "global neural network" is associated with the entire network or at least with multiple local areas of the network. The distinction between the "local neural network" and the "global neural network" should be flexible to allow for various configurations and hierarchies of the network. For example, the local network can be specific to a store, while the global network can cover an entire retail chain, even including networks from different companies participating in data sharing for mutual benefit.

[0070] The computing device is further configured to implement, process, or manage the local neural network.

[0071] In its dual-functional role, the computing device not only implements, manages, and processes a local neural network but also serves as a central communication link through its interface module. The interface module specifically acts as a bridge to facilitate the interaction between the computing device and another external entity, such as another computing device or system. The task of this external entity is to implement, manage, or process a global neural network. Thus, while the computing device is directly involved in the complex processing and operation of the local neural network, its interface module extends its functionality by enabling seamless two-way communication with the global neural network, thereby strengthening the computing device's core role in a broader interconnected system.

[0072] The computing device or system may further include a non-transitory storage medium that stores at least the data included in the signals to be sent through the interface module and / or the data included in the signals that have been received through the interface module. The non-transitory storage medium may further store additional data that is not included in any signals to be sent using the interface module and is not included in any signals that have been received using the interface module.

[0073] The computing device or system may further include a processing unit configured to participate in managing the local neural network at least in cooperation with the non-transitory storage medium and the interface module.

[0074] The computing device or system may further include other elements, including but not limited to an input module and / or an output module. A human-machine interface is an example of an interface that can serve as an input module and / or an output module.

[0075] In the context of the present disclosure, examples of the computing device or system include a terminal in a store or shop, a gateway device in the store or shop, and / or a server that coordinates data across a local group of stores or shops.

[0076] By implementing the local neural network, the computing device or system processes and analyzes data related to the wearer or potential wearer of a head-mounted device for predictive and personalized service provision.

[0077] "Data" is data associated with one or more locations within a larger group of locations. These locations can be, for example, stores (such as physical stores or virtual stores that allow products to be sold on an online marketplace), ECP offices, hospitals, or a user's home when the user directly collects or uses data at home. "Data" can encompass data generated within the physical scope of a store and / or data related to the operation of the store, which can be generated online or through other channels. This can include online customer reviews related to store products, virtual fitting room data, or interactions with social media. In relation to the wearer or potential wearer of a head-mounted device, in-store data has a logical association with one or more individuals who wear or are interested in wearing a head-mounted device. In-store data can refer to a wide range of data, including one or more of the following: biometric data (such as head size and shape, interpupillary distance), eye-tracking data, visual acuity test results, prescription data, eye health assessments, usage patterns, preference data, performance data, purchase history, demographic data, feedback data, interaction data, environmental or situational data, technology compatibility data, behavioral data (such as non-verbal cues), etc.

[0078] "Data" can also include data related to parameter values of vision impairment control solutions that are used or can be used by the wearer.

[0079] Vision impairment control solutions can be selected from one or more of the following elements:

[0080] • Myopia control spectacle frames,

[0081] • Myopia control contact lenses,

[0082] • Orthokeratology contact lenses,

[0083] • Treatment using atropine,

[0084] • Treatment using red light therapy, and

[0085] • Without using any vision impairment control solution.

[0086] "Data" can also include data related to parameter values of hearing impairment control solutions that are used or can be used by the wearer.

[0087] Hearing impairment control can be selected from one or more of the following elements:

[0088] • Hearing aids,

[0089] • Cochlear implants,

[0090] • Bone-anchored hearing systems,

[0091] • Assistive listening devices

[0092] • Personal sound amplification products,

[0093] • Tinnitus maskers,

[0094] • Hearing rehabilitation programs,

[0095] • Ambient sound amplifiers,

[0096] • Hearing protection with enhanced features such as selective amplification and / or noise reduction capabilities.

[0097] In the context of the present disclosure, the application utilizes or takes full advantage of predictions output by a local neural network for a wearer or potential wearer of a head-mounted device to provide personalized services. For example, the personalized service can be guidance for an end user (such as a wearer or potential wearer or an ophthalmic care practitioner), or it can be an input for another in-store application targeted at the end user. Examples of services include fitting optimization, virtual try-on enhancement, and visual / auditory characteristic prediction.

[0098] Now refer to Figure 1 , which depicts an exemplary computing system according to the present disclosure.

[0099] The computing system includes a local computing device or system (120) implementing a local neural network, and further includes a central computing device or system (130) implementing a global neural network. The central computing device or system can be one or more remote servers. For example, the global neural network can be implemented as an application in a cloud-based environment. The global neural network can be configured to communicate with other data sources (150), such as other local neural networks, a cloud database storing anonymized data from multiple wearers, or an external API providing real-time weather data.

[0100] The local computing device or system and the central computing device or system are configured to communicate with each other using a communication channel or communication link. Such a communication link is symbolically represented by a pair of interface modules (124, 132).

[0101] The local computing device or system further includes an input module (122) configured to obtain, receive, collect, or access in-store data from one or more data sources (such as, for example, one or more databases (112), one or more sensors (114), and / or one or more human-machine interfaces (116)), while the local neural network uses this data to build a local model. The input module is designed to accommodate a wide variety of inputs from these data sources.

[0102] A sensor or sensing module is a device or part of a device that can detect and respond to a certain type of input from the physical environment. In the context of the present disclosure, the sensor may include one or more cameras for capturing images of the wearer or potential wearer or images of the environment. These sensors may further include an ambient light sensor for analyzing lighting conditions, a proximity sensor for detecting nearby objects, an accelerometer for measuring the direction and speed of head movement, an eye tracker (especially an infrared eye tracker) for identifying the direction of gaze, a time-of-flight sensor for sensing the gaze distance, and the like.

[0103] A human-machine interface is a device or software that allows a human to interact with a machine. In the context of the present disclosure, the human-machine interface may be adapted to receive, for example, tactile input or auditory input (such as voice commands). Examples of other suitable human-machine interfaces include a gesture recognition system and a graphical user interface on a connected device. For example, some human-machine interfaces may allow the provision of declarative data about the wearer (such as their age, gender, laterality, facial features, prescription, etc.), or the retrieval of compiled data about the frame of the head-mounted device (referred to as "frame framing data"), all of which data can be provided as input to the system.

[0104] The local computing device or system further includes an output module (126) configured to output or provide a prediction by processing in-store data using a local model.

[0105] A possible use case for a local neural network is for fitting optimization. Fitting optimization involves at least determining the contour of the eyewear frame on the wearer as presented in an input image in order to derive at least one fitting parameter of the eyewear frame on the wearer from said contour. The above-mentioned term "fitting parameter" will be interpreted in a broad sense. A fitting parameter can generally refer to the position of a specific part of the eyewear frame relative to the wearer's face, such as, for example, the eyewear bridge on the wearer's nose and / or the position of the eyewear temple relative to the wearer's temple, etc. When the wearer wears the eyewear frame, the fitting parameter can also refer to fitting parameters in the field of optometry, such as pupillary distance, fitting height, position of the center of rotation of the eye, rake angle, wrap angle, eye-lens distance, and / or center of rotation of the eye-lens distance, etc. The contour of the eyewear frame is also commonly referred to as a "bounding box". Outlining the frame (and / or the lenses in the frame) is cumbersome for an ECP. To do it faster, errors or at least inaccuracies may occur, and thus the quality of the determination of the fitting parameters may be poor. For this reason, it is advantageous to perform an automatic "frame detection" process that involves processing an image in which the frame is visible in order to identify the position and shape of the frame. Some known frame detection processes involve artificial intelligence tools that are trained to provide a bounding box as an output when an image of a wearer wearing an eyewear frame is provided as an input. In the use case according to the present disclosure, a combination of a high-resolution camera, a 3D scanner, and an accelerometer can be used as sensors (114) to provide in-store data. These sensors can collect detailed images of the wearer's face, precise measurements of the head shape and size, and head movement patterns. Using these sensors, digital measurement tools, and a human-machine interface (116), an eye care practitioner (ECP) can take pictures and modify the lens frame fixation points in order to fit the lenses to the wearer's or potential wearer's face. A database (112) can store pictures, lens frame fixation points, wearer-specific data, ECP-specific data, etc. The in-store data from the above data sources is provided as an input to a local model that can be trained to process the in-store data and thus provide the lens frame fixation points as an output. Therefore, the output of the local model can be used to automatically adjust and detect fitting measurements, optimize the actual and virtual frame fitting, and / or to identify design problems, thus contributing to a richer lens configuration model. The local model can learn from the adjustments made by the ECP to better determine the lens frame fixation points and may also take into account other in-store data, such as wearer-specific data (such as the face of each wearer or potential wearer), store-specific data (such as lighting conditions and spatial sensor arrangements), ECP-specific data indicating the operating habits of each ECP, etc.

[0106] Another possible use case for the local neural network is for virtual try-on enhancement. In this use case, cameras and environmental sensors in kiosks can be used to collect user images of the wearer or potential wearer as in-store data. The lighting, contrast, and background of different kiosks may vary, which means that the in-store data from different kiosks has systematic deviations. Each kiosk can further provide a virtual try-on application that is configured to use the user image of a specific user (whether a wearer or a potential wearer) to generate a virtual view of the user wearing a synthetic frame. Then, the local model can learn from the collected user images and may adjust for image characteristics such as contrast and brightness.

[0107] Another possible use case for the local neural network is to determine at least one future value of at least one visual characteristic of a wearer based on the type and value of the characteristics of the vision impairment control solution used by the wearer.

[0108] The visual characteristics of the wearer can be any one of the following elements:

[0109] - The spherical refractive error of the right or left eye of the wearer,

[0110] - The spherical equivalent of the right or left eye,

[0111] - The spherical equivalent of the right or left eye,

[0112] - The diopter of the right or left eye,

[0113] - The prescription of the right or left eye,

[0114] - The axial length of the left or right eye,

[0115] - The average of the axial length of the left eye and the axial length of the right eye,

[0116] - The corneal parameter of the left or right eye,

[0117] - An indication of myopia in the left or right eye, and

[0118] - The change over time of at least one of the spherical refractive error, monocular spherical equivalent, binocular spherical equivalent, diopter, prescription, axial length, corneal parameter, or indication.

[0119] Another possible use case for the local neural network is to determine at least one future value of at least one auditory characteristic of a wearer based on the type and value of the characteristics of the hearing impairment control solution used by the wearer.

[0120] The hearing characteristics of the wearer can be any one of the following factors:

[0121] Auditory threshold level

[0122] Speech recognition score,

[0123] Frequency-specific hearing loss,

[0124] Dynamic range,

[0125] Tinnitus characteristics

[0126] Difference between bone conduction level and air conduction level,

[0127] Loudness growth, and

[0128] Hearing aid usage pattern.

[0129] In these cases, at least one physiological parameter of the wearer can also be considered. The physiological parameter can be:

[0130] Age of at least one model wearer or the wearer,

[0131] Gender of at least one model wearer or the wearer,

[0132] Race of at least one model wearer or the wearer,

[0133] Location of at least one model wearer or the wearer,

[0134] Number of myopic parents of at least one model wearer or the wearer,

[0135] Duration of near work achieved by at least one model wearer or the wearer, and

[0136] Length of time spent outdoors by at least one model wearer or the wearer.

[0137] All of these embodiments shown in the previously described use cases can be combined. For example, an image of a virtual frame can be output as a replacement for the image of the real eyeglass frame worn by the wearer on the input picture of the wearer. To this end, several consecutive method steps can be considered: identifying the contour of the real eyeglass frame worn by the wearer in the input picture, determining the anchor points of the real eyeglass frame on the wearer's face, and using the anchor points to place the image of the virtual eyeglass frame in the input picture. This scenario demonstrates the possibility of training a local neural network for a simple individual task or for a single combined task. For example, the local neural network can be trained only to identify the contour of the real eyeglass frame worn by the wearer in the input picture (as in the first-mentioned use case), or it can also be trained only to use the anchor points as further input data in order to place the image of the virtual eyeglass frame in the input picture. However, the latter requires the determination of the anchor points in a preprocessing step, which can be done completely automatically by any suitable technical means or through human-computer interaction. Alternatively, assuming a sufficient amount of training data and computing resources, the local neural network can be trained to directly generate an output picture corresponding to the input picture in a "black box" manner, replacing the real frame with the image of the virtual eyeglass frame without explicitly identifying the contour of the eyeglass frame or determining the anchor points of the real eyeglass frame on the wearer's face.

[0138] Although operating in a similar architecture to the local neural network, the global neural network is configured to build its global model using a different set of data inputs (150). Notably, this includes using historical models built from aggregated data similar to the data collected by the input modules of various local computing devices or systems. This aggregated data represents the cumulative insights extracted from multiple local models, each of which is generated from in-store data within its respective retail environment.

[0139] The local neural network participates in the federated learning process with the global neural network, thus allowing the two neural networks to refine and update their respective models. Federated learning is a machine learning method in which models are trained on multiple decentralized devices or servers that hold local data samples without exchanging this data. This method maintains data privacy and reduces the need for data centralization.

[0140] Optionally, the global neural network can be interconnected with a number of local neural networks, acting as a central node in a broader networked ecosystem. In this configuration, the global neural network may initially be one of the local neural networks that has evolved or been designated to serve as the global network. This unique position enables it to utilize and synthesize a wide variety of data patterns and trends from across the network, thereby enhancing its predictive capabilities.

[0141] By fully leveraging the data across the entire network, the global neural network can refine and update its global model, benefiting from a broad perspective that encompasses various retail environments and consumer interactions. This approach ensures that the learning process is enriched by a diverse and comprehensive dataset while adhering to the principles of federated learning, which do not exchange actual data samples, thus safeguarding individual data privacy and reducing the need for centralized data storage.

[0142] Now refer to Figure 2 , which provides the general structure of an artificial neural network in an exemplary embodiment. This general structure applies to both local neural networks and global neural networks. It is a simplified representation designed to convey the key elements of the network structure rather than an accurate description of the network complexity in actual applications.

[0143] The illustrated artificial neural network includes multiple layers of neurons. At the ends, there is an input layer (210) and an output layer (250), with one or more hidden layers (230) sandwiched between them. Each of these layers contains a number of neurons (212) interconnected with each neuron in the adjacent layer. These interconnections are referred to as "weights".

[0144] Artificial neural networks are commonly used to create and adjust models. The training phase involves providing labeled training data as input to the input layer. In machine learning, "training data" refers to the raw data from which the model learns, and "labels" refer to the corresponding results or categories that the model is trained to "predict". Weights are numerical parameters in the neural network that are adjusted during the training phase to improve the predictions output by the output layer. After the training phase is completed, the artificial neural network enters the production phase to receive raw data that is not necessarily labeled as input and output the corresponding predictions.

[0145] For clarity of presentation, Figure 2 only a few layers and a small number of neurons are shown. In reality, a neural network contains more layers, densely populated with a large number of interconnected neurons, creating countless weights or connections that far exceed what can be represented by a single schematic diagram.

[0146] In the present disclosure, the term "local weight" refers to the connection between two neurons within a local neural network, while "global weight" similarly refers to the connection within a global neural network. Generally, there is a one-to-one correspondence between local weights and global weights, meaning that the structure of the local neural network (including the number of layers, the number of neurons within each layer, and the function of each neuron) reflects the structure of the global neural network. However, it should be noted that this one-to-one correspondence is not strictly necessary for the operation of the system, as long as the local network and the global network share a compatible structure that allows the global weights to be mapped or transformed into the local network environment.

[0147] Now refer to Figure 3 , which provides a communication scheme between the components shown in the exemplary embodiments. Figure 1 The communication scheme between the components shown.

[0148] It can be considered that at a given moment, the input module (122) has obtained (302) various in-store data through one or more interactions with one or more human-machine interfaces, or through one or more queries to one or more databases, or through the normal operation of one or more sensors, including specific in-store data related to a specific wearer or potential wearer.

[0149] The obtained in-store data is sent to the processing unit of the local computing device or system (120) and processed using the local model. Specifically, this means feeding the in-store data associated with a specific individual (wearer or potential wearer) as input into the input layer (210) of the local neural network. By using the local model to process the obtained in-store data, the output module (126) obtains (310) personalized recommendations for a specific wearer or potential wearer. This means that by feeding the in-store data associated with a specific individual as input into the input layer (210) of the local neural network, the output layer (210) of the local neural network generates personalized recommendations for that specific individual.

[0150] Then, both the in-store data and the personalized recommendations can be associated with a specific individual and, from the perspective of the local model, can be considered to form a labeled data set similar to the labeled training data. In this case, the personalized recommendation is the label.

[0151] Then, when new in-store data is provided, the local model can continuously learn and provide new personalized recommendations, and through continuous learning, the local weights can continuously evolve.

[0152] To better understand the input and output in the context of such a system, reference can be made to the two use cases already mentioned.

[0153] In the use case related to fitting optimization, the input may include detailed images of the wearer's face, measurements of the head shape and size, head movement patterns, and lens frame fixation points adjusted on the captured picture of the wearer's face through ECP, etc. The output may include optimized lens frame fixation points, adjusted fitting measurements, or identified design problems.

[0154] In use cases related to virtual try-on, the input may include user images of the wearer or potential wearer, and data derived from these user images, such as body attributes and environmental conditions. The input may further include other in-store data, such as user-related data, such as age, prescription, user preferences, wearing patterns, activity patterns, etc. The output may include renderings, such as virtual try-on images adjusted for some image characteristics such as contrast and brightness.

[0155] In both of these use cases, the input mainly consists of detailed in-store data related to the wearer's body attributes and environmental conditions captured through various sensors and interfaces. These outputs are customized recommendations and enhancements - in the case of fitting optimization, they are precise lens frame fixation points and fitting measurements; for virtual try-on, they are real virtual views with synthetic frames adjusted for environmental variables.

[0156] However, the present disclosure is not limited to these specific use cases.

[0157] More generally, a local model can accept many types of in-store data related to the wearer or potential wearer as input, which can be provided in various formats (as variables, text, images, audio signals, video signals, etc.), and describe information including but not limited to the following:

[0158] User demographic information and preferences, such as the wearer's age, gender, and occupation, personal style preferences (modern, classic, sport, etc.), preferred frame materials (metal, plastic, composite, etc.), or color preferences for frames and lenses,

[0159] Optical measurements and eye health data, such as prescription details (spherical power, cylindrical power, axial length, etc.), pupil distance and other eye measurements, information about any specific eye diseases (astigmatism, presbyopia, myopia, etc.), or previous glasses prescriptions and their comfort levels,

[0160] Lifestyle and usage data, such as information about the wearer's daily activities and environment (outdoor, office work, screen exposure, etc.), specific needs (such as UV protection, anti-reflective coating, or blue light filtering), or requirements based on sports or other activities,

[0161] Biometric and physiological data, such as head shape and ear-to-nose measurements, skin color, eye color, and facial features, or any allergies or sensitivities to certain materials,

[0162] Historical data and feedback, such as previous purchase records and their feedback, changes in prescriptions over time, or the wearer's adjustment and repair history.

[0163] The local model can accept as input many types of in-store data that are specific to the store rather than to a particular wearer or potential wearer. Such store-specific data can describe environmental factors, equipment and technology specifications, operating practices, store layout and design, environmental conditions, digital and virtual interface settings, and so on.

[0164] The output of the model can include various types of recommendations, which can be provided in various formats (as variables, text, images, audio signals, video signals, etc.) and describe information including but not limited to the following:

[0165] Frame recommendations, such as suggested frame styles that match the wearer's personal style, face shape, and color preferences, or recommendations for frame materials based on skin sensitivity and lifestyle needs,

[0166] Lens type and feature recommendations, such as specific lens types or settings (single vision, bifocal, progressive, etc.) based on prescription and age, recommendations for lens coatings and treatments (anti-reflective, scratch-resistant, UV protection coatings), or suggested tint colors and intensities based on usage scenarios (e.g., photochromic lenses for outdoor use),

[0167] Customized eyewear solutions, including customized recommendations for eyewear suitable for specific activities or occupations (e.g., computer glasses, sports glasses), solutions for complex prescriptions or unique eye diseases, or adaptive recommendations based on changes in prescription or eye health over time,

[0168] Interactive features, such as virtual try-on renderings with recommended frames and lenses, or simulations showing visual differences brought about by various lens treatments,

[0169] Health and / or comfort optimization, such as recommendations for glasses that minimize eye strain or fatigue (especially important for lifestyles involving frequent screen use), or recommendations for glasses that meet specific eye health conditions.

[0170] Since some wearer-specific inputs may be affected by store-specific conditions, providing in-store data that describes store-specific conditions as input to the local model can enable the model to adjust the personalized recommendations of the output accordingly.

[0171] For example, in-store data related to the wearer or potential wearer is sourced from a specific, limited group of wearers or potential wearers. This group mainly includes individuals who have visited or participated in the store, whether in a physical or virtual environment. Therefore, this data reflects the experiences, preferences, and interactions of a specific subset of customers, providing a targeted and relevant basis for the training of the local neural network and subsequent recommendations.

[0172] Similarly, in-store data sources related to an ECP are from a single ECP or a small group of ECPs working in or for a store.

[0173] Similarly, in-store data is sourced from one or more sensors located in the store, and this data may have systematic biases. For example, when the sensor is a camera, several factors may cause such systematic biases, such as lighting conditions, the quality and clarity of the camera, the camera angle and positioning, the distance of the object from the camera, the store layout and design, color calibration, and image processing settings.

[0174] To enhance personalized recommendations provided to wearers or potential wearers, promoting local models among a larger group of wearers or potential wearers is proposed.

[0175] This promotion can be performed asynchronously, e.g., concurrently or periodically. Alternatively, this promotion can be performed according to a predefined scheme regarding obtaining in-store data, e.g., whenever a new in-store data volume exceeding a predefined threshold is obtained since the last executed promotion.

[0176] The promotion process involves a series of basic actions, which are now described in an exemplary scenario where multiple local computing devices or systems (120) each implement a corresponding local neural network for a specific store, and each local computing device or system is configured to communicate with the same central computing device or system (130).

[0177] First, the weights inferred from the local neural networks (which encapsulate the correlations between the accumulated training data and their respective labels) are sent (306) by the corresponding local computing devices or systems (120) to the central computing device or system (130) implementing the global neural network. The central computing device or system (130) may also receive (304) relevant data from other sources (150), such as a centralized database storing demographic information about wearers, environmental data, or information collected from other devices in the network.

[0178] A generalized model is constructed by the central computing device or system (130). This may involve selecting a subgroup of stores or local neural networks for the promotion step and calculating the average of the neural network weights from the selected stores or local neural networks to establish a global neural network model. Based on the federated learning (FL) principle, this model represents the collective wisdom of the selected stores or local neural networks.

[0179] Subsequently, the global weights from the global model are sent back (308) to the respective local computing devices or systems.

[0180] Upon receipt, these global weights are integrated into the local model, resulting in an updated local model. This can involve selectively replacing certain local weights while preserving others. Various methods can be utilized for this purpose, such as sending only the global weights designated for replacing local weights, or transmitting a broader set of global weights and using a filtering mechanism at the local neural network module to select specific weights for replacement. Whichever approach is taken, the modified local model constitutes a personalized adaptation of the generalized model, thus retaining the unique aspects of the wearer while incorporating insights from the global neural network.

[0181] Finally, personalized recommendations (310) can be determined based on the updated local model. This approach enables each local neural network to remain adaptable to specific local in-store data while dynamically adapting to new situations, thereby leveraging the collective learning accumulated from a broader set of wearers or potential wearers.

[0182] The present disclosure further introduces an enhancement to eye disease management, leveraging federated learning to improve the predictability and efficacy of interventions tailored to individual needs.

[0183] The field of eye disease management encompasses a wide range of developmental eye diseases such as myopia, hyperopia, astigmatism, and presbyopia. Management strategies are determined by eye care practitioners (ECPs) and typically involve interventions aimed at controlling or mitigating the progression of these diseases in specific individuals.

[0184] Known myopia control management tools have been provided as aids to assist ECPs in determining specific management strategies for particular individuals. These known myopia control management tools utilize various strategies to demonstrate the potential benefits of myopia control. These tools typically present the progression of a patient's myopia over time by plotting the evolution of the spherical equivalent based on standard myopia control treatment efficacies obtained from the literature. However, these techniques may not be fully suitable for iterative decision-making as they rely on static treatment efficacies that do not account for the variability of individual patients and do not adapt to changes occurring over time. Additionally, data aggregation in centralized systems used for these predictions poses risks to privacy and limits the dynamic updating of treatment plans.

[0185] It is further known that machine learning algorithms are employed to predict the progression of eye diseases based on various inputs such as refractive state, age, gender, and other demographic or environmental factors. These prediction models use longitudinal and cross-sectional data to predict changes in eye diseases and can be applied using various classical machine learning techniques. For example, some methods use linear prediction models such as support vector regression (SVR) and Gaussian process regression (GPR), which analyze data collected from various sources to predict outcomes. However, these techniques require the aggregation of potentially sensitive data at a central location, which may raise concerns about data privacy and be prone to leakage. Additionally, this approach generally cannot achieve real-time updates and cannot easily integrate new data without a full reprocessing, which limits its utility for continuous patient care.

[0186] The disclosure herein explores a transformative approach of federated learning for ophthalmic care practitioners (ECPs) to predict the effectiveness of various myopia control interventions. Different from traditional centralized data processing methods, federated learning enables the secure processing of data across a network of devices, thus enabling common insights to be obtained without compromising patient privacy.

[0187] This involves using a federated network for predictive analysis of treatment efficacy, ensuring that sensitive data remains within local devices. The application of this method enables dynamic adjustments based on real-time data from multiple users, thereby enhancing the adaptability and accuracy of myopia management programs.

[0188] Federated learning not only ensures the security of patient data but also enhances management programs by integrating new data points from continuous patient interactions and treatments.

[0189] While the primary focus has been on myopia, the principles of federated learning can be widely applied to other eye diseases. The generality of federated learning enables it to adapt to situations where customized interventions can significantly benefit from predictive modeling.

[0190] The predictive capabilities of federated learning are used to evaluate the efficacy of interventions across a range of eye diseases, which involves adjusting treatments based on the predicted rate of progression or the patient's response to certain therapies.

[0191] As federated learning continuously integrates new data, it facilitates dynamic adjustments to treatment plans, such as quickly adjusting interventions for presbyopia when the initial strategy shows suboptimal efficacy.

[0192] By leveraging networks that learn from a wide variety of data inputs, federated learning supports a more personalized approach to ophthalmic care, thereby optimizing resource utilization and enhancing patient treatment outcomes.

[0193] In addition, federated learning supports the development of decision-making tools that help practitioners select the most effective treatment plan based on predictive models that simulate various outcomes.

[0194] The integration of the federated learning model with existing electronic health record (EHR) systems ensures seamless access to predictive insights during patient visits, thus enhancing the decision-making process for single and / or combined interventions.

[0195] As Figures 1 to 3 shown, the general description of how to implement federated learning in a retail environment similarly applies to any service environment, and particularly to an eye health service environment.

[0196] When implementing federated learning in an eye health service environment, the same steps are implemented: collecting data, processing and analyzing data, and network communication between the local neural network and the global neural network.

[0197] The differences only relate to the nature of the data collected, processed, analyzed, and sent.

[0198] Data collection involves obtaining specific data related to eye diseases of one or both eyes of an individual through the input module of a computing device. This data can include, but is not limited to, biometric data, health metrics, environmental exposures, and historical health records.

[0199] The data collected is processed and analyzed by the local neural network within the computing device. This local processing enables dynamic adaptation of the model based on local data inputs without compromising the privacy of individual data.

[0200] The local neural network communicates with the global neural network via an interface module. This communication involves the sending and receiving of signals that facilitate the federated learning process, thus enabling enhanced prediction accuracy through collective learning from a variety of data sources.

[0201] Finally, the services provided are predictive and personalized, and utilize the analyzed data to predict the efficacy of candidate treatments for specific diseases of an individual's eye, thus assisting healthcare providers in making informed decisions.

[0202] Specific examples of data related to eye diseases can include, but are not limited to, one or more of the following elements, or combinations thereof: age, prescriptions, visual acuity measurements, intraocular pressure readings, retinal scan results, corneal topography data, patient-reported symptoms, historical treatment responses, genetic information related to eye health, environmental and lifestyle data.

[0203] Data related to eye diseases can include immediate or recent measurements and observations, such as the most recent visual acuity measurement indicating current visual clarity.

[0204] Data related to eye diseases can include historical or long-term data that tracks the evolution of one or more eye diseases over time, such as the progression of prescription changes over the years.

[0205] Data related to eye diseases (whether current and / or historical) form a multi-dimensional array that is fed into a local neural network

[0206] Specific examples of candidate treatments can include, but are not limited to, one or more of the following elements, or combinations thereof: pharmaceutical treatments (such as eye drops for glaucoma), surgical interventions (such as LASIK or cataract surgery), wearable vision correction devices (such as glasses or contact lenses), phototherapy (such as photodynamic therapy), regenerative medicine methods (such as stem cell injections).

[0207] Now regarding Figures 4 to 6 Describe an exemplary embodiment. In this embodiment, the individual's eye disease is myopia, and the efficiency of candidate treatments is evaluated based on their ability to control the evolution of myopia over time.

[0208] In the context of myopia management, the data collected can include, but are not limited to, one or more of the following elements, or combinations thereof: the historical progression of myopia (in diopters) over time, measurements from an autorefractor or similar device for tracking changes in refractive error, lifestyle data (such as time spent on near work versus outdoor activities), genetic factors that may make an individual predisposed to myopia, response to previous myopia control treatments, such as the use of atropine or orthokeratology.

[0209] Candidate treatments specifically targeting myopia may involve, but are not limited to, one or more of the following elements, or combinations thereof: prescribing corrective lenses with specific optical properties to slow progression, administering low-dose (e.g., 0.01%) atropine eye drops, using overnight-worn orthokeratology lenses to reshape the cornea, recommendations for increasing outdoor activities based on epidemiological data.

[0210] Figures 4 to 6 These concepts are demonstrated by showing various scenarios of how federated learning can be used to simulate the treatment efficacy of myopia control based on the integration of a wide variety of data inputs. These figures depict how different treatment strategies may alter the progression of myopia over time, thus providing an intuitive and quantifiable way for ECPs to evaluate the potential benefits of each treatment option and make adjustments based on real-time data feedback.

[0211] Figure 4Depicts for an individual's eye:

[0212] Measurements of the spherical equivalent 15 (in diopters) and axial length 25 (in mm) at the first time t = 0 months,

[0213] Predicted values of the spherical equivalent 11 and axial length 21 at 3 months, 6 months, and 12 months from the first time t, under the assumption of uncorrected myopia and no application of any myopia control strategies,

[0214] Predicted values of the spherical equivalent 12 and axial length 22 at 3 months, 6 months, and 12 months from the first time t, under the assumption of correcting myopia by prescribing corrective lenses and no application of any myopia control strategies,

[0215] Predicted values of the spherical equivalent 13 and axial length 23 at 3 months, 6 months, and 12 months from the first time t, under the assumption of applying the first myopia control strategy,

[0216] Predicted values of the spherical equivalent 14 and axial length 24 at 3 months, 6 months, and 12 months from the first time t, under the assumption of applying the second myopia control strategy.

[0217] For example, both the first myopia control strategy and the second myopia control strategy involve prescribing corrective lenses with specific optical properties to slow down progression, and the second myopia control strategy further involves prescribing 0.01% atropine eye drops.

[0218] Figure 5 Depicts for an individual's eye, after the first myopia control strategy has been applied between the first time t and the second time t' = 3 months after the first time t:

[0219] Measurements of the spherical equivalent 35 (in diopters) and axial length 45 (in mm) at the first time t and the second time t',

[0220] At Figure 4 Under the same assumptions as in

[0221] At Figure 4 Under the same assumptions as in

[0222] i.e., myopia is corrected but no myopia control strategies are applied, predicted values of the spherical equivalent 31 and axial length 41 at 3 months, 6 months, and 12 months from the first time t,

[0221] At Figure 4 Under the same assumptions as in

[0222] i.e., myopia is corrected by prescribing corrective lenses and no myopia control strategies are applied, predicted values of the spherical equivalent 32 and axial length 42 at 3 months, 6 months, and 12 months from the first time t,After applying a first myopia control strategy between a first time t and a second time t', and assuming that the first myopia control strategy is continued after the second time t', updated predicted values of the equivalent spherical refractive power 33 and the axial length 43 at 6 months and 12 months from the first time t.

[0223] After applying a first myopia control strategy between a first time t and a second time t', and assuming that a second myopia control strategy is applied after the second time t', updated predicted values of the equivalent spherical refractive power 34 and the axial length 44 at 6 months and 12 months from the first time t.

[0224] Figure 6 Describes, for an individual's eye, after applying a first myopia control strategy between a first time t and a second time t' = 3 months after the first time t and after applying a second myopia control strategy between the second time t' and a third time t'' = 6 months after the first time t:

[0225] Measured values of the equivalent spherical refractive power 56 (in diopters) and the axial length 66 (in mm) at the first time t, the second time t' and the third time t''.

[0226] At Figure 4 And Figure 5 Under the same assumptions as in

[0227] At Figure 4 And Figure 5 Under the same assumptions as in

[0228] After applying a first myopia control strategy between a first time t and a second time t', and after applying a first myopia control strategy between the second time t' and a third time t'', and assuming that the first myopia control strategy is continued after the third time t'', updated predicted values of the equivalent spherical refractive power 53 and the axial length 63 at 12 months and 18 months from the first time t.

[0229] After a first myopia control strategy has been applied between a first time t and a second time t', and after a second myopia control strategy has been applied between the second time t' and a third time t'', under the assumption that the first myopia control strategy is applied after the third time t'', updated predicted values of the equivalent spherical power 54 and the axial length 64 at 12 months and 18 months from the first time t, and

[0230] After a first myopia control strategy has been applied between a first time t and a second time t', and after a second myopia control strategy has been applied between the second time t' and a third time t'', under the assumption that the second myopia control strategy is continued to be applied after the third time t'', updated predicted values of the equivalent spherical power 55 and the axial length 65 at 12 months and 18 months from the first time t.

[0231] This general description is intended to present exemplary embodiments of the invention. Variations, modifications, and substitutions will be apparent to those skilled in the art, and these can be made without departing from the scope of the invention. The specific configurations of the components and the way they interact are merely illustrative, and alternative configurations and interactions are within the scope of the appended claims.

[0232] In accordance with this general description, the following specific embodiments are used to further illustrate the proposed invention. These embodiments correspond to different use cases, and each embodiment presents a unique method of operating a personalized smart eyewear. Each embodiment covers Figure 3 the same general process already depicted therein, which generally involves data collection, model creation, model generalization, and the actual application of the model to various functions in the form of personalized recommendations.

Claims

1. A computing device, comprising: an interface module configured to send and receive signals between the local neural network and the global neural network, in: The local neural network implemented by the computing device is configured to process and analyze data related to a wearer or potential wearer of the head wearable device for predictive and personalized service provision, and The interface module enables the local neural network to participate in a federated learning process with the global neural network through sent and received signals.

2. A method comprising: collecting data related to a wearer or potential wearer of the head wearable device through an input module of the computing device, processing the in-store data via a local neural network within the computing device for use in providing at least one predictive and personalized service, Sending and receiving signals between the local neural network and the global neural network through an interface module in the computing device, Wherein, the local neural network participates in a federated learning process with the global neural network, and the federated learning process is facilitated by the sending and receiving of the signal.

3. The method of claim 2, wherein: The data includes an image captured by an imaging device, the image depicting a wearer or potential wearer during an actual fitting of the head wearable device, Processing the data includes determining frame points corresponding to an outline of the head wearable device in the image, wherein the federated learning process enhances the determination of the frame points.

4. The method of claim 2, wherein: The data includes real-time or static facial data of the potential wearer, Processing the data includes generating a simulated appearance of the potential wearer wearing the head wearable device based on the facial data, and performing a virtual try-on process, wherein the federated learning process enhances the rendering of the simulated appearance.

5. The method of claim 4, wherein: The data further includes feedback collected by the input module from the potential wearer regarding the perceived comfort of the virtual try-on, and Processing the data further includes adjusting, by the local neural network, the rendering of the simulated appearance based on the feedback in a subsequent virtual try-on.

6. The method of claim 2, wherein: The in-store data includes one or more measurements or characteristics of at least one eye of the potential wearer, and Processing the data includes analyzing the one or more measurements or characteristics to determine a prescription for the potential wearer, wherein the federated learning process enhances the accuracy of the prescription.

7. The method according to any one of claims 2 to 6, wherein: The federated learning process includes: merging or combining weights from the specific model of the local neural network with weights from other specific models of other local neural networks to create a general model that takes advantage of specific features from all of the specific models, and The local neural network is updated based on the weights of the general model, thereby enhancing the capability of the local neural network.

8. The method according to any one of claims 2 to 7, wherein: The local neural network facilitates continuous learning based on corrections or feedback associated with previous outputs of the local neural network.

9. The method according to any one of claims 2 to 8, wherein: The data also includes wearer-specific information, such as personal data, IoT data, and wearer preferences, and the local neural network makes full use of the wearer-specific information to enhance the accuracy and personalization of the service provided.

10. A non-transitory computer-readable storage medium having stored thereon a computer program comprising instructions which, when executed by a processor, cause the processor to perform the method of any one of claims 2 to 9.

11. The computing device of claim 1, wherein: The data related to the wearer or potential wearer of the head wearable device includes in-store data.

12. The computing device of claim 1 or 11, wherein: The data includes characteristics of a vision impairment control solution used by the wearer or potential wearer, and the predictive and personalized service provision includes determining future values ​​of visual characteristics of the wearer or potential wearer.

13. The computing device of claim 12, wherein: The visual impairment is myopia.

14. The computing device of any one of claims 1 and 11 to 13, wherein: The data includes characteristics of a hearing impairment control solution used by the wearer or potential wearer, and the predictive and personalized service provision includes determining future values ​​of hearing characteristics of the wearer or potential wearer.

Images