Intelligent glasses fitting system and method based on deep learning

Abstract

The invention discloses an intelligent glasses fitting system and method based on deep learning, relates to the field of data processing and matching, and provides data support for subsequent training and testing of a U-Net neural network model by collecting eye image data and corresponding interpupillary distance data. Therefore, the obtained final U-Net neural network model has a good analysis capability on the eye image data and the interpupillary distance data, and an accurate analysis model is provided for subsequently obtaining the interpupillary distance data of the to-be-fitted person through the eye image data of the to-be-fitted person. Therefore, an optometrist does not need to manually measure the interpupillary distance data of the person to be subjected to optometry, time is saved, and the efficiency of the whole optometry process is improved; and the trial frame is adjusted according to the pupil distance data of the to-be-fitted person obtained through analysis and measurement of the final U-Net neural network model, repeated adjustment of the trial frame is not needed, the trial frame is adjusted to be optimal at a time, and the adjustment efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of data processing and matching, and in particular, relates to a glasses intelligent fitting system and method based on deep learning. Background Art

[0002] Glasses fitting refers to the process of examining and evaluating the eyes to determine the appropriate degree, type and style of glasses; it usually includes the following steps: vision test, eye health check, glasses assessment, glasses prescription, glasses trial and glasses delivery, etc.; glasses fitting is a professional process that needs to be performed by trained ophthalmologists or optometrists; they will provide personalized glasses recommendations based on the customer's specific situation to ensure that the customer can get the best vision correction effect.

[0003] Chinese patent application CN115809351A discloses an AR glasses control method, device, AR glasses, medium and program. In some embodiments of the present disclosure, the AR glasses obtain the user's current singing voice; according to the current singing voice, the target lyrics segment that the user is about to sing is controlled to be displayed in the AR space, wherein the AR space is a virtual space constructed by the AR glasses according to the current real environment. The AR glasses predict the target lyrics segment that the user is about to sing according to the user's current singing voice, so as to display the target lyrics segment that the user is about to sing in the AR space. The AR glasses are applied to singing scenes to enhance the user experience.

[0004] The traditional method of fitting glasses requires going to a physical store and having your eyes tested by an optometrist. Before the test, the pupil distance needs to be measured. Currently, the pupil distance measurement when fitting glasses is mainly for both eyes and is performed manually by an optometrist. Not only is the measurement efficiency low, but the human eyes are not absolutely symmetrical. If the glasses are fitted according to the pupil distance of both eyes, the left and right eyes of the buyer will each have a certain degree of prismatic power after wearing the glasses. Such glasses will cause great discomfort to the buyer. Summary of the invention

[0005] In response to the problems in the related art, the present invention proposes a glasses intelligent fitting system and method based on deep learning to overcome the above-mentioned technical problems existing in the existing related technology.

[0006] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0007] The present invention is a method for intelligent fitting of glasses based on deep learning, comprising the following steps:

[0008] S1, collecting multiple groups of eye image data and corresponding pupil distance data obtained by frontal photography under the same conditions; constructing an initial U-Net neural network model and using the eye image data and pupil distance data to train and test the initial U-Net neural network model to obtain a final U-Net neural network model;

[0009] S2, constructing a glasses database and collecting eye image data of the person to be fitted and desired glasses feature data; using the final U-Net neural network model to perform image analysis on the eye image data of the person to be fitted to obtain the pupil distance data of the person to be fitted; adjusting the trial glasses frame according to the pupil distance data of the person to be fitted;

[0010] S3, matching the eye image data of the person to be fitted and the expected glasses feature data with the data in the glasses database to obtain multiple pairs of glasses that are successfully matched; the person to be fitted uses the adjusted trial frames to try on the multiple pairs of glasses that are successfully matched one by one to obtain the glasses that are most suitable for the person to be fitted;

[0011] The same conditions refer to the same external factors such as shooting distance and external light intensity. By collecting eye image data and corresponding pupil distance data, data support is provided for the subsequent training and testing of the U-Net neural network model. Then, by limiting the shooting conditions, it is convenient to analyze the eye image data obtained by the shooting and train the model, which reduces the complexity of model training and testing, so that the final U-Net neural network model has good analysis capabilities for eye image data and pupil distance data, and provides an accurate analysis model for obtaining the pupil distance data of the person to be fitted through the eye image data of the person to be fitted. Therefore, there is no need for the optometrist to manually measure the pupil distance data of the person to be fitted, which saves time and improves the efficiency of the entire optometry process. The trial frame is adjusted according to the pupil distance data of the person to be fitted obtained by the analysis and measurement of the final U-Net neural network model, and the trial frame does not need to be repeatedly adjusted, and is adjusted to the best at one time, which improves the adjustment efficiency. In this scheme, the expected glasses feature data of the person to be fitted is collected, and the personal hobbies and actual needs of the person to be fitted are comprehensively considered, so that the matched glasses can meet the needs of the person to be fitted in many aspects.

[0012] Preferably, the S1 comprises the following steps:

[0013] S11, collect multiple groups of eye images and corresponding pupil distance data obtained by taking the eye under the same conditions, and obtain an eye image sample set a={a 1 ,a 2 ,...,a i ,...,a a′} and pupil distance data matrix ai represents the i-th eye image collected, a′ represents the total number of eye images collected; the pupil distance data matrix as follows,

[0014]

[0015] in, and Respectively represent the binocular pupil distance data, left eye pupil distance data, right eye pupil distance data and frame width data corresponding to the i-th eye image;

[0016] S12, construct an initial U-Net neural network model, set the total number of layers of the initial U-Net neural network model to a~, and each layer of the initial U-Net neural network model contains b 1 convolutional layers and b 2 pooling layer; set the pooling window size of the pooling layer to b 21 and a stride of b 22 ; Reinitialize the b 1 The parameters of the convolutional layer are obtained to obtain the initial parameter matrix b′ of the convolutional layer; as follows,

[0017]

[0018] Among them, b i ' 1 , b i ' 2 , b i ' 3 and b i ' 4 Respectively represent the convolution kernel size, step size, number of padded pixels, and number of channels of the convolution layer of the i-th layer of the initial U-Net neural network model;

[0019] The activation function of the initial U-Net neural network model is a linear rectification function;

[0020] S13, according to the eye image sample set a={a 1 ,a 2 ,...,a i ,...,a a′} and pupil distance data matrix Perform training and testing operations on the initial U-Net neural network model to obtain the final U-Net neural network model;

[0021] Since human eyes are not absolutely symmetrical, for some special people with extremely asymmetrical eyes, it is impossible to match glasses based on only measuring the pupil distance data of both eyes without measuring the pupil distance data of the left eye and the right eye. Therefore, in this solution, by collecting the pupil distance data of both eyes, the left eye pupil distance data, the right eye pupil distance data and the frame width data corresponding to the eye image data, the two eyes and the left and right eyes are analyzed and processed separately, and finally the most suitable glasses for the purchaser can be matched.

[0022] Preferably, the S13 comprises the following steps:

[0023] S131, setting a training data ratio; and performing an eye image sample set a={a 1 ,a 2 ,...,a i ,...,a a′} and pupil distance data matrix Performing data division to obtain an eye image training sample set, an eye image test sample set, an pupil distance training data matrix, and a pupil distance test data matrix;

[0024] S132, setting a training error threshold; inputting the eye image training sample set and the pupil distance training data matrix into the initial U-Net neural network model to perform a training operation on the initial U-Net neural network model, and when the training error is less than the training error threshold during the training operation, stopping the training to obtain a trained U-Net neural network model;

[0025] S133, set the accuracy threshold to The eye image test sample set and the pupil distance test data matrix are input into the trained U-Net neural network model for testing to obtain the first test accuracy.

[0026] when When the trained U-Net neural network model is used as the final U-Net neural network model; otherwise, the trained U-Net neural network model is optimized to obtain the final U-Net neural network model;

[0027] By using the eye image training sample set, eye image test sample set, pupil distance training data matrix and pupil distance test data matrix to train, test and optimize the initial U-Net neural network model respectively, the U-Net neural network model can meet the requirements for measuring the eye image data, which provides a more accurate analysis model for the subsequent analysis of the eye image data of the person to be fitted to obtain the pupil distance data of the person to be fitted.

[0028] Preferably, optimizing the trained U-Net neural network model in S133 includes the following steps:

[0029] S1331. Establishing the northern goshawk population represents the i-th northern goshawk, c represents the size of the northern goshawk population; the search space dimension of the northern goshawk population is set to Using each convolution layer initial parameter in the convolution layer initial parameter matrix b′ as the initial position of the first northern goshawk in the northern goshawk population;

[0030] Then set the maximum number of iterations of the northern goshawk population to c~ 1 And the current number of iterations is c~ 2 ;

[0031] S1332, calculating the initial positions of other northern goshawks in the northern goshawk population according to the initial position of the first northern goshawk, and obtaining an initial position matrix set c′={c 1 ′,c′ 2 ,...,c i ′,...,c c ′},c i ′ represents the initial position matrix of the i-th northern goshawk; as follows,

[0032]

[0033] Among them, c i ' j1 、c i ' j2 、c i ' j3 and c i ' j4 They respectively represent the dimensional components of the convolution kernel size, step size, number of padded pixels, and number of channels of the convolution layer of the jth layer of the trained U-Net neural network model in the initial position matrix of the i-th northern goshawk;

[0034] S1333, constructing the northern goshawk population The fitness function as follows,

[0035]

[0036] Among them, α is the correction amount;

[0037] According to the fitness function Calculate the initial position matrix set c′ = {c 1 ′,c′ 2 ,...,c i′,...,c c ′}, and obtain an initial fitness value set; select the maximum fitness value in the initial fitness value set as the initial optimal fitness value; and use the initial position matrix of the northern goshawk corresponding to the initial optimal fitness value as the initial optimal position matrix;

[0038] S1334, set the first iteration number to An iterative operation is started, and an adaptive probability d is randomly generated in each round of iteration; each northern goshawk in the northern goshawk population performs a global search according to the adaptive probability d, and updates the position of each northern goshawk;

[0039] when When , each northern goshawk in the northern goshawk population stops the global search and starts the local search to update the position of each northern goshawk;

[0040] S1335, when When the iteration operation is stopped, the final position matrix set d′={d 1 ′,d 2 ′,...,d i ′,...,d c ′},d i ' represents the final position matrix of the i-th northern goshawk in the northern goshawk population;

[0041] According to the fitness function Calculate the final position matrix set d′={d 1 ′,d 2 ′,...,d i ′,...,d c The fitness value of each final position matrix in ′}, and the final position matrix corresponding to the fitness value with the largest fitness value of each final position matrix is ​​selected as the final optimal position matrix;

[0042] Substituting each position dimension component in the final optimal position matrix into the trained U-Net neural network model to obtain an optimized U-Net neural network model;

[0043] S1336: Input the eye image test sample set and the pupil distance test data matrix described in S133 into the optimized U-Net neural network model for testing, and obtain the second test accuracy.

[0044] when When the optimized U-Net neural network model is used as the final U-Net neural network model; otherwise, the operations from S1331 to S1335 are repeated for the optimized U-Net neural network model until So far, the final U-Net neural network model is obtained;

[0045] The Northern Goshawk optimization algorithm is used to iteratively optimize the relevant parameters of the convolution layer of the trained U-Net neural network model whose test results do not meet the requirements. Since these parameters have an impact on the analysis effect of the U-Net neural network model, the analysis effect of the trained U-Net neural network model will become better and better by continuously iteratively optimizing the relevant parameters of the convolution layer, and finally meet the requirements.

[0046] Preferably, the S1332 includes the following steps:

[0047] S13321, set product value For each initial parameter and product value in the initial parameter matrix b′ of the convolution layer Multiplying them to obtain a producted convolutional layer initial parameter matrix; performing binary encoding on each parameter in the producted convolutional layer initial parameter matrix to obtain a encoded convolutional layer initial parameter matrix;

[0048] S13322. Obtain the length value of each data in the initial parameter matrix of the encoded convolutional layer Generate two random positive integers and and The first The data of the first The data of the bit is inverted to obtain the initial parameter matrix of the convolution layer after inversion;

[0049] S13323, binary decoding is performed on each data in the inverted convolutional layer initial parameter matrix to obtain a decoded convolutional layer initial parameter matrix, and each data in the decoded convolutional layer initial parameter matrix is ​​divided by the product value Obtaining an initial parameter matrix of the processed convolutional layer; using the initial parameter matrix of the processed convolutional layer as the initial position of the second northern goshawk in the northern goshawk population;

[0050] S13324, repeat S13321 to S13323, initialize the position of each northern goshawk in the northern goshawk population, and obtain the initial position matrix set c′={c 1 ′,c′ 2 ,...,c i ′,...,cc '};

[0051] By binary encoding the position data of the northern goshawk and inverting the number of random bits, multiple randomly generated initial positions of the northern goshawk are obtained. Since the multiple initial positions are randomly generated, the initial solution space of the northern goshawk population is richer, so it is more likely to obtain the optimal solution through iteration, that is, the optimal parameters of the convolution layer of the U-Net neural network model.

[0052] Preferably, the negation operation includes the following process:

[0053] If the number of a certain bit of each data in the initial parameter matrix of the encoded convolutional layer is 0, it is set to 1; if the number of a certain bit of each data in the initial parameter matrix of the encoded convolutional layer is 1, it is set to 0;

[0054] Preferably, the global search performed by each northern goshawk in the northern goshawk population according to the adaptive probability d in S1334 includes the following steps:

[0055] S13341, setting a global search probability threshold; when d is greater than or equal to the global search probability threshold, selecting the first global search strategy to update the position of each northern goshawk; otherwise, selecting the second global search strategy to update the position of each northern goshawk;

[0056] The first global search strategy comprises the following steps:

[0057] Set the first replacement digit interval [e 11 ,e 12 ],e 11 and e 12 respectively represent the lower limit and the upper limit of the first replacement digit interval; and perform binary encoding on the initial optimal position matrix in the manner of S13321 to obtain the encoded initial optimal position matrix;

[0058] For each position data in the initial optimal position matrix after encoding, 11 bit and e 12 The data between the bits are exchanged with the data between the corresponding bits of the initial positions of other northern goshawks in the northern goshawk population to obtain an initial optimal position matrix after the exchange;

[0059] The initial optimal position matrix after the exchange is binary decoded and processed in the manner of S13323 to obtain a first round position set matrix;

[0060] The second global search strategy comprises the following steps:

[0061] Set the second replacement digit interval [e21 ,e 22 ]; e 21 and e 22 respectively represent the lower limit and the upper limit of the second replacement digit interval; and perform binary encoding on the initial optimal position matrix in the manner of S13321 to obtain the encoded initial optimal position matrix;

[0062] For each position data in the initial optimal position matrix after encoding, 11 bit and e 12 The data between the bits are inverted to obtain the inverted initial optimal position matrix;

[0063] The inverted initial optimal position matrix is ​​binary decoded and processed in the manner of S13323 to obtain a first round position set matrix;

[0064] By setting the first global search strategy and the second global search strategy and randomly selecting the first global search strategy or the second global search strategy through adaptive probability, the search method of the northern goshawk population for the optimal parameters of the convolutional layer is more diverse and the search for the optimal parameters is more efficient.

[0065] Preferably, S2 comprises the following steps:

[0066] S21, setting a person to be fitted and storing the data of each pair of glasses in the store in advance into a glasses database; and the person to be fitted inputs the feature data of the glasses he / she expects in advance to obtain a feature data set of the glasses expected;

[0067] S22, using the method of S11 to collect the eye image of the person to be fitted, to obtain the eye image to be fitted; using the final U-Net neural network model to perform image analysis on the eye image to be fitted, to obtain a first eye dataset e to be fitted 1 ′={e 1 ' 1 ,e 1 ' 2 ,e 1 ' 3 ,e 1 ' 4},e 1 ' 1 、e 1 ' 2 、e 1 ' 3 and e 1 ' 4 Respectively represent the pupil distance data of both eyes, the pupil distance data of the left eye, the pupil distance data of the right eye and the frame width data of the person to be fitted;

[0068] S23, using an automatic ophthalmometer to perform a refractive examination on the person to be fitted, to obtain a second eye data set to be fitted e′ 2i represents the i-th examination result parameter obtained by performing a refractive examination on the person to be fitted using an automatic ophthalmometer, Indicates the total number of examination result parameters obtained by using an automatic ophthalmometer to perform refractive examination on the person to be fitted;

[0069] S24, according to the first eye data set to be fitted 1 ′={e 1 ' 1 ,e 1 ' 2 ,e 1 ' 3 ,e 1 ' 4}Adjust the trial frame to obtain the adjusted trial frame;

[0070] The final U-Net neural network model is used to analyze the eye images to obtain the pupil distance data of both eyes, the left eye pupil distance data, the right eye pupil distance data and the frame width data. These data can guide the adjustment of the trial frame, making the adjustment efficiency of the trial frame higher.

[0071] Preferably, the S24 includes the following process:

[0072] S241, according to the first eye data set to be fitted 1 ′={e 1 ' 1 ,e 1 ' 2 ,e 1 ' 3 ,e 1 ' 4 e in} 1 ' 4 Adjust the distance between the two sides of the trial frame;

[0073] S242, according to e 1 ' 1 、e 1 ' 2 and e 1 ' 3 Adjust the center position of the trial frame to ensure that the optical center of the lens matches the wearer's pupil distance;

[0074] S243, after adjustment in S241 and S242, the adjusted trial frame is obtained;

[0075] By adjusting the distance between the two sides of the trial frame according to the frame width data in the first eye data set to be fitted, the glasses are adapted to the facial features of the person to be fitted and ensure comfortable wearing, thus providing the person to be fitted with a better purchasing experience;

[0076] Preferably, S3 comprises the following steps:

[0077] S31, the second eye data set to be fitted The expected eye feature data set is merged to obtain a final eye data set to be fitted; the final eye data set to be fitted is matched with each eye data in the eye database to obtain a eye set to be screened;

[0078] S32, using the adjusted trial glasses frame to let the person to be fitted perform a wearing test on each pair of glasses in the glasses to be screened, and obtaining the glasses most suitable for the person to be fitted according to the results of the wearing test on each pair of glasses in the glasses to be screened;

[0079] By matching the glasses database, the most suitable glasses can be obtained, making it easier for the person to choose.

[0080] A glasses intelligent fitting system based on deep learning, comprising a first eye image data acquisition module, a pupil distance data acquisition module, a U-Net model building module, a second eye image data acquisition module, a feature data acquisition module, an image analysis module, a trial frame adjustment module and a data matching module;

[0081] The first eye image data acquisition module is used to acquire multiple groups of eye image data obtained by taking pictures in the same condition;

[0082] The pupil distance data acquisition module is used to acquire pupil distance data corresponding to the eye image data acquired by the first eye image data acquisition module;

[0083] The U-Net model building module is used to build an initial U-Net neural network model and use the eye image data and pupil distance data to train and test the initial U-Net neural network model to obtain a final U-Net neural network model;

[0084] The second eye image data acquisition module is used to acquire eye image data of the person to be fitted;

[0085] The feature data collection module is used to collect the feature data of glasses expected by the person to be fitted;

[0086] The image analysis module is used to use the final U-Net neural network model to perform image analysis on the eye image data of the person to be fitted, so as to obtain the pupil distance data of the person to be fitted;

[0087] The trial glasses frame adjustment module is used to adjust the trial glasses frame according to the pupil distance data of the person to be fitted;

[0088] The data matching module is used to match the eye image data of the person to be fitted and the expected glasses feature data with the data in the glasses database to obtain multiple pairs of glasses that are successfully matched.

[0089] The present invention has the following beneficial effects:

[0090] 1. In the present invention, a first eye image data acquisition module, an interpupillary distance data acquisition module, a U-Net model construction module, a second eye image data acquisition module, a feature data acquisition module, an image analysis module, a trial frame adjustment module and a data matching module are set; by collecting eye image data and corresponding interpupillary distance data, data support is provided for subsequent training and testing of the U-Net neural network model, and then by limiting the shooting conditions, it is convenient to analyze the eye image data obtained by shooting and train the model, which reduces the complexity of model training and testing, so that the final U-Net neural network model obtained has good analysis capabilities for eye image data and interpupillary distance data, and provides an accurate analysis model for obtaining the interpupillary distance data of the person to be fitted through the eye image data of the person to be fitted in the future; Therefore, there is no need for the optometrist to manually measure the pupil distance data of the person to be fitted, which saves time and improves the efficiency of the entire optometry process; and the trial frame is adjusted according to the pupil distance data of the person to be fitted obtained by the final U-Net neural network model analysis and measurement, without the need to repeatedly adjust the trial frame, and it can be adjusted to the best at one time, thereby improving the adjustment efficiency; in this scheme, by collecting the expected glasses feature data of the person to be fitted, the personal preferences and actual needs of the person to be fitted are comprehensively considered, so that the matched glasses can meet the needs of the person to be fitted in many aspects; among them, by collecting the binocular pupil distance data, the left eye pupil distance data, the right eye pupil distance data and the frame width data corresponding to the eye image data, the binocular and left and right monocular analysis and processing are performed separately, and finally the most suitable glasses for the purchaser can be matched.

[0091] 2. In the present invention, the initial U-Net neural network model is trained, tested and optimized by using an eye image training sample set, an eye image test sample set, an pupil distance training data matrix and an pupil distance test data matrix, so that the U-Net neural network model can meet the requirements for measuring the eye image data, and provides a more accurate analysis model for the subsequent analysis of the eye image data of the person to be fitted to obtain the pupil distance data of the person to be fitted.

[0092] 3. In the present invention, the relevant parameters of the convolution layer of the trained U-Net neural network model whose test results do not meet the requirements are continuously iterated and optimized by adopting the northern goshawk optimization algorithm. Since these parameters have an influence on the analysis effect of the U-Net neural network model, the analysis effect of the trained U-Net neural network model will be made better and better by continuously iterating and optimizing the relevant parameters of the convolution layer, and finally meet the requirements; wherein, by binary encoding the position data of the northern goshawk and inverting the number of random bits therein, a plurality of randomly generated initial positions of the northern goshawk are obtained. Since the plurality of initial positions are randomly generated, the initial solution space of the northern goshawk population is richer, so that it is more likely to obtain the optimal solution through iteration, that is, the optimal parameters of the convolution layer of the U-Net neural network model.

[0093] 4. In the present invention, the final U-Net neural network model is used to perform image analysis on the eye image to be fitted, and the pupil distance data of the eyes, the left eye pupil distance data, the right eye pupil distance data and the frame width data of the person to be fitted are obtained. These data can guide the adjustment of the trial frame, making the adjustment efficiency of the trial frame higher, thereby improving the efficiency of the entire fitting.

[0094] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying creative work.

[0096] Figure 1 The present invention is a flowchart of glasses fitting by a glasses intelligent fitting system based on deep learning. DETAILED DESCRIPTION

[0097] The following will be combined with the drawings in the embodiments of the invention to clearly and completely describe the technical solutions in the embodiments of the invention. Obviously, the described embodiments are only part of the embodiments of the invention, not all of the embodiments. Based on the embodiments in the invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the invention.

[0098] In the description of the present invention, it is necessary to understand that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.

[0099] The present invention is a method for intelligent fitting of glasses based on deep learning, comprising the following steps:

[0100] S1, collecting multiple groups of eye image data and corresponding pupil distance data obtained by frontal photography under the same conditions; constructing an initial U-Net neural network model and using the eye image data and pupil distance data to train and test the initial U-Net neural network model to obtain a final U-Net neural network model;

[0101] The S1 comprises the following steps:

[0102] S11, collect multiple groups of eye images and corresponding pupil distance data obtained by taking the eye under the same conditions, and obtain an eye image sample set a={a 1 ,a 2 ,...,a i ,...,a a′} and pupil distance data matrix a i represents the i-th eye image collected, a′ represents the total number of eye images collected; the pupil distance data matrix as follows,

[0103]

[0104] in, and Respectively represent the binocular pupil distance data, left eye pupil distance data, right eye pupil distance data and frame width data corresponding to the i-th eye image;

[0105] S12, construct an initial U-Net neural network model, and set the total number of layers of the initial U-Net neural network model to Each layer of the initial U-Net neural network model contains b 1 convolutional layers and b 2 pooling layer; set the pooling window size of the pooling layer to b 21 and a stride of b 22 ; Reinitialize the b 1 The parameters of the convolutional layer are obtained to obtain the initial parameter matrix b′ of the convolutional layer; as follows,

[0106]

[0107] Among them, b i ' 1 , b i ' 2 , b i ' 3 and b i ' 4 Respectively represent the convolution kernel size, step size, number of padded pixels, and number of channels of the convolution layer of the i-th layer of the initial U-Net neural network model;

[0108] The activation function of the initial U-Net neural network model is a linear rectification function;

[0109] S13, according to the eye image sample set a={a 1 ,a 2 ,...,a i ,...,a a′} and pupil distance data matrix Perform training and testing operations on the initial U-Net neural network model to obtain the final U-Net neural network model;

[0110] The S13 comprises the following steps:

[0111] S131, setting a training data ratio; and performing an eye image sample set a={a 1 ,a 2 ,...,a i ,...,a a′} and pupil distance data matrix Performing data division to obtain an eye image training sample set, an eye image test sample set, an pupil distance training data matrix, and a pupil distance test data matrix;

[0112] S132, setting a training error threshold; inputting the eye image training sample set and the pupil distance training data matrix into the initial U-Net neural network model to perform a training operation on the initial U-Net neural network model, and when the training error is less than the training error threshold during the training operation, stopping the training to obtain a trained U-Net neural network model;

[0113] S133, set the accuracy threshold to The eye image test sample set and the pupil distance test data matrix are input into the trained U-Net neural network model for testing to obtain the first test accuracy.

[0114] when When the trained U-Net neural network model is used as the final U-Net neural network model; otherwise, the trained U-Net neural network model is optimized to obtain the final U-Net neural network model;

[0115] Optimizing the trained U-Net neural network model in S133 includes the following steps:

[0116] S1331. Establishing the northern goshawk population represents the i-th northern goshawk, c represents the size of the northern goshawk population; the search space dimension of the northern goshawk population is set to Using each convolution layer initial parameter in the convolution layer initial parameter matrix b′ as the initial position of the first northern goshawk in the northern goshawk population;

[0117] Then set the maximum number of iterations of the northern goshawk population to And the current number of iterations is

[0118] S1332, calculating the initial positions of other northern goshawks in the northern goshawk population according to the initial position of the first northern goshawk, and obtaining an initial position matrix set c′={c 1 ′,c′ 2 ,...,c i ′,...,c c ′},c i ′ represents the initial position matrix of the i-th northern goshawk; as follows,

[0119]

[0120] Among them, c i ' j1 、c i ' j2 、c i ' j3 and c i ' j4 They respectively represent the dimensional components of the convolution kernel size, step size, number of padded pixels, and number of channels of the convolution layer of the jth layer of the trained U-Net neural network model in the initial position matrix of the i-th northern goshawk;

[0121] The S1332 includes the following steps:

[0122] S13321, set product value For each initial parameter and product value in the initial parameter matrix b′ of the convolution layer Multiplying them to obtain a producted convolutional layer initial parameter matrix; performing binary encoding on each parameter in the producted convolutional layer initial parameter matrix to obtain a encoded convolutional layer initial parameter matrix;

[0123] S13322. Obtain the length value of each data in the initial parameter matrix of the encoded convolutional layer Generate two random positive integers and and The first The data of the first The data of the bit is inverted to obtain the initial parameter matrix of the convolution layer after inversion;

[0124] S13323, binary decoding is performed on each data in the inverted convolutional layer initial parameter matrix to obtain a decoded convolutional layer initial parameter matrix, and each data in the decoded convolutional layer initial parameter matrix is ​​divided by the product value Obtaining an initial parameter matrix of the processed convolutional layer; using the initial parameter matrix of the processed convolutional layer as the initial position of the second northern goshawk in the northern goshawk population;

[0125] S13324, repeat S13321 to S13323, initialize the position of each northern goshawk in the northern goshawk population, and obtain the initial position matrix set c′={c 1 ′,c′ 2 ,...,c i ′,...,c c '};

[0126] The negation operation includes the following process:

[0127] If the number of a certain bit of each data in the initial parameter matrix of the encoded convolutional layer is 0, it is set to 1; if the number of a certain bit of each data in the initial parameter matrix of the encoded convolutional layer is 1, it is set to 0;

[0128] The global search performed by each northern goshawk in the northern goshawk population according to the adaptive probability d in S1334 includes the following steps:

[0129] S13341, setting a global search probability threshold; when d is greater than or equal to the global search probability threshold, selecting the first global search strategy to update the position of each northern goshawk; otherwise, selecting the second global search strategy to update the position of each northern goshawk;

[0130] The first global search strategy comprises the following steps:

[0131] Set the first replacement digit interval [e 11 ,e 12 ],e 11 and e 12 respectively represent the lower limit and the upper limit of the first replacement digit interval; and perform binary encoding on the initial optimal position matrix in the manner of S13321 to obtain the encoded initial optimal position matrix;

[0132] For each position data in the initial optimal position matrix after encoding, 11 bit and e 12 The data between the bits are exchanged with the data between the corresponding bits of the initial positions of other northern goshawks in the northern goshawk population to obtain an initial optimal position matrix after the exchange;

[0133] The initial optimal position matrix after the exchange is binary decoded and processed in the manner of S13323 to obtain a first round position set matrix;

[0134] The second global search strategy comprises the following steps:

[0135] Set the second replacement digit interval [e 21 ,e 22 ]; e 21 and e 22 respectively represent the lower limit and the upper limit of the second replacement digit interval; and perform binary encoding on the initial optimal position matrix in the manner of S13321 to obtain the encoded initial optimal position matrix;

[0136] For each position data in the initial optimal position matrix after encoding, 11 bit and e 12 The data between the bits are inverted to obtain the inverted initial optimal position matrix;

[0137] The inverted initial optimal position matrix is ​​binary decoded and processed in the manner of S13323 to obtain a first round position set matrix;

[0138] S1333, constructing the northern goshawk population The fitness function as follows,

[0139]

[0140] Among them, α is the correction amount;

[0141] According to the fitness function Calculate the initial position matrix set c′ = {c 1 ′,c′ 2 ,...,c i ′,...,cc ′}, and obtain an initial fitness value set; select the maximum fitness value in the initial fitness value set as the initial optimal fitness value; and use the initial position matrix of the northern goshawk corresponding to the initial optimal fitness value as the initial optimal position matrix;

[0142] S1334, set the first iteration number to An iterative operation is started, and an adaptive probability d is randomly generated in each round of iteration; each northern goshawk in the northern goshawk population performs a global search according to the adaptive probability d, and updates the position of each northern goshawk;

[0143] when When , each northern goshawk in the northern goshawk population stops the global search and starts the local search to update the position of each northern goshawk;

[0144] S1335, when When the iteration operation is stopped, the final position matrix set d′={d 1 ′,d 2 ′,...,d i ′,...,d c ′},d i ' represents the final position matrix of the i-th northern goshawk in the northern goshawk population;

[0145] According to the fitness function Calculate the final position matrix set d′={d 1 ′,d 2 ′,...,d i ′,...,d c The fitness value of each final position matrix in ′}, and the final position matrix corresponding to the fitness value with the largest fitness value of each final position matrix is ​​selected as the final optimal position matrix;

[0146] Substituting each position dimension component in the final optimal position matrix into the trained U-Net neural network model to obtain an optimized U-Net neural network model;

[0147] S1336: Input the eye image test sample set and the pupil distance test data matrix described in S133 into the optimized U-Net neural network model for testing, and obtain the second test accuracy.

[0148] when When the optimized U-Net neural network model is used as the final U-Net neural network model; otherwise, the operations from S1331 to S1335 are repeated for the optimized U-Net neural network model until So far, the final U-Net neural network model is obtained;

[0149] S2, constructing a glasses database and collecting eye image data of the person to be fitted and desired glasses feature data; using the final U-Net neural network model to perform image analysis on the eye image data of the person to be fitted to obtain the pupil distance data of the person to be fitted; adjusting the trial glasses frame according to the pupil distance data of the person to be fitted;

[0150] The S2 comprises the following steps:

[0151] S21, setting a person to be fitted and storing the data of each pair of glasses in the store in advance into a glasses database; and the person to be fitted inputs the feature data of the glasses he / she expects in advance to obtain a feature data set of the glasses expected;

[0152] S22, using the method of S11 to collect the eye image of the person to be fitted, to obtain the eye image to be fitted; using the final U-Net neural network model to perform image analysis on the eye image to be fitted, to obtain a first eye dataset e to be fitted 1 ′={e 1 ' 1 ,e 1 ' 2 ,e 1 ' 3 ,e 1 ' 4},e 1 ' 1 、e 1 ' 2 、e 1 ' 3 and e 1 ' 4 Respectively represent the pupil distance data of both eyes, the pupil distance data of the left eye, the pupil distance data of the right eye and the frame width data of the person to be fitted;

[0153] S23, using an automatic ophthalmometer to perform a refractive examination on the person to be fitted, to obtain a second eye data set to be fitted e′ 2i represents the i-th examination result parameter obtained by performing a refractive examination on the person to be fitted using an automatic ophthalmometer, Indicates the total number of examination result parameters obtained by using an automatic ophthalmometer to perform refractive examination on the person to be fitted;

[0154] S24, according to the first eye data set to be fitted1 ′={e 1 ' 1 ,e 1 ' 2 ,e 1 ' 3 ,e 1 ' 4}Adjust the trial frame to obtain the adjusted trial frame;

[0155] The S24 includes the following process:

[0156] S241, according to the first eye data set to be fitted 1 ′={e 1 ' 1 ,e 1 ' 2 ,e 1 ' 3 ,e 1 ' 4 e in} 1 ' 4 Adjust the distance between the two sides of the trial frame to adapt to the facial features of different customers and ensure comfortable wearing;

[0157] S242, according to e 1 ' 1 、e 1 ' 2 and e 1 ' 3 Adjust the center position of the trial frame to ensure that the optical center of the lens matches the wearer's pupil distance;

[0158] S243, after adjustment in S241 and S242, the adjusted trial frame is obtained;

[0159] S3, matching the eye image data of the person to be fitted and the expected glasses feature data with the data in the glasses database to obtain multiple pairs of glasses that are successfully matched; the person to be fitted uses the adjusted trial frames to try on the multiple pairs of glasses that are successfully matched one by one to obtain the glasses that are most suitable for the person to be fitted;

[0160] The S3 comprises the following steps:

[0161] S31, the second eye data set to be fitted The expected eye feature data set is merged to obtain a final eye data set to be fitted; the final eye data set to be fitted is matched with each eye data in the eye database to obtain a eye set to be screened;

[0162] S32, using the adjusted trial glasses frame to let the person to be fitted perform a wearing test on each pair of glasses in the glasses set to be screened, and obtaining the glasses most suitable for the person to be fitted according to the results of the wearing test on each pair of glasses in the glasses set to be screened.

[0163] A glasses intelligent fitting system based on deep learning, comprising a first eye image data acquisition module, a pupil distance data acquisition module, a U-Net model building module, a second eye image data acquisition module, a feature data acquisition module, an image analysis module, a trial frame adjustment module and a data matching module;

[0164] The first eye image data acquisition module is used to acquire multiple groups of eye image data obtained by taking pictures in the same condition;

[0165] The pupil distance data acquisition module is used to acquire pupil distance data corresponding to the eye image data acquired by the first eye image data acquisition module;

[0166] The U-Net model building module is used to build an initial U-Net neural network model and use the eye image data and pupil distance data to train and test the initial U-Net neural network model to obtain a final U-Net neural network model;

[0167] The second eye image data acquisition module is used to acquire eye image data of the person to be fitted;

[0168] The feature data collection module is used to collect the feature data of glasses expected by the person to be fitted;

[0169] The image analysis module is used to use the final U-Net neural network model to perform image analysis on the eye image data of the person to be fitted, so as to obtain the pupil distance data of the person to be fitted;

[0170] The trial glasses frame adjustment module is used to adjust the trial glasses frame according to the pupil distance data of the person to be fitted;

[0171] The data matching module is used to match the eye image data of the person to be fitted and the expected glasses feature data with the data in the glasses database to obtain multiple pairs of glasses that are successfully matched.

[0172] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0173] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can understand and use the invention well.

Claims

1. A glasses intelligent fitting method based on deep learning, characterized in that: The following steps are involved: S1, collecting multiple groups of eye images and corresponding pupil distance data obtained by taking the images in the same condition, and obtaining an eye image sample set and a pupil distance data matrix; The initial U-Net neural network model is trained and tested according to the eye image sample set and the pupil distance data matrix to obtain a trained U-Net neural network model; the trained U-Net neural network model is then optimized to obtain a final U-Net neural network model; S2, using the final U-Net neural network model to analyze the eye image data of the person to be fitted, to obtain a first eye data set to be fitted; adjusting the trial frame according to the first eye data set to be fitted, to obtain an adjusted trial frame; Performing a refractive examination on the person to be fitted with the adjusted trial glasses frame to obtain a second eye data set to be fitted; S3: combining the second eye data set to be fitted and the glasses feature data expected by the person to be fitted and matching them with the data in the glasses database to obtain multiple pairs of glasses that are successfully matched.

2. The method for intelligent eyeglass fitting based on deep learning according to claim 1, characterized in that: The S1 comprises the following steps: S11, collecting multiple groups of eye images and corresponding pupil distance data obtained by frontal view shooting under the same conditions, to obtain an eye image sample set and a pupil distance data matrix; S12, construct an initial U-Net neural network model, set the total number of layers of the initial U-Net neural network model to a~, each layer of the initial U-Net neural network model includes b1 convolutional layers and b2 pooling layers; set the pooling window size of the pooling layer to b 21 and a stride of b 22 ; Initialize the parameters of the b1 convolutional layers again to obtain the initial parameter matrix of the convolutional layer; S13. Perform training and testing operations on the initial U-Net neural network model according to the eye image sample set and the pupil distance data matrix to obtain a final U-Net neural network model.

3. The method for intelligent eyeglass fitting based on deep learning according to claim 2, characterized in that: The S13 comprises the following steps: S131, setting a training data ratio; dividing the eye image sample set and the pupil distance data matrix according to the training data ratio to obtain an eye image training sample set, an eye image test sample set, a pupil distance training data matrix, and a pupil distance test data matrix; S132, setting a training error threshold; inputting the eye image training sample set and the pupil distance training data matrix into the initial U-Net neural network model to perform a training operation on the initial U-Net neural network model, and when the training error is less than the training error threshold during the training operation, stopping the training to obtain a trained U-Net neural network model; S133, set the accuracy threshold to The eye image test sample set and the pupil distance test data matrix are input into the trained U-Net neural network model for testing to obtain the first test accuracy. when When the trained U-Net neural network model is used as the final U-Net neural network model; otherwise, the trained U-Net neural network model is optimized to obtain the final U-Net neural network model.

4. The method for intelligent eyeglass fitting based on deep learning according to claim 3, characterized in that: Optimizing the trained U-Net neural network model in S133 includes the following steps: S1331, constructing a northern goshawk population; using each convolution layer initial parameter in the convolution layer initial parameter matrix as the initial position of the first northern goshawk in the northern goshawk population; and setting the maximum number of iterations of the northern goshawk population to And the current number of iterations is S1332, calculating the initial positions of other northern goshawks in the northern goshawk population according to the initial position of the first northern goshawk, to obtain an initial position matrix set; S1333, constructing the fitness function of the northern goshawk population; calculating the fitness value corresponding to each initial position in the initial position matrix set according to the fitness function, and obtaining an initial fitness value set; selecting the maximum fitness value in the initial fitness value set as the initial optimal fitness value; the initial position matrix of the northern goshawk corresponding to the initial optimal fitness value is used as the initial optimal position matrix; S1334, set the first iteration number to An iterative operation is started, and an adaptive probability d is randomly generated in each round of iteration; each northern goshawk in the northern goshawk population performs a global search according to the adaptive probability d, and updates the position of each northern goshawk; when When , each northern goshawk in the northern goshawk population stops the global search and starts the local search to update the position of each northern goshawk; S1335, when When , the iteration operation is stopped to obtain a final position matrix set; the fitness value of each final position matrix in the final position matrix set is calculated according to the fitness function, and the final position matrix corresponding to the fitness value with the largest fitness value of each final position matrix is ​​selected as the final optimal position matrix; each position dimension component in the final optimal position matrix is ​​substituted into the trained U-Net neural network model to obtain an optimized U-Net neural network model; S1336: Input the eye image test sample set and the pupil distance test data matrix described in S133 into the optimized U-Net neural network model for testing, and obtain the second test accuracy. when When the optimized U-Net neural network model is used as the final U-Net neural network model; otherwise, the operations from S1331 to S1335 are repeated for the optimized U-Net neural network model until So far, the final U-Net neural network model is obtained.

5. The method for intelligent eyeglass fitting based on deep learning according to claim 4, characterized in that: The S1332 includes the following steps: S13321, set product value For each initial parameter and product value in the initial parameter matrix of the convolution layer Multiplying them to obtain a product post-convolutional layer initial parameter matrix; performing binary encoding on each parameter in the product post-convolutional layer initial parameter matrix to obtain a encoded convolutional layer initial parameter matrix; S13322. Obtain the length value of each data in the initial parameter matrix of the encoded convolutional layer Generate two random positive integers and and The first The data of the first The data of the bit is inverted to obtain the initial parameter matrix of the convolution layer after inversion; S13323, binary decoding is performed on each data in the inverted convolutional layer initial parameter matrix to obtain a decoded convolutional layer initial parameter matrix, and each data in the decoded convolutional layer initial parameter matrix is ​​divided by the product value Obtaining an initial parameter matrix of the processed convolutional layer; using the initial parameter matrix of the processed convolutional layer as the initial position of the second northern goshawk in the northern goshawk population; S13324. Repeat S13321 to S13323 to initialize the position of each northern goshawk in the northern goshawk population to obtain the initial position matrix set.

6. The method for intelligent eyeglass fitting based on deep learning according to claim 5, characterized in that: The negation operation includes the following process: If the number of a certain bit of each data in the initial parameter matrix of the encoded convolutional layer is 0, it is set to 1; if the number of a certain bit of each data in the initial parameter matrix of the encoded convolutional layer is 1, it is set to 0.

7. The method for intelligent eyeglass fitting based on deep learning according to claim 1, characterized in that: The S2 comprises the following steps: S21, setting a person to be fitted and storing the data of each pair of glasses in the store in advance into a glasses database; and the person to be fitted inputs the feature data of the glasses he / she expects in advance to obtain a feature data set of the glasses expected; S22, collecting eye images of the person to be fitted by the method of S11 to obtain eye images to be fitted; performing image analysis on the eye images to be fitted by using the final U-Net neural network model to obtain a first eye data set to be fitted; S23, performing a refractive examination on the person to be fitted, to obtain a second eye data set to be fitted; S24, adjusting the trial spectacles frame according to the first eye data set to be fitted to obtain an adjusted trial spectacles frame.

8. The method for intelligent eyeglass fitting based on deep learning according to claim 7, characterized in that: The S24 includes the following process: S241, adjusting the distance between two sides of the trial glasses frame according to the first eye data set to be fitted; S242, adjusting the center position of the trial glasses frame according to the first eye data set to be fitted, so as to ensure that the optical center of the lens matches the pupil distance of the wearer; S243, after adjustment in S241 and S242, the adjusted trial glasses frame is obtained.

9. The method for intelligent eyeglass fitting based on deep learning according to claim 1, characterized in that: The S3 comprises the following steps: S31, merging the second eye data set to be fitted and the expected glasses feature data set to obtain a final eye data set to be fitted; matching the final eye data set to be fitted with each glasses data in the glasses database to obtain a glasses set to be screened; S32, using the adjusted trial glasses frame to let the person to be fitted perform a wearing test on each pair of glasses in the glasses set to be screened, and obtaining the glasses most suitable for the person to be fitted according to the results of the wearing test on each pair of glasses in the glasses set to be screened.

10. A system for implementing the deep learning-based intelligent eyeglass fitting method according to any one of claims 1 to 9, characterized in that: It includes a first eye image data acquisition module, a pupil distance data acquisition module, a U-Net model building module, a second eye image data acquisition module, a feature data acquisition module, an image analysis module, a trial glasses frame adjustment module and a data matching module.

Images