Image processing method and system for intelligent glasses

Abstract

The invention discloses an image processing method and system for intelligent glasses. The method comprises the following steps: acquiring to-be-processed image information; the to-be-processed image information comprises M pieces of first to-be-processed information and M pieces of second to-be-processed information; determining first radius information and second radius information based on the to-be-processed image information; and determining sight line estimation result information based on the first radius information and the second radius information. Therefore, stable and reliable sight line estimation result information can be provided.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction, and in particular to an image processing method and system for smart glasses. Background Art

[0002] Gaze estimation refers to the use of technical means to detect, track or estimate the gaze direction of a person or animal. Eye tracking is a major application area of ​​gaze estimation. By using equipment such as eye trackers, the gaze point and gaze path of human eyes on the screen or in the real world can be monitored and recorded in real time. This is very useful for understanding people's attention distribution, cognitive processes and user experience in specific tasks. In computer vision, gaze estimation can be used to identify and track the position of human eyes. This usually involves using a camera to capture images or videos, and using image processing techniques to locate and track the position of the eyes to determine the direction of the gaze.

[0003] In the three-dimensional eyeball model, the existing mainstream method is to accurately model the human cornea, using the reflection and refraction phenomenon that occurs when light enters the human cornea, and solving the corneal curvature center and pupil center of the eyeball by establishing a geometric model of the cornea, and then reconstructing the optical axis of the eye, and then reconstructing the visual axis based on the relationship between the optical axis and the visual axis of the human eye, and finally solving the gaze point. However, in the rotationally symmetric eyeball model, because the reverse extension line of the reflected light on the corneal surface does not meet the particularity that it will converge at the center of the corneal curvature, but only intersects at a certain point on the optical axis, and with the increase of age, the front surface of the cornea will become more curved, so the error of using corneal features will be large. Therefore, an image processing method and system for smart glasses are provided, which is conducive to providing stable and reliable line of sight estimation result information. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an image processing method and system for smart glasses, which is conducive to providing stable and reliable line of sight estimation result information.

[0005] In order to solve the above technical problems, the first aspect of the embodiment of the present invention discloses an image processing method for smart glasses, the method comprising:

[0006] S1, obtaining image information to be processed; the image information to be processed includes M first information to be processed and M second information to be processed;

[0007] S2, determining first radius information and second radius information based on the image information to be processed;

[0008] S3: Determine line of sight estimation result information based on the first radius information and the second radius information.

[0009] As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining the first radius information and the second radius information based on the image information to be processed includes:

[0010] S21, performing fusion processing on the image information to be processed to obtain pre-processed image information; the pre-processed image information includes the M first pre-processed information and the M second pre-processed information;

[0011] S22, determining first eyeball center information and second eyeball center information based on the preprocessed image information;

[0012] S23: Determine first radius information and second radius information based on the preprocessed image information, the first eyeball center information and the second eyeball center information.

[0013] As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining the first eye center information and the second eye center information based on the preprocessed image information includes:

[0014] S221: for any of the first preprocessing information, use a first fitting model to perform calculation processing on the first preprocessing information to obtain third preprocessing information corresponding to the first preprocessing information;

[0015] Wherein, the first fitting model is:

[0016] KL=ZL+s L *(NL);

[0017] Wherein, KL is the third preprocessing information, ZL is the pupil position corresponding to the first preprocessing information, NL is the gaze vector corresponding to the first preprocessing information, and s L is the first fixation vector length coefficient;

[0018] S222: for any second preprocessing information, use a second fitting model to perform calculation processing on the second preprocessing information to obtain fourth preprocessing information corresponding to the second preprocessing information;

[0019] Wherein, the second fitting model is:

[0020] KR=ZR+s R *(NR);

[0021] Wherein, KR is the fourth preprocessing information, ZR is the pupil position corresponding to the second preprocessing information, NR is the gaze vector corresponding to the second preprocessing information, and s R is the second fixation vector length coefficient;

[0022] S223, using a third fitting model, fusing the M pieces of the first preprocessed information and the third preprocessed information corresponding to the M pieces of the first preprocessed information to obtain first eye center information;

[0023] Wherein, the third fitting model is:

[0024]

[0025] In the formula, g L is the first eyeball center information, For the i a the third pre-processing information, For the i a a fixation vector corresponding to the first preprocessed information, for The transposed matrix of For the i a The pupil position corresponding to the first preprocessed information, α L is the first fusion coefficient, i a is a positive integer greater than or equal to 1 and less than or equal to M;

[0026] S224, using a fourth fitting model, fusing the M pieces of the second preprocessed information and the fourth preprocessed information corresponding to the M pieces of the second preprocessed information to obtain second eyeball center information;

[0027] Wherein, the fourth fitting model is:

[0028]

[0029] In the formula, g R is the second eyeball center information, For the i b the fourth pre-processing information, For the i b a fixation vector corresponding to the second preprocessing information, for The transposed matrix of For the i b The pupil position corresponding to the second preprocessing information, α R is the second fusion coefficient, i b is a positive integer greater than or equal to 1 and less than or equal to M.

[0030] As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining the first radius information and the second radius information based on the preprocessed image information, the first eye center information and the second eye center information includes:

[0031] S231, for any of the first preprocessing information, analyzing and processing the first preprocessing information to obtain first intersection information corresponding to the first preprocessing information;

[0032] S232: for any second pre-processed information, analyze and process the second pre-processed information to obtain second intersection information corresponding to the second pre-processed information;

[0033] S233, using a fifth fitting model, calculating and processing the first intersection information, the second intersection information, the first eyeball center information, and the second eyeball center information to obtain first radius information and second radius information;

[0034] Wherein, the fifth fitting model is:

[0035]

[0036] Wherein, ZBJ is the first radius information, YBJ is the second radius information, For the i c The first intersection information, For the i c The second intersection information, g L is the first eyeball center information, g R is the second eyeball center information, i c is a positive integer greater than or equal to 1 and less than or equal to M.

[0037] As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining the line of sight estimation result information based on the first radius information and the second radius information includes:

[0038] S31, obtaining third eyeball center information and fourth eyeball center information;

[0039] S32, determining a first optical axis coordinate and a second optical axis coordinate based on the third eyeball center information, the fourth eyeball center information, the first radius information, and the second radius information;

[0040] S33, calculating and processing the first optical axis coordinate and the second optical axis coordinate to obtain first visual axis information and second visual axis information;

[0041] S34: Determine line of sight estimation result information based on the first visual axis information and the second visual axis information.

[0042] As an optional implementation manner, in the first aspect of the embodiment of the present invention, determining the first optical axis coordinates and the second optical axis coordinates based on the third eye center information, the fourth eye center information, the first radius information, and the second radius information includes:

[0043] Using a sixth fitting model, calculating and processing the third eyeball center information, the fourth eyeball center information, the first radius information, and the second radius information to obtain a first optical axis coordinate and a second optical axis coordinate;

[0044] Wherein, the sixth fitting model is:

[0045] (x L ,y L ,z L )=P*(x3,y3,z3)-Q*w L *(x3,y3,z3);

[0046] (x R ,y R ,z R )=P*(x4,y4,z4)-Q*w R *(x4,y4,z4);

[0047] In the formula, (x L ,y L ,z L ) is the first optical axis coordinate, (x R ,y R ,z R ) is the second optical axis coordinate, P and Q are the first weight coefficient and the second weight coefficient respectively, (x3, y3, z3) is the third eye center information, (x4, y4, z4) is the fourth eye center information, w L is the first radius information, w R is the second radius information.

[0048] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the calculating and processing the first optical axis coordinate and the second optical axis coordinate to obtain the first visual axis information and the second visual axis information includes:

[0049] S331, acquiring a first horizontal angle, a second horizontal angle, a first vertical angle, and a second vertical angle;

[0050] S332, using a seventh fitting model, fusing the first optical axis coordinate, the second optical axis model, the first horizontal angle, the second horizontal angle, the first vertical angle, and the second vertical angle to obtain first visual axis information and second visual axis information;

[0051] Wherein, the seventh fitting model is:

[0052] b L =C*(x L ,y L ,z L )*cosθ 1L +D*sinθ 2L ;

[0053] b R =E*(x R ,y R ,z R )*cosθ 1R +F*sinθ 2R ;

[0054] Where b L is the first visual axis information, b R is the second visual axis information, C, D, E and F are the first correction factor, the second correction factor, the third correction factor and the fourth correction factor respectively, (x L ,y L ,z L ) is the first optical axis coordinate, (x R ,y R ,z l ) is the second optical axis coordinate, θ 1L ,θ 2L ,θ 1R ,θ 2R They are respectively the first horizontal angle, the second horizontal angle, the first vertical angle and the second vertical angle.

[0055] A second aspect of an embodiment of the present invention discloses an image processing system for smart glasses, the system comprising:

[0056] An acquisition module, used for acquiring image information to be processed; the image information to be processed includes M first information to be processed and M second information to be processed;

[0057] A first calculation module, used for determining first radius information and second radius information based on the image information to be processed;

[0058] The second calculation module is used to determine the line of sight estimation result information based on the first radius information and the second radius information.

[0059] A third aspect of an embodiment of the present invention discloses another image processing system for smart glasses, the system comprising:

[0060] processor;

[0061] a memory coupled to the processor and storing executable program code;

[0062] The processor calls the executable program code stored in the memory to execute part or all of the steps in the image processing method for smart glasses disclosed in the first aspect of the embodiment of the present invention.

[0063] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute part or all of the steps of the image processing method for smart glasses disclosed in the first aspect of an embodiment of the present invention.

[0064] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0065] In an embodiment of the present invention, image information to be processed is obtained; the image information to be processed includes M first information to be processed and M second information to be processed; first radius information and second radius information are determined based on the image information to be processed; and sight line estimation result information is determined based on the first radius information and the second radius information. It can be seen that the present application is conducive to providing stable and reliable sight line estimation result information. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 A schematic diagram of a flow chart of an image processing method for smart glasses disclosed in an embodiment of the present invention;

[0068] Figure 2 A schematic diagram of the structure of an image processing system for smart glasses disclosed in an embodiment of the present invention;

[0069] Figure 3 The present invention is a schematic diagram of the structure of another image processing system for smart glasses disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0071] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0072] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0073] The present invention discloses an image processing method and system for smart glasses, which are conducive to providing stable and reliable line of sight estimation result information. Detailed descriptions are given below.

[0074] Embodiment 1

[0075] See also Figure 1 , Figure 1 1 is a flow chart of an image processing method for smart glasses disclosed in an embodiment of the present invention. Figure 1 The described image processing method for smart glasses is applied to an image processing system of smart glasses, such as a local server or a cloud server for image processing management of smart glasses, and the embodiments of the present invention are not limited thereto. Figure 1 As shown, an image processing method for smart glasses may include the following operations:

[0076] S1, obtaining image information to be processed; the image information to be processed includes M first information to be processed and M second information to be processed;

[0077] S2, determining first radius information and second radius information based on the image information to be processed;

[0078] S3: Determine line of sight estimation result information based on the first radius information and the second radius information.

[0079] It should be noted that the smart glasses are smart contact lenses, and a ring sensor is provided on the smart glasses;

[0080] It should be noted that the specific implementation steps for obtaining the image information to be processed include: the user first wears the smart contact lens and adjusts the smart contact lens to the center of the pupil. Based on the fit characteristics of the smart contact lens and the human eye, the annular sensor in the smart contact lens can be used to simulate the pupil contour of the human eye, and the annular sensor can be used to provide real-time feedback of the current sensor's position information based on the entire eye.

[0081] It should be noted that the use of the embodiments of the present application eliminates the need for the current mainstream pupil detection method, which is complex and not robust to lighting and wearing position, to locate the pupil position. Through the rapid positioning of the annular sensor, the image information to be processed can be accurately acquired.

[0082] It should be noted that the image processing method for smart glasses of the present application can be applied to head-mounted devices. Users can look at any position and watch at will by wearing head-mounted devices (eye trackers or eye pattern acquisition devices), and use near-eye infrared cameras to record multiple frames of eye images to form image sequences. At present, most eye tracking products require users to perform a complex user calibration process (five-point calibration or nine-point calibration) before using the eye tracking system. This method only requires users to look at any position at will during image acquisition and obtain the image information to be processed.

[0083] It can be seen that implementing the image processing method for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0084] In an optional embodiment, determining the first radius information and the second radius information based on the image information to be processed includes:

[0085] S21, performing fusion processing on the image information to be processed to obtain pre-processed image information; the pre-processed image information includes the M first pre-processed information and the M second pre-processed information;

[0086] S22, determining first eyeball center information and second eyeball center information based on the preprocessed image information;

[0087] S23: Determine first radius information and second radius information based on the preprocessed image information, the first eyeball center information and the second eyeball center information.

[0088] It should be noted that the fusion processing of the image information to be processed to obtain the preprocessed image information is to perform deprojection on the image information to be processed using the deprojection model to obtain the preprocessed image information. For example, after obtaining the image information to be processed, each of the first information to be processed and the second information to be processed is independently deprojected into a three-dimensional pupil circle in three-dimensional space. The space of the three-dimensional pupil circle projection is regarded as a cone with the three-dimensional pupil circle as the base and the camera focus as the vertex, and the three-dimensional pupil circle is the intersection of the cone and the image. Therefore, the deprojection of the three-dimensional pupil circle can be obtained by reconstructing the cone based on the three-dimensional pupil circle, wherein the three-dimensional pupil circle deprojection means that the pupil circle has two circular solutions at any given distance. After deprojection, the first preprocessing information and the second preprocessing information can be obtained, and the first preprocessing information and the second preprocessing information both include information such as the pupil position, the gaze vector and the pupil radius.

[0089] It should be noted that the solution projection model may be set by the user or obtained based on historical experience.

[0090] It can be seen that implementing the image processing method for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0091] In yet another optional embodiment, determining the first eye center information and the second eye center information based on the preprocessed image information includes:

[0092] S221: for any of the first preprocessing information, use a first fitting model to perform calculation processing on the first preprocessing information to obtain third preprocessing information corresponding to the first preprocessing information;

[0093] Wherein, the first fitting model is:

[0094] KL=ZL+s L *(NL);

[0095] Wherein, KL is the third preprocessing information, ZL is the pupil position corresponding to the first preprocessing information, NL is the gaze vector corresponding to the first preprocessing information, and s L is the first fixation vector length coefficient;

[0096] It should be noted that the first gaze vector length coefficient may be set by a user or obtained based on historical data, and the embodiment of the present invention does not make any specific limitation thereto.

[0097] S222: for any second preprocessing information, use a second fitting model to perform calculation processing on the second preprocessing information to obtain fourth preprocessing information corresponding to the second preprocessing information;

[0098] Wherein, the second fitting model is:

[0099] KR=ZR+s R *(NR);

[0100] Wherein, KR is the fourth preprocessing information, ZR is the pupil position corresponding to the second preprocessing information, NR is the gaze vector corresponding to the second preprocessing information, and s R is the second fixation vector length coefficient;

[0101] It should be noted that the second gaze vector length coefficient may be set by a user or obtained according to historical data, and the embodiment of the present invention does not make any specific limitation thereto.

[0102] S223, using a third fitting model, fusing the M pieces of the first preprocessed information and the third preprocessed information corresponding to the M pieces of the first preprocessed information to obtain first eye center information;

[0103] Wherein, the third fitting model is:

[0104]

[0105] In the formula, g L is the first eyeball center information, For the i a the third pre-processing information, For the i d a fixation vector corresponding to the first preprocessed information, for The transposed matrix of For the i a The pupil position corresponding to the first preprocessed information, α L is the first fusion coefficient, i a is a positive integer greater than or equal to 1 and less than or equal to M;

[0106] It should be noted that the first fusion coefficient may be set by a user or obtained based on historical data, and the embodiment of the present invention does not make any specific limitation thereto.

[0107] S224, using a fourth fitting model, fusing the M pieces of the second preprocessed information and the fourth preprocessed information corresponding to the M pieces of the second preprocessed information to obtain second eyeball center information;

[0108] Wherein, the fourth fitting model is:

[0109]

[0110] In the formula, gR is the second eyeball center information, For the i b the fourth pre-processing information, For the i b a fixation vector corresponding to the second preprocessing information, for The transposed matrix of For the i b The pupil position corresponding to the second preprocessing information, α R is the second fusion coefficient, i b is a positive integer greater than or equal to 1 and less than or equal to M.

[0111] It should be noted that the second fusion coefficient may be set by a user or obtained based on historical data, and the embodiment of the present invention does not make any specific limitation thereto.

[0112] It can be seen that implementing the image processing method for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0113] In another optional embodiment, determining the first radius information and the second radius information based on the preprocessed image information, the first eyeball center information and the second eyeball center information includes:

[0114] S231, for any of the first preprocessing information, analyzing and processing the first preprocessing information to obtain first intersection information corresponding to the first preprocessing information;

[0115] S232: for any second pre-processed information, analyze and process the second pre-processed information to obtain second intersection information corresponding to the second pre-processed information;

[0116] S233, using a fifth fitting model, calculating and processing the first intersection information, the second intersection information, the first eyeball center information, and the second eyeball center information to obtain first radius information and second radius information;

[0117] Wherein, the fifth fitting model is:

[0118]

[0119] Wherein, ZBJ is the first radius information, YBJ is the second radius information, For the i c The first intersection information, For the i c The second intersection information, g L is the first eyeball center information, gR is the second eyeball center information, i c is a positive integer greater than or equal to 1 and less than or equal to M.

[0120] It should be noted that, in the formula, and Used to represent g L arrive The distance and g R arrive distance.

[0121] It should be noted that, for any first preprocessing information, the first preprocessing information is analyzed and processed to obtain the first intersection information corresponding to the first preprocessing information, which is analyzed and processed by a first analysis model, and the first analysis model may be set by a user or may be obtained by training based on historical data;

[0122] It should be noted that, for any second pre-processed information, the second pre-processed information is analyzed and processed to obtain the second intersection information corresponding to the second pre-processed information, which is analyzed and processed by a second analysis model, and the second analysis model may be set by a user or may be obtained by training based on historical data;

[0123] It can be seen that implementing the image processing method for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0124] In yet another optional embodiment, determining the line of sight estimation result information based on the first radius information and the second radius information includes:

[0125] S31, obtaining third eyeball center information and fourth eyeball center information;

[0126] S32, determining a first optical axis coordinate and a second optical axis coordinate based on the third eyeball center information, the fourth eyeball center information, the first radius information, and the second radius information;

[0127] S33, calculating and processing the first optical axis coordinate and the second optical axis coordinate to obtain first visual axis information and second visual axis information;

[0128] S34: Determine line of sight estimation result information based on the first visual axis information and the second visual axis information.

[0129] It should be noted that the third eye center information and the fourth eye center information are obtained by wearing a head-mounted device (eye tracker or eye pattern acquisition device) and gazing at any position, and obtaining feedback through the ring sensor.

[0130] It can be seen that implementing the image processing method for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0131] In another optional embodiment, determining the first optical axis coordinates and the second optical axis coordinates based on the third eye center information, the fourth eye center information, the first radius information, and the second radius information includes:

[0132] Using a sixth fitting model, calculating and processing the third eyeball center information, the fourth eyeball center information, the first radius information, and the second radius information to obtain a first optical axis coordinate and a second optical axis coordinate;

[0133] Wherein, the sixth fitting model is:

[0134] (x L ,y L ,z L )=P*(x3,y3,z3)-Q*w L *(x3,y3,z3);

[0135] (x R ,y R ,z R )=P*(x4,y4,z4)-Q*w R *(x4,y4,z4);

[0136] In the formula, (x L ,y L ,z L ) is the first optical axis coordinate, (x R ,y R ,z R ) is the second optical axis coordinate, P and Q are the first weight coefficient and the second weight coefficient respectively, (x3, y3, z3) is the third eye center information, (x4, y4, z4) is the fourth eye center information, w L is the first radius information, w R is the second radius information.

[0137] It should be noted that the first weight coefficient and the second weight coefficient may be set by a user or acquired based on historical information, and the embodiment of the present invention does not specifically limit this.

[0138] It can be seen that implementing the image processing method for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0139] In yet another optional embodiment, the calculating and processing the first optical axis coordinate and the second optical axis coordinate to obtain the first visual axis information and the second visual axis information includes:

[0140] S331, acquiring a first horizontal angle, a second horizontal angle, a first vertical angle, and a second vertical angle;

[0141] S332, using a seventh fitting model, fusing the first optical axis coordinate, the second optical axis model, the first horizontal angle, the second horizontal angle, the first vertical angle, and the second vertical angle to obtain first visual axis information and second visual axis information;

[0142] Wherein, the seventh fitting model is:

[0143] b L =C*(x L ,y L ,z L )*cosθ 1L +D*sinθ 2L ;

[0144] b R =E*(x R ,y R ,z R )*cosθ 1R +F*sinθ 2R ;

[0145] Where b L is the first visual axis information, b R is the second visual axis information, C, D, E and F are the first correction factor, the second correction factor, the third correction factor and the fourth correction factor respectively, (x L ,y L ,z L ) is the first optical axis coordinate, (x R ,y R ,z R ) is the second optical axis coordinate, θ 1L ,θ 2L ,θ 1R ,θ 2R They are respectively the first horizontal angle, the second horizontal angle, the first vertical angle and the second vertical angle.

[0146] It should be noted that the first correction factor, the second correction factor, the third correction factor and the fourth correction factor may be set by a user or acquired based on historical data, and the embodiment of the present invention does not specifically limit this.

[0147] It should be noted that the first correction factor, the second correction factor, the third correction factor and the fourth correction factor are used to correct the first visual axis information and the second visual axis information so as to obtain optimal first visual axis information and second visual axis information.

[0148] Preferably, the first horizontal angle, the second horizontal angle, the first vertical angle and the second vertical angle are 5°, -5°, 1.5° and 1.5° respectively.

[0149] It can be seen that implementing the image processing method for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0150] In yet another optional embodiment, determining line of sight estimation result information based on the first visual axis information and the second visual axis information includes:

[0151] Using an eighth fitting model, calculating and processing the first visual axis information and the second visual axis information to obtain visual line estimation result information;

[0152] Wherein, the eighth fitting model is:

[0153]

[0154] Where R is the line of sight estimation result information, H is the position adjustment parameter, and μ are the fifth and sixth correction parameters, b L is the first visual axis information, b R It is the second visual axis information.

[0155] It should be noted that the position adjustment parameter may be set by a user or obtained based on historical data, and the position adjustment parameter is used to optimize the line of sight estimation result information;

[0156] It should be noted that the fifth correction parameter and the sixth correction parameter can be set by the user or obtained based on historical data. The fifth correction parameter and the sixth correction parameter are respectively used to correct the first visual axis information and the second visual axis information so as to reduce the difference between the first visual axis information and the second visual axis information.

[0157] It can be seen that implementing the image processing method for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0158] Embodiment 2

[0159] See also Figure 2 , Figure 2: is a schematic diagram of the structure of an image processing system for smart glasses disclosed in an embodiment of the present invention. Figure 2 The described image processing system for smart glasses is applied to an image processing device for smart glasses, such as a local server or a cloud server for image processing of smart glasses, which is not limited in the embodiments of the present invention. Figure 2 As shown, the image processing system for smart glasses includes:

[0160] The acquisition module 201 is used to acquire the image information to be processed; the image information to be processed includes M first information to be processed and M second information to be processed;

[0161] A first calculation module 202, configured to determine first radius information and second radius information based on the image information to be processed;

[0162] The second calculation module 203 is used to determine line of sight estimation result information based on the first radius information and the second radius information.

[0163] It can be seen that implementing the image processing system for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0164] Embodiment 3

[0165] See also Figure 3 , Figure 3 is a structural diagram of another image processing system for smart glasses disclosed in an embodiment of the present invention. Figure 3 The described image processing system for smart glasses is applied to an image processing device of smart glasses, such as a local server or a cloud server for image processing management of smart glasses, and the embodiments of the present invention are not limited thereto. Figure 3 As shown, the image processing system for smart glasses includes:

[0166] Processor 301;

[0167] A memory 302 coupled to the processor 301 and storing executable program codes;

[0168] The processor 301 calls the executable program code stored in the memory 302 to execute part or all of the steps of the image processing method for smart glasses described in Example 1.

[0169] It can be seen that implementing the image processing system for smart glasses described in the embodiment of the present invention is conducive to providing stable and reliable line of sight estimation result information.

[0170] Embodiment 4

[0171] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps in the image processing method for smart glasses described in the first embodiment.

[0172] Embodiment 5

[0173] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the image processing method for smart glasses described in the first embodiment.

[0174] The system embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative labor.

[0175] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0176] Finally, it should be noted that the image processing method and system for smart glasses disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image processing method for smart glasses, characterized in that: The method comprises: S1, obtaining image information to be processed; the image information to be processed includes M first information to be processed and M second information to be processed; S2, determining first radius information and second radius information based on the image information to be processed; S3: Determine line of sight estimation result information based on the first radius information and the second radius information.

2. The image processing method for smart glasses according to claim 1, characterized in that: The determining the first radius information and the second radius information based on the image information to be processed includes: S21, performing fusion processing on the image information to be processed to obtain pre-processed image information; the pre-processed image information includes the M first pre-processed information and the M second pre-processed information; S22, determining first eyeball center information and second eyeball center information based on the preprocessed image information; S23: Determine first radius information and second radius information based on the preprocessed image information, the first eyeball center information and the second eyeball center information.

3. The image processing method for smart glasses according to claim 2, characterized in that: The determining of the first eyeball center information and the second eyeball center information based on the pre-processed image information includes: S221: for any of the first preprocessing information, use a first fitting model to perform calculation processing on the first preprocessing information to obtain third preprocessing information corresponding to the first preprocessing information; Wherein, the first fitting model is: KL=ZL+s L *(NL); Wherein, KL is the third preprocessing information, ZL is the pupil position corresponding to the first preprocessing information, NL is the gaze vector corresponding to the first preprocessing information, and s L is the first fixation vector length coefficient; S222: for any second preprocessing information, use a second fitting model to perform calculation processing on the second preprocessing information to obtain fourth preprocessing information corresponding to the second preprocessing information; Wherein, the second fitting model is: KR=ZR+s R *(NR); Wherein, KR is the fourth preprocessing information, ZR is the pupil position corresponding to the second preprocessing information, NR is the gaze vector corresponding to the second preprocessing information, and s R is the second fixation vector length coefficient; S223, using a third fitting model, fusing the M pieces of the first preprocessed information and the third preprocessed information corresponding to the M pieces of the first preprocessed information to obtain first eye center information; Wherein, the third fitting model is: In the formula, g L is the first eyeball center information, For the i a the third pre-processing information, For the i a a fixation vector corresponding to the first preprocessed information, for The transposed matrix of For the i a The pupil position corresponding to the first preprocessed information, α L is the first fusion coefficient, i a is a positive integer greater than or equal to 1 and less than or equal to M; S224, using a fourth fitting model, fusing the M pieces of the second preprocessed information and the fourth preprocessed information corresponding to the M pieces of the second preprocessed information to obtain second eyeball center information; Wherein, the fourth fitting model is: In the formula, g R is the second eyeball center information, For the i b the fourth pre-processing information, For the i b a fixation vector corresponding to the second preprocessing information, for The transposed matrix of For the i b The pupil position corresponding to the second preprocessing information, α R is the second fusion coefficient, i b is a positive integer greater than or equal to 1 and less than or equal to M.

4. The image processing method for smart glasses according to claim 2, characterized in that: The determining of the first radius information and the second radius information based on the preprocessed image information, the first eyeball center information and the second eyeball center information comprises: S231, for any of the first preprocessing information, analyzing and processing the first preprocessing information to obtain first intersection information corresponding to the first preprocessing information; S232: for any second pre-processed information, analyze and process the second pre-processed information to obtain second intersection information corresponding to the second pre-processed information; S233, using a fifth fitting model, calculating and processing the first intersection information, the second intersection information, the first eyeball center information, and the second eyeball center information to obtain first radius information and second radius information; Wherein, the fifth fitting model is: Wherein, ZBJ is the first radius information, YBJ is the second radius information, For the i c The first intersection information, For the i c The second intersection information, g L is the first eyeball center information, g R is the second eyeball center information, i c is a positive integer greater than or equal to 1 and less than or equal to M.

5. The image processing method for smart glasses according to claim 1, characterized in that: The determining, based on the first radius information and the second radius information, line of sight estimation result information includes: S31, obtaining third eyeball center information and fourth eyeball center information; S32, determining a first optical axis coordinate and a second optical axis coordinate based on the third eyeball center information, the fourth eyeball center information, the first radius information, and the second radius information; S33, calculating and processing the first optical axis coordinate and the second optical axis coordinate to obtain first visual axis information and second visual axis information; S34: Determine line of sight estimation result information based on the first visual axis information and the second visual axis information.

6. The image processing method for smart glasses according to claim 5, characterized in that: The determining of the first optical axis coordinate and the second optical axis coordinate based on the third eyeball center information, the fourth eyeball center information, the first radius information and the second radius information comprises: Using a sixth fitting model, calculating and processing the third eyeball center information, the fourth eyeball center information, the first radius information, and the second radius information to obtain a first optical axis coordinate and a second optical axis coordinate; Wherein, the sixth fitting model is: (x L ,y L ,z L )=P*(x3,y3,z3)-Q*w L *(x3,y3,z3); ( x R ,y R ,z R )=P*(x4,y4,z4)-Q*w R x(x4,y4,z4); In the formula, (x L ,y L ,z L ) is the first optical axis coordinate, (x R ,y R ,z R ) is the second optical axis coordinate, P and Q are the first weight coefficient and the second weight coefficient respectively, (x3, y3, z3) is the third eye center information, (x4, y4, z4) is the fourth eye center information, w L is the first radius information, w R is the second radius information.

7. The image processing method for smart glasses according to claim 5, characterized in that: The calculating and processing the first optical axis coordinate and the second optical axis coordinate to obtain the first visual axis information and the second visual axis information includes: S331, acquiring a first horizontal angle, a second horizontal angle, a first vertical angle, and a second vertical angle; S332, using a seventh fitting model, fusing the first optical axis coordinate, the second optical axis model, the first horizontal angle, the second horizontal angle, the first vertical angle, and the second vertical angle to obtain first visual axis information and second visual axis information; Wherein, the seventh fitting model is: b L =C*(x L ,y L ,z L )*cosθ 1L +D*sinθ 2L ; b R =E*(x R ,y R ,z R )*cosθ 1R +F*sinθ 2R ; Where b L is the first visual axis information, b R is the second visual axis information, C, D, E and F are the first correction factor, the second correction factor, the third correction factor and the fourth correction factor respectively, (x L ,y L ,z L ) is the first optical axis coordinate, (x R ,y R ,z R ) is the second optical axis coordinate, θ 1L ,θ 2L ,θ 1R ,θ 2R They are respectively the first horizontal angle, the second horizontal angle, the first vertical angle and the second vertical angle.

8. An image processing system for smart glasses, characterized in that: The system comprises: An acquisition module, used for acquiring image information to be processed; the image information to be processed includes M first information to be processed and M second information to be processed; A first calculation module, used for determining first radius information and second radius information based on the image information to be processed; The second calculation module is used to determine the line of sight estimation result information based on the first radius information and the second radius information.

9. An image processing system for smart glasses, characterized in that: The system comprises: processor; a memory coupled to the processor and storing executable program code; The processor calls the executable program code stored in the memory to execute the image processing method for smart glasses according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the image processing method for smart glasses according to any one of claims 1 to 7.

Images