Authentication method and authentication system for smart glasses interconnection, readable storage medium

Abstract

An authentication method and system for smart glasses interconnection, and a readable storage medium. The authentication method comprises: establishing an interconnection platform; the smart glasses enter the interconnection platform after sending an interconnection request; judging whether the smart glasses entering the interconnection platform are at the same space-time position; if yes, judging whether the smart glasses are in a closed space; if no, pairing and connecting different smart glasses through a password input mode; when it is judged that the smart glasses are in a closed space, directly pairing and connecting different smart glasses; when it is judged that the smart glasses are not in a closed space, pairing and connecting different smart glasses through a password input mode; and the smart glasses successfully paired and connected enter the same virtual conference scene.

Description

Technical Field

[0001] This disclosure pertains to the field of smart glasses, specifically relating to an authentication method, authentication system, and readable storage medium for interconnecting smart glasses. Background Technology

[0002] With the development of smart glasses, they have gradually evolved from standalone display and interaction devices into multi-device interconnected scenarios. Compared to the inconvenience of traditional displays, the compactness, flexibility, and ease of use of smart glasses are increasingly favored by people. Summary of the Invention

[0003] This disclosure provides an authentication method, authentication system, and readable storage medium for interconnected smart glasses.

[0004] In a first aspect, embodiments of this disclosure provide an authentication method for interconnecting smart glasses, comprising: establishing an interconnection platform;

[0005] After the smart glasses send an interconnection request, they enter the interconnection platform.

[0006] Determine whether the smart glasses entering the interconnection platform are in the same spatiotemporal location; if so, determine whether the smart glasses are in a closed space; if not, pair and connect different smart glasses by entering a password.

[0007] When it is determined that the smart glasses are in the enclosed space, pairing and connection between different smart glasses are directly performed.

[0008] When it is determined that the smart glasses are not in the enclosed space, pairing and connection between different smart glasses are performed by entering a password.

[0009] Once the smart glasses are successfully paired and connected, they enter the same virtual meeting scene.

[0010] In some embodiments, determining whether the smart glasses entering the interconnected platform are in the same spatiotemporal location includes:

[0011] The network time protocol is used to control the timestamp error of different smart glasses to be less than or equal to a first threshold; different smart glasses are set to have the same video resolution and frame rate, and video frame images are captured synchronously; different smart glasses are set to have the same audio sampling rate, and different audio information is recorded synchronously; the ambient light brightness information around the smart glasses is tested, and the ambient light brightness information is uploaded to the data processing platform;

[0012] The smart glasses are used to determine whether they are candidate terminals in the same space by matching visual feature points.

[0013] The smart glasses are determined to be candidate terminals in the same space by matching audio signals.

[0014] The system determines whether the smart glasses are candidate terminals in the same space by matching ambient light brightness information.

[0015] The multimodal fusion is used to determine whether the candidate terminals are in the same space.

[0016] In some embodiments, determining whether the smart glasses are candidate terminals in the same space by matching visual feature points includes:

[0017] Feature points are extracted from the images captured by the smart glasses, and the number of extracted feature points is set to a second threshold.

[0018] The feature points are matched and verified to select valid matching points that conform to spatial perspective relationships;

[0019] Determine whether the number of valid matching points is greater than or equal to a set percentage of the second threshold;

[0020] If yes, then the smart glasses are the candidate terminals in the same space; if no, then the smart glasses cannot be the candidate terminals.

[0021] In some embodiments, determining whether the smart glasses are candidate terminals in the same space by matching audio signals includes:

[0022] Extract the Mel-frequency cepstral coefficients and short-time energy at the moment of sound source location from the audio signals of the different smart glasses;

[0023] Calculate the cosine similarity of the Mel frequency cepstral coefficients of the audio signals of different smart glasses. If the cosine similarity is not lower than a third threshold, then the audio signals of different smart glasses have the same characteristics.

[0024] The starting time of the sound source is located by the peak value of the short-time energy, and the time difference of the audio signals of different smart glasses is calculated. If the time difference is less than the fourth threshold, the smart glasses are candidate terminals in the same space.

[0025] In some embodiments, determining whether the smart glasses are candidate terminals in the same space by matching ambient light brightness information includes:

[0026] Define an initial time point, detect the ambient light brightness information of different smart glasses in real time, and extract the ambient light brightness value according to the set time period;

[0027] Starting from the initial time point, the brightness value of the ambient light of the smart glasses is subjected to curve fitting;

[0028] Determine whether the similarity of the fitting curves of different smart glasses is above the fifth threshold; if so, the smart glasses are candidate terminals in the same space.

[0029] In some embodiments, determining whether the candidate terminals are in the same space through multimodal fusion includes:

[0030] Weights are assigned to the matching of visual feature points, audio signals, and ambient light brightness information of the candidate terminals, respectively, and the weighted scores of the candidate terminals are calculated.

[0031] If the weighted score of the candidate terminal is greater than the sixth threshold, then the candidate terminals are determined to be in the same space.

[0032] If the weighted score of the candidate terminal is less than the seventh threshold, it is determined that the candidate terminals are not in the same space; the seventh threshold is less than the sixth threshold.

[0033] If the weighted score of the candidate terminal is greater than or equal to the seventh threshold and less than or equal to the sixth threshold, then 3D reconstruction is performed based on the image data captured by the candidate terminal.

[0034] If the alignment of the point cloud after 3D reconstruction is greater than the eighth threshold, then the candidate terminals are determined to be in the same space.

[0035] In some embodiments, the password input method includes hand password input, head password input, voice password input, video image password input, and absolute display space password input.

[0036] In some embodiments, the method of inputting the absolute display space password includes:

[0037] Determine the position coordinates of the display interface of the smart glasses in the first direction in the initial state; set the maximum and minimum values ​​of the distance change between the smart glasses and the display interface in the first direction;

[0038] When inputting a password, the position of the display interface of the smart glasses in the first direction remains unchanged, and the distance between the smart glasses and the display interface along the first direction is adjusted to vary between the maximum value and the minimum value, so as to input the corresponding password value to different password input positions;

[0039] When the distance between the smart glasses and the display interface along the first direction decreases, the password value input at the corresponding password input position increases; when the distance between the smart glasses and the display interface along the first direction increases, the password value input at the corresponding password input position decreases.

[0040] If no change in distance along the first direction is detected between the smart glasses and the display interface within a set time, then the password value input at the current password input position is determined to be the password value corresponding to the distance between the smart glasses and the display interface along the first direction.

[0041] In some embodiments, the method of inputting the absolute display space password further includes:

[0042] Determine the position coordinates of the display interface of the smart glasses in the second direction in the initial state; set an adjustment gradient threshold for the distance between the smart glasses and the display interface in the second direction;

[0043] When entering a password, the position of the display interface of the smart glasses in the second direction remains unchanged, the change value of the distance between the smart glasses and the display interface along the second direction is detected, and the change value of the distance is compared with the gradient threshold.

[0044] When the absolute value of the distance change is greater than the gradient threshold, the password input bit is moved along the second direction;

[0045] When the distance change value is negative, the password input position is moved to the left along the second direction; when the distance change value is positive, the password input position is moved to the right along the second direction.

[0046] In some embodiments, the method of inputting the absolute display space password further includes:

[0047] Let the total number of password input bits be M.

[0048] When nk is satisfied T ≤ ≤(n+1)k T If so, then move n password input bits to the left or right along the second direction;

[0049] in, x is the distance change value, k T The gradient threshold;

[0050] When the number of password input positions moved is greater than the difference between M and the current position of the password input position, the movement stops when the last password input position is reached, or the movement continues to cycle to the first password input position when the last password input position is reached.

[0051] Secondly, this disclosure also provides an authentication system for interconnecting smart glasses, including: a main control module configured to receive interconnection requests sent by the smart glasses and control the smart glasses to enter the interconnection platform after sending the interconnection request;

[0052] The processing module is configured to determine whether the smart glasses entering the interconnection platform are in the same spatiotemporal location; it is also configured to determine whether the smart glasses are in a closed space.

[0053] The main control module is also configured to, when it is determined that the smart glasses entering the interconnection platform are not in the same spatiotemporal location, control the pairing connection between different smart glasses through password input; when it is determined that the smart glasses are in the enclosed space, control the direct pairing connection between different smart glasses; and when it is determined that the smart glasses are not in the enclosed space, control the pairing connection between different smart glasses through password input.

[0054] The main control module is also configured to control the successfully paired smart glasses to enter the same virtual meeting scene.

[0055] Thirdly, embodiments of this disclosure also provide a readable storage medium having a program stored thereon that implements the above-described authentication method when executed by a processor. Attached Figure Description

[0056] The accompanying drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0057] Figure 1A This is a flowchart of a smart glasses interconnection authentication method in an embodiment of this disclosure.

[0058] Figure 1B This is another flowchart of the smart glasses interconnection authentication method in the embodiments of this disclosure.

[0059] Figure 2 A flowchart for determining whether smart glasses entering the interconnected platform are in the same spatiotemporal location.

[0060] Figure 3A This is a flowchart illustrating the absolute display space password input method in this embodiment of the present disclosure.

[0061] Figure 3B This is a block diagram illustrating the principle of the absolute display space password input method in this embodiment of the present disclosure.

[0062] Figure 3C This is a schematic diagram showing the relative positions of the smart glasses and the display interface in an embodiment of this disclosure.

[0063] Figure 3D This is a schematic diagram of the password input interface for the absolute display space password input method in this embodiment of the present disclosure.

[0064] Figure 4A This is a diagram showing the password information for the left and right hands in an embodiment of this disclosure.

[0065] Figure 4B This is a comparison chart of the security levels of cryptographic combinations and traditional cryptographic combinations in the embodiments of this disclosure.

[0066] Figure 5 This is a schematic diagram of head posture in the head password input method of this embodiment.

[0067] Figure 6 This is a schematic diagram of the password input interface and principle of the voice password input method in this embodiment of the present disclosure.

[0068] Figure 7 This is a schematic diagram of the password input interface and principle of the video image password input method in this embodiment of the present disclosure.

[0069] Figure 8 This is a block diagram illustrating the principle of the authentication system for interconnected smart glasses in this embodiment of the present disclosure. Detailed Implementation

[0070] To enable those skilled in the art to better understand the technical solutions of the embodiments of this disclosure, the following describes in further detail, with reference to the accompanying drawings and specific embodiments, an authentication method, authentication system, and readable storage medium for interconnected smart glasses provided by the embodiments of this disclosure.

[0071] Embodiments of this disclosure will be described more fully below with reference to the accompanying drawings; however, the embodiments shown may be embodied in different forms and should not be construed as limited to the embodiments set forth in this disclosure. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this disclosure.

[0072] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of objects. "Above," "below," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0073] As used herein, “parallel” and “perpendicular” include the described situation and situations that are similar to the described situation, within an acceptable range of deviation, which is determined by those skilled in the art taking into account the measurement under discussion and the error associated with the measurement of a particular quantity (i.e., the limitations of the measurement system). For example, “parallel” includes absolute parallelism and approximate parallelism, where an acceptable range of deviation for approximate parallelism may be, for example, within 5°; “perpendicular” includes absolute perpendicularity and approximate perpendicularity, where an acceptable range of deviation for approximate perpendicularity may also be, for example, within 5°.

[0074] It should be understood that when a layer or element is referred to as being on another layer or substrate, it can mean that the layer or element is directly on the other layer or substrate, or that there is an intermediate layer between the layer or element and the other layer or substrate.

[0075] This document describes exemplary embodiments with reference to sectional views and / or plan views, which are idealized exemplary drawings. In the drawings, the thickness of layers and regions is enlarged for clarity. Therefore, variations in shape relative to the drawings are contemplated due to, for example, manufacturing techniques and / or tolerances. Therefore, exemplary embodiments should not be construed as limited to the shapes of the regions shown herein, but rather include shape deviations due to, for example, manufacturing processes. Thus, the regions shown in the drawings are schematic in nature, and their shapes are not intended to show the actual shapes of the regions of the device, nor are they intended to limit the scope of the exemplary embodiments.

[0076] Currently, the interconnection of multiple smart glasses devices inevitably faces security issues, especially in outdoor environments where there are many and varied devices, posing serious risks to the security of multi-device connections.

[0077] To address the aforementioned problems in the related technologies, in a first aspect, embodiments of this disclosure provide an authentication method for interconnecting smart glasses, such as... Figure 1A and Figure 1B As shown, it includes: Step S101: Establishing an interconnection platform; After the smart glasses send an interconnection request, they enter the interconnection platform;

[0078] Step S102: Determine whether the smart glasses entering the interconnection platform are in the same space-time location; if yes, proceed to step S103: Determine whether the smart glasses are in a closed space; if no, proceed to step S104: Pair and connect different smart glasses by entering a password.

[0079] When it is determined that the smart glasses are in an enclosed space, step S105 is executed: pairing and connecting different smart glasses directly; this method can greatly improve the pairing efficiency of different smart glasses.

[0080] If it is determined that the smart glasses are not in a closed space, then step S104 is executed: pairing and connecting different smart glasses by entering a password;

[0081] Smart glasses that have successfully paired and connected enter the same virtual meeting scene. Multiple smart glasses can share file content within the same virtual meeting scene.

[0082] In some embodiments, step S103: determine whether the smart glasses are in a closed space, such as a conference room or a living room. The determination of a closed space is mainly achieved by extracting spatial feature parameters, such as information about doors, windows, and walls in the space, as well as 3D spatial information reconstructed from data captured by the camera, to determine whether it is a closed space.

[0083] The authentication method in this embodiment solves the connection permission problem when multiple smart glasses are interconnected, and can prevent other smart glasses that do not need to be interconnected or are malicious from accidentally entering the meeting, thereby ensuring the security of interconnection of multiple smart glasses.

[0084] In this embodiment, the application scenarios of interconnecting multiple smart glasses can be divided into smart glasses interconnection scenarios in the same space and smart glasses interconnection scenarios in different spaces.

[0085] In some embodiments, step S102: determining whether the smart glasses entering the interconnected platform are in the same spatiotemporal location includes: such as Figure 2 As shown, step S201: Control the timestamp error of different smart glasses to be less than or equal to a first threshold through the network time protocol; set different smart glasses to have the same video resolution and frame rate, and synchronously capture video frame images; set different smart glasses to have the same audio sampling rate, and synchronously record different audio information; test the ambient light brightness information around the smart glasses, and upload the ambient light brightness information to the data processing platform.

[0086] In this step, the first threshold can be 1ms. The timestamp error between different smart glasses should be less than or equal to the first threshold to avoid misjudgments caused by data asynchrony. The smart glasses' cameras are accessed via a vision library, and the same resolution and frame rate are set for each camera. The smart glasses' microphones are accessed via an audio library, and the same audio sampling rate is set for each microphone. The ambient light intensity around the smart glasses is tested using an ambient light sensor.

[0087] Step S202: Determine whether the smart glasses are candidate terminals in the same space by matching visual feature points.

[0088] Step S203: Determine whether the smart glasses are candidate terminals in the same space by matching the audio signals.

[0089] Step S204: Determine whether the smart glasses are candidate terminals in the same space by matching ambient light brightness information.

[0090] Step S205: Determine whether candidate terminals are in the same space through multimodal fusion.

[0091] In this embodiment, the smart glasses first send an interconnection request, and the smart glasses that have sent the interconnection request will automatically log in to the interconnection platform. After entering the interconnection platform, the current spatiotemporal position of the smart glasses will be determined first to determine whether the smart glasses that have entered the interconnection platform are in the same spatiotemporal position. The determination method mainly adopts a multimodal fusion method of visual detection, audio detection and ambient light brightness detection.

[0092] In this embodiment, the hardware mainly includes smart glasses with a camera, microphone, ambient light sensor, and display and network functions; and a data processing platform capable of remote data processing. The software mainly includes a visual library capable of extracting video feature points and reconstructing 3D; an audio library capable of extracting audio features and software for recording; data comparison software after optical detection; and a multimodal fusion model based on TensorFlow / PyTorch.

[0093] In some embodiments, step S202: determining whether the smart glasses are candidate terminals in the same space by matching visual feature points includes: step S11: extracting feature points from the image captured by the smart glasses, and setting the number of extracted feature points to a second threshold.

[0094] In this step, color images captured by different smart glasses are converted to grayscale images, and keypoints and descriptor parameters (i.e., feature points) are extracted using the feature detection function cv2.ORB_create(), with the number of these feature points (i.e., keypoints and descriptor parameters) set to 2000. The second threshold can be set arbitrarily and is not limited to 2000.

[0095] Step S12: Perform feature matching and verification on the feature points, and select valid matching points that conform to the spatial perspective relationship.

[0096] In this step, the brute-force matcher BFMatcher is used to match key points and descriptor parameters, and incorrect matches are filtered out by the "ratio test". Then, homography verification is performed. For the matched feature points, the homography matrix is ​​calculated using the RANSAC algorithm, and "valid matching points" that conform to the spatial perspective relationship are selected.

[0097] Step S13: Determine whether the number of valid matching points is greater than or equal to the set percentage of the second threshold; if yes, the smart glasses are candidate terminals in the same space; if no, the smart glasses cannot be candidate terminals.

[0098] In this step, the logical relationship is determined by checking whether the number of valid matching points is not less than 20% of the second threshold, thus filtering out candidate results in the same space. The percentage of the second threshold is not limited to 20% and can be other proportions.

[0099] In some embodiments, step S203: determining whether smart glasses are candidate terminals in the same space by matching audio signals includes: step S21: extracting the Mel frequency cepstral coefficients of audio signals from different smart glasses and the short-time energy at the moment of locating the sound source.

[0100] In this step, the Librosa library is used to extract the Mel frequency cepstral coefficients and the short-time energy (STE) at the moment of locating the sound source from the audio signals of different smart glasses. The Mel frequency cepstral coefficients and the STE at the moment of locating the sound source are characteristic points that characterize the audio signals.

[0101] Step S22: Calculate the cosine similarity of the Mel frequency cepstral coefficients of the audio signals of different smart glasses. If the cosine similarity is not lower than the third threshold, then the audio signal characteristics of different smart glasses are consistent.

[0102] In this step, the third threshold can be 95%, but is not limited to that value.

[0103] Step S23: Locate the start time of the sound source by the peak value of the short-time energy, and calculate the time difference of the audio signals of different smart glasses. If the time difference is less than the fourth threshold, the smart glasses are candidate terminals in the same space.

[0104] In this step, the fourth threshold can be 10ms, but is not limited to that value.

[0105] In some embodiments, step S204: determining whether the smart glasses are candidate terminals in the same space by matching ambient light brightness information includes: step S31: defining an initial time point, detecting the ambient light brightness information of different smart glasses in real time, and extracting the brightness value of ambient light according to a set time period.

[0106] Step S32: Starting from the initial time point, perform curve fitting on the ambient light brightness value of the smart glasses.

[0107] Step S33: Determine whether the similarity of the fitting curves of different smart glasses is above the fifth threshold; if so, the smart glasses are candidate terminals in the same space.

[0108] In this step, the fifth threshold can be 90%, but is not limited to that value.

[0109] In some embodiments, step S205: determining whether candidate terminals are in the same space through multimodal fusion includes: step S41: setting the weights for matching visual feature points, audio signals, and ambient light brightness information of candidate terminals respectively, and calculating the weighted score of candidate terminals.

[0110] In this step, the matching weight for visual feature points can be set to 0.5, the matching weight for audio signals to 0.4, and the matching weight for ambient light brightness information to 0.1.

[0111] Step S42: If the weighted score of the candidate terminal is greater than the sixth threshold, then the candidate terminals are determined to be in the same space.

[0112] In this step, the sixth threshold can be 0.8, but is not limited to this value.

[0113] Step S43: If the weighted score of the candidate terminal is less than the seventh threshold, it is determined that the candidate terminals are not in the same space; the seventh threshold is less than the sixth threshold.

[0114] In this step, the seventh threshold can be 0.5, but is not limited to this value.

[0115] Step S44: If the weighted score of the candidate terminal is greater than or equal to the seventh threshold and less than or equal to the sixth threshold, then perform 3D reconstruction based on the image data captured by the candidate terminal.

[0116] Step S45: If the alignment of the 3D reconstructed point cloud is greater than the eighth threshold, then the candidate terminals are determined to be in the same space.

[0117] In this step, the eighth threshold can be 80%, but is not limited to that value.

[0118] In some embodiments, password input methods include hand password input, head password input, voice password input, video image password input, and absolute display space password input.

[0119] In some embodiments, the absolute display of the spatial password input includes: such as Figure 3A , Figure 3B , Figure 3C and Figure 3D As shown, step S301: Determine the position coordinates of the display interface of the smart glasses in the first direction in the initial state; set the maximum and minimum values ​​of the distance change between the smart glasses and its display interface in the first direction.

[0120] In this step, an IMU (Inertial Measurement Unit) is integrated into the smart glasses. The IMU uses a spatial attitude sensor composed of an accelerometer and a gyroscope to monitor the spatial attitude changes of the smart glasses, thereby monitoring the distance changes between the smart glasses and their display interface.

[0121] In addition, the smart glasses also include a main processing unit, a timer unit, an audio unit, and a display unit. The main processing unit includes a main control module and a processing module, and is used for processing audio and video content, storing content, processing data, and analyzing data. The display unit is the display component of the smart glasses, used to provide users with displayable content. The audio unit is mainly used for voice interaction with the user. The timer unit uses a crystal oscillator and response circuitry to set the time and send the time result to the main processing unit to assist the main processing unit in completing the corresponding data processing and triggering settings for corresponding functions.

[0122] Step S302: When entering the password, keep the position of the display interface of the smart glasses unchanged in the first direction, and adjust the distance between the smart glasses and the display interface along the first direction to vary between the maximum and minimum values, so as to input the corresponding password value to different password input positions.

[0123] In this step, for example, assuming the spatial coordinates of the smart glasses in the first direction are s0, then in the initial state, the position coordinates of the smart glasses' display interface in the first direction are s0+s. When entering a password, the position coordinates of the display interface in the first direction remain fixed.

[0124] When a user receives a voice prompt from the smart glasses, they are prompted to enter a password. Initially, the display shows password input boxes; the number of input boxes corresponds to the number of digits in the password. The first password input box can be empty, set to an initial value such as 5, or set to extreme values ​​like 0 or 9. The password value in the input boxes on the display is adjusted by detecting changes in the distance between the smart glasses and the display using an IMU (Inertial Measurement Unit).

[0125] Step S303: When the distance between the smart glasses and its display interface along the first direction decreases, the password value entered in the corresponding password input field increases; when the distance between the smart glasses and its display interface along the first direction increases, the password value entered in the corresponding password input field decreases.

[0126] In this step, for example, the initial password value can be set to 5. The IMU (Inertial Measurement Unit) detects changes in the distance between the smart glasses and their display interface, thereby adjusting the input password value. For instance, if the distance between the display interface and the smart glasses decreases relative to their initial distance, the password value on the input field of the display interface can be increased synchronously; conversely, if the distance increases relative to their initial distance, the password value on the input field can be decreased synchronously. However, the actual method of setting the initial password value and adjusting the password value is not limited to this method.

[0127] Step S304: If no change in distance between the smart glasses and its display interface along the first direction is detected within a set time, then the password value entered at the current password input position is determined to be the password value corresponding to the distance between the smart glasses and its display interface along the first direction.

[0128] Users can input password values ​​for other password input fields using the same password input methods as described above.

[0129] In this embodiment, the absolute display space password input method, since the smart glasses are worn on the user's head, can synchronously adjust the distance between the user's eyes and the display interface by changing the distance between the smart glasses and the display interface. This adjusts the user's viewing distance to be farther or closer, thereby providing the user with a visual impact and experience, effectively improving the user's interactive experience of viewing the content displayed on the display interface; at the same time, it can also effectively improve the accuracy of password setting.

[0130] In this embodiment, adjusting the distance between the smart glasses and its display interface can be achieved by adjusting the distance between the user's head and the display interface, or by adjusting the distance between the user's body and the display interface, such as by moving the user's head forward or backward along a first direction, or by tilting the user's body forward or backward along a first direction.

[0131] In some embodiments, the absolute display space password input method further includes: step S305: determining the position coordinates of the display interface of the smart glasses in the second direction in the initial state; setting an adjustment gradient threshold for the distance between the smart glasses and its display interface in the second direction.

[0132] In this step, assume that the position coordinate of the smart glasses in the second direction is k0, and the position coordinate of the display interface in the second direction in the initial state is k0+k. That is, in the initial state, the relative distance between the smart glasses and the display interface in the second direction is k.

[0133] Step S306: When entering the password, keep the position of the smart glasses' display interface unchanged in the second direction, detect the change in distance between the smart glasses and its display interface along the second direction, and compare the change in distance with the gradient threshold; when the absolute value of the change in distance is greater than the gradient threshold, move the password input position along the second direction.

[0134] When the distance change value is negative, the password input position is moved to the left along the second direction; when the distance change value is positive, the password input position is moved to the right along the second direction.

[0135] In this step, when the password is entered, the position coordinates of the fixed display interface in the second direction remain unchanged. The password input position is adjusted by changing the position coordinates of the smart glasses in the second direction.

[0136] In some embodiments, the absolute display spatial password input method further includes: step S307: setting the total number of password input bits to M, when nk is satisfied T ≤ ≤(n+1)k T If n is the password input position, then move n positions to the left or right along the second direction.

[0137] in, x represents the distance change value, k T This is the gradient threshold. When... When x is positive, shift the password input position n positions to the right along the second direction; when When the value of x is negative, the password input position is shifted to the left by n bits along the second direction.

[0138] Step S308: When the number of password input positions moved is greater than the difference between M and the current password input position, the movement stops when the last password input position is reached, or the movement continues to cycle to the first password input position when the last password input position is reached.

[0139] In this embodiment, the first direction and the second direction are perpendicular, and the position of the display interface is fixed. By changing the position of the smart glasses in the first direction and the second direction, the relative position change between the smart glasses and the display interface is measured by the IMU inertial measurement unit, thereby adjusting the value of the password and the password input position.

[0140] In this embodiment, since the position of the display interface is fixed, the relative position between the user and the smart glasses is also fixed. When the relative position between the smart glasses and the display interface changes, the relative position between the user and the display interface also changes accordingly. Thus, the user can perceive the changes in the display interface in the first and second directions, thereby further enhancing the visual interaction experience when the user views the content displayed on the display interface.

[0141] In some embodiments, the hand password input method includes: when different smart glasses are not in the same enclosed space, a voice prompt is made through the audio playback unit of the smart glasses, playing the message "Is it convenient to join the meeting using a hand password?"; the smart glasses user can answer "yes" or "no" by voice. When it is convenient for the user to join the meeting using a hand password, a combination of left and right hands is used to input the hand password.

[0142] In some embodiments, such as Figure 4A As shown, the five fingers of both hands are numbered 1-5. When a smart glasses user joins the membership via hand password input, the password values ​​for different input positions are set by configuring the finger numbers and different combinations of fingers. Table 1 below shows the truth table for gesture passwords.

[0143] Table 1

[0144] left hand right hand password - 1 1 - 2 2 - 3 3 - 4 4 - 5 5 1 - 6 1 1 7 1 2 8 1 3 9 1 4 10 1 5 11 2 - 12 2 1 13 2 2 14 2 3 15 2 4 16 2 5 17 3 - 18 3 1 19 3 2 20 3 3 21 3 4 22 3 5 23 4 - 24 4 1 25 4 2 26 4 3 27 4 4 28 4 5 29 5 - 30 5 1 31 5 2 32 5 3 33 5 4 34 5 5 35

[0145] In this embodiment, as Figure 4BAs shown, multiple passwords can be set using combinations of fingers from both hands. The system allows setting the password length. Figure 4B As can be seen, passwords set via finger combination are significantly more secure than traditional decimal passwords, especially when the password has four or more digits. In this embodiment, the security level of the password is far superior to that of traditional passwords. Using hand-based password input effectively avoids security issues caused by external devices lacking connectivity, malicious smart glasses attacks, or unauthorized access to meetings.

[0146] In some embodiments, when a user enters a hand gesture password, the smart glasses combine the intelligent voice playback system with gesture input. When the user hears the voice prompt, "Please enter the first digit of the password," they can enter the password information by using the corresponding gesture. When the smart glasses detect the user's entered password, they will verbally remind the user that the password is valid and inform them of the current detection result; or when the smart glasses detect that the user's entered password is invalid, they will remind the user to pay attention to their gesture and re-enter the password.

[0147] In some embodiments, when it is inconvenient for the user to input the password using their hands, a head-based password input method is used. Head-based password input methods include: inputting the password through head posture detection, such as... Figure 5 As shown, passwords are set by a combination of head movements, and the number of digits in the password can be set as needed.

[0148] In some embodiments, when using head-based password input, the smart glasses' voice playback unit combines with the head-based password input. When the user hears "Please enter the first digit of the password," the user can input the password information by making the corresponding head movements. When the smart glasses effectively detect the user's password input, they will verbally remind the user that the password input is valid and inform them of the current detection result. Alternatively, when the smart glasses detect that the user's password input is invalid, they will remind the user to pay attention to their head posture and re-enter the password. Table 2 below shows the truth table for head posture passwords.

[0149] Table 2

[0150] action password Turn left 1 Turn right 2 Turn left first, then turn right. 3 Turn right first, then turn left 4

[0151] In some embodiments, when users are in a situation where interaction is inconvenient, they can enter passwords via voice. For example... Figure 6 As shown, the voice password input method allows password setting to be completed by measuring the duration of the voice input. The voice password input method can also select the actual user's voice by recording the user's voiceprint information.

[0152] When the user is not speaking, no corresponding password number is displayed on the password input interface. When the user begins to speak, the password input interface initially displays "0," and then the password value changes accordingly based on the duration of the speech; for example, the password value increases as the speech duration increases. Additionally, a time threshold is set. If the smart glasses do not detect any user speech within the time threshold, the password number is locked at a certain value, and the password input value for the corresponding password input field is determined. The user can use the above password input method to input password values ​​for other password input fields.

[0153] In some embodiments, such as Figure 7 As shown, the video image password input method includes: using the camera in the smart glasses to detect the size of the user's eye opening and closing in their facial expressions, thereby inputting the password. The smart glasses can provide voice prompts to the user to input the password based on the size of their eye opening and closing. Specifically, for example, the initial value of the eye opening and closing is first set to the password value 5, and then the password value is adjusted by changing the size of the eye opening and closing, with the adjustment range between 0 and 9. When it is detected that the user's eye opening and closing no longer changes within a set time (e.g., a unit of time), the password input value for the current password input position is locked to the current value. The password values ​​for other password input positions are then entered using the above password input method.

[0154] The authentication method for interconnecting smart glasses provided in this embodiment solves the connection permission problem when multiple smart glasses are interconnected, and can prevent other smart glasses that do not need to be interconnected or are malicious from accidentally entering the meeting, thereby ensuring the security of interconnecting multiple smart glasses.

[0155] Based on the authentication methods in the above embodiments, this disclosure also provides an authentication system for interconnected smart glasses, such as... Figure 8 As shown, the system includes: a main control module 1, configured to receive interconnection requests sent by smart glasses and control the smart glasses after sending the interconnection request to enter the interconnection platform; a processing module 2, configured to determine whether the smart glasses entering the interconnection platform are in the same spatiotemporal location; and further configured to determine whether the smart glasses are in a closed space; the main control module 1 is also configured to, when it is determined that the smart glasses entering the interconnection platform are not in the same spatiotemporal location, control the pairing connection between different smart glasses through password input; when it is determined that the smart glasses are in a closed space, control the direct pairing connection between different smart glasses; when it is determined that the smart glasses are not in a closed space, control the pairing connection between different smart glasses through password input; the main control module 1 is also configured to control the successfully paired smart glasses to enter the same virtual meeting scene.

[0156] Secondly, this disclosure also provides a readable storage medium having a program stored thereon, which, when executed by a processor, implements the authentication method described in the above embodiments.

[0157] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.

Claims

1. An authentication method for interconnecting smart glasses, characterized in that, include: Establish an interconnected platform; After the smart glasses send an interconnection request, they enter the interconnection platform. Determine whether the smart glasses entering the interconnected platform are in the same spatiotemporal location; If yes, determine whether the smart glasses are in a closed space; if no, pair and connect different smart glasses by entering a password. When it is determined that the smart glasses are in the enclosed space, pairing and connection between different smart glasses are directly performed. When it is determined that the smart glasses are not in the enclosed space, pairing and connection between different smart glasses are performed by entering a password. Once the smart glasses are successfully paired and connected, they enter the same virtual meeting scene.

2. The authentication method according to claim 1, characterized in that, The determination of whether the smart glasses entering the interconnected platform are in the same spatiotemporal location includes: The network time protocol is used to control the timestamp error of different smart glasses to be less than or equal to a first threshold; different smart glasses are set to have the same video resolution and frame rate, and video frame images are captured synchronously; different smart glasses are set to have the same audio sampling rate, and different audio information is recorded synchronously; the ambient light brightness information around the smart glasses is tested, and the ambient light brightness information is uploaded to the data processing platform; The smart glasses are used to determine whether they are candidate terminals in the same space by matching visual feature points. The smart glasses are determined to be candidate terminals in the same space by matching audio signals. The system determines whether the smart glasses are candidate terminals in the same space by matching ambient light brightness information. The multimodal fusion is used to determine whether the candidate terminals are in the same space.

3. The authentication method according to claim 2, characterized in that, The step of determining whether the smart glasses are candidate terminals in the same space by matching visual feature points includes: Feature points are extracted from the images captured by the smart glasses, and the number of extracted feature points is set to a second threshold. The feature points are matched and verified to select valid matching points that conform to spatial perspective relationships; Determine whether the number of valid matching points is greater than or equal to a set percentage of the second threshold; If yes, then the smart glasses are the candidate terminals in the same space; if no, then the smart glasses cannot be the candidate terminals.

4. The authentication method according to claim 2, characterized in that, The step of determining whether the smart glasses are candidate terminals in the same space by matching audio signals includes: Extract the Mel-frequency cepstral coefficients and short-time energy at the moment of sound source location from the audio signals of the different smart glasses; Calculate the cosine similarity of the Mel frequency cepstral coefficients of the audio signals of different smart glasses. If the cosine similarity is not lower than a third threshold, then the audio signals of different smart glasses have the same characteristics. The starting time of the sound source is located by the peak value of the short-time energy, and the time difference of the audio signals of different smart glasses is calculated. If the time difference is less than the fourth threshold, the smart glasses are candidate terminals in the same space.

5. The authentication method according to claim 2, characterized in that, The step of determining whether the smart glasses are candidate terminals in the same space by matching ambient light brightness information includes: Define an initial time point, detect the ambient light brightness information of different smart glasses in real time, and extract the ambient light brightness value according to the set time period; Starting from the initial time point, the brightness value of the ambient light of the smart glasses is subjected to curve fitting; Determine whether the similarity of the fitting curves of different smart glasses is above the fifth threshold; if so, the smart glasses are candidate terminals in the same space.

6. The authentication method according to any one of claims 2-5, characterized in that, The step of determining whether the candidate terminals are in the same space through multimodal fusion includes: Weights are assigned to the matching of visual feature points, audio signals, and ambient light brightness information of the candidate terminals, respectively, and the weighted scores of the candidate terminals are calculated. If the weighted score of the candidate terminal is greater than the sixth threshold, then the candidate terminals are determined to be in the same space. If the weighted score of the candidate terminal is less than the seventh threshold, it is determined that the candidate terminals are not in the same space; the seventh threshold is less than the sixth threshold. If the weighted score of the candidate terminal is greater than or equal to the seventh threshold and less than or equal to the sixth threshold, then 3D reconstruction is performed based on the image data captured by the candidate terminal. If the alignment of the point cloud after 3D reconstruction is greater than the eighth threshold, then the candidate terminals are determined to be in the same space.

7. The authentication method according to claim 1, characterized in that, The password input methods include hand password input, head password input, voice password input, video image password input, and absolute display space password input.

8. The authentication method according to claim 7, characterized in that, The methods for inputting the absolute display space password include: Determine the position coordinates of the display interface of the smart glasses in the first direction in the initial state; set the maximum and minimum values ​​of the distance change between the smart glasses and the display interface in the first direction; When inputting a password, the position of the display interface of the smart glasses in the first direction remains unchanged, and the distance between the smart glasses and the display interface along the first direction is adjusted to vary between the maximum value and the minimum value, so as to input the corresponding password value to different password input positions; When the distance between the smart glasses and the display interface along the first direction decreases, the password value input at the corresponding password input position increases; when the distance between the smart glasses and the display interface along the first direction increases, the password value input at the corresponding password input position decreases. If no change in distance along the first direction is detected between the smart glasses and the display interface within a set time, then the password value input at the current password input position is determined to be the password value corresponding to the distance between the smart glasses and the display interface along the first direction.

9. The authentication method according to claim 8, characterized in that, The method for inputting the absolute display space password also includes: Determine the position coordinates of the display interface of the smart glasses in the second direction in the initial state; set an adjustment gradient threshold for the distance between the smart glasses and the display interface in the second direction; When entering a password, the position of the display interface of the smart glasses in the second direction remains unchanged, the change value of the distance between the smart glasses and the display interface along the second direction is detected, and the change value of the distance is compared with the gradient threshold. When the absolute value of the distance change is greater than the gradient threshold, the password input bit is moved along the second direction; When the distance change value is negative, the password input position is moved to the left along the second direction; when the distance change value is positive, the password input position is moved to the right along the second direction.

10. The authentication method according to claim 9, characterized in that, The method for inputting the absolute display space password also includes: Let the total number of password input bits be M. When nk is satisfied T ≤ ≤(n+1)k T If so, then move n password input bits to the left or right along the second direction; in, x is the distance change value, k T The gradient threshold; When the number of password input positions moved is greater than the difference between M and the current position of the password input position, the movement stops when the last password input position is reached, or the movement continues to cycle to the first password input position when the last password input position is reached.

11. An authentication system for interconnected smart glasses, characterized in that, include: The main control module is configured to receive the interconnection request sent by the smart glasses and control the smart glasses to enter the interconnection platform after sending the interconnection request; The processing module is configured to determine whether the smart glasses entering the interconnection platform are in the same spatiotemporal location; it is also configured to determine whether the smart glasses are in a closed space. The main control module is also configured to, when it is determined that the smart glasses entering the interconnection platform are not in the same spatiotemporal location, control the pairing connection between different smart glasses through password input; when it is determined that the smart glasses are in the enclosed space, control the direct pairing connection between different smart glasses; and when it is determined that the smart glasses are not in the enclosed space, control the pairing connection between different smart glasses through password input. The main control module is also configured to control the successfully paired smart glasses to enter the same virtual meeting scene.

12. A readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the authentication method as described in any one of claims 1-10.

Images