Display control method and system for smart glasses

Abstract

The present application relates to a display control method and system for smart glasses, the method comprising: the smart glasses detect the duration of a user watching an electronic product; when the smart glasses determine that the duration is greater than a time threshold, the first image of the electronic product is collected, and when the first image is identified and determined not to be a timely processed document, the user is prompted to perform an eye relief operation; after the user confirms, the smart glasses perform an eye relief operation, the operation specifically comprising: the smart glasses control the middle part of the lens to display a smaller object for a first set time, and then control the middle part to display a larger object for a second set time. The technical solution provided by the present application has the advantage of high user experience.

Description

Technical Field

[0001] The present application relates to the field of communications and electronic technology, and in particular to a display control method and system for smart glasses. Background Art

[0002] Smart glasses, also known as smart mirrors, are like smartphones, with independent operating systems. Users can install software, games and other programs provided by software service providers. Smart glasses can be controlled by voice or motion to complete functions such as adding schedules, map navigation, interacting with friends, taking photos and videos, and making video calls with friends. They can also access wireless networks through mobile communication networks.

[0003] Like smartphones, users of smart glasses may experience visual fatigue when using them. Existing smart glasses are unable to relieve users' visual fatigue through display, which reduces the user experience. Summary of the invention

[0004] The embodiments of the present application provide a display control method and system for smart glasses, which can relieve visual fatigue and improve user experience through display control.

[0005] In a first aspect, an embodiment of the present application provides a display control method of smart glasses, the method comprising the following steps:

[0006] Smart glasses detect how long users are looking at electronic products;

[0007] When the smart glasses determine that the duration is greater than the time threshold, the smart glasses collect a first image of the electronic product, and when the smart glasses identify the first image and determine that the document does not belong to timely processing, prompt the user to perform an eye relief operation;

[0008] After the user confirms, the smart glasses perform an eye relief operation, which specifically includes: the smart glasses control the middle part of the lens to display a smaller object for a first set time, and then control the middle part to display a larger object for a second set time.

[0009] In a second aspect, a display control system of smart glasses is provided, the system comprising:

[0010] A detection unit, used to detect the duration of a user viewing the electronic product;

[0011] The processing unit is used to control the camera to capture the first image of the electronic product when it is determined that the duration is greater than the time threshold, and when the first image is identified and determined not to be a document to be processed in a timely manner, prompt the user to perform an eye relief operation; after the user confirms, the eye relief operation is performed, and the operation specifically includes: the smart glasses control the middle part of the lens to display a smaller object for a first set time, and then control the middle part to display a larger object for a second set time.

[0012] In a third aspect, an embodiment of the present application provides a server, comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps in the first aspect of the embodiment of the present application.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiment of the present application.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiment of the present application. The computer program product may be a software installation package.

[0015] The implementation of the embodiments of the present application has the following beneficial effects:

[0016] It can be seen that the technical solution of the present application is that the smart glasses detect the duration of the user watching the electronic product; when the smart glasses determine that the duration is greater than the time threshold, the first image of the electronic product is collected, and when the first image is identified and determined not to be a timely processed document, the user is prompted to perform an eye relief operation; after the user confirms, the smart glasses perform an eye relief operation, which may specifically include: the smart glasses control the middle part of the lens to display a smaller object (such as text or scenery) for a first set time, and then control the middle part to display a larger object for a second set time. In this way, in the first set time, a smaller object can be used to simulate a distant scenery, and then a larger object can be used to simulate a nearby object. In this way, the change of distant and near objects causes the change of the curvature of the lens, thereby achieving the purpose of adjusting the user's visual fatigue, thereby improving the user's experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a schematic diagram of the structure of a pair of smart glasses provided in an embodiment of the present application;

[0019] Figure 2 is a flow chart of a display control method of smart glasses provided in an embodiment of the present application;

[0020] Figure 3 It is a structural diagram of a display control system of smart glasses provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0023] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0024] See also Figure 1 , Figure 1 A schematic diagram of the structure of smart glasses is provided. Figure 1As shown, the smart glasses may include: a frame and lenses, wherein the smart glasses integrate a processor, a memory, an audio component (such as an earphone), a camera, a communication component (such as a Bluetooth module, wifi, a mobile communication module) and a display screen in the frame, wherein the display screen is integrated with the lenses, and the above-mentioned processor, memory, audio component (such as an earphone), camera and display screen can be connected through a bus, and of course can also be connected through other methods. This application does not limit the specific display method of the above-mentioned components.

[0025] Visual fatigue is a syndrome caused by prolonged improper use of the eyes (such as close-range visual observation with high tension, staring at flashing targets, targets with brightness that is too high / too low, excessive use of the eyes, etc.), which results in eye symptoms such as blurred vision, eye swelling, dryness, tearing, and eye socket pain. In severe cases, it can develop into systemic discomfort such as headaches, dizziness, and fatigue.

[0026] Long-term use of electronic products will cause visual fatigue, so it is necessary to relieve visual fatigue. One way to relieve visual fatigue is to relieve the lens of the glasses, that is, to relieve visual fatigue by enlarging or shrinking the lens of the glasses, thereby increasing the time users use electronic products. The eyes adjust the curvature of the lens through the ciliary body (containing smooth muscle) to see objects at different distances. When looking at distant objects, the ciliary body relaxes and the convexity of the lens decreases: when looking at nearby objects, the ciliary body contracts and the convexity of the lens increases. The change of the lens can relieve visual fatigue. For example, when users look at electronic products for too long, they will go outside the window to look at the distant scenery, such as mountains, forests, etc., to relieve eye fatigue. This is based on this principle, that is, to achieve changes in the curvature of the lens by looking at objects near and far with the eyes, thereby achieving relief of eye fatigue. Based on this principle, the present application adjusts eye fatigue by controlling the reality of smart glasses, thereby improving the user's experience of using smart glasses.

[0027] See also Figure 2 , Figure 2 A display control method for smart glasses is provided, which can be performed as follows: Figure 1 The smart glasses shown are implemented as Figure 2 As shown, the display control method of the above-mentioned smart glasses is as follows Figure 2 As shown, the following steps are included:

[0028] Step S201, the smart glasses detect the duration of the user watching the electronic product;

[0029] Step S202: When the smart glasses determine that the duration is greater than the time threshold, they collect a first image of the electronic product, and when they identify the first image and determine that it is not a timely processed document, they prompt the user to perform an eye relief operation;

[0030] Step S203: After the user confirms, the smart glasses perform an eye relief operation, which may specifically include: the smart glasses control the middle of the lens to display a smaller object (such as text or scenery) for a first set time, and then control the middle to display a larger object for a second set time.

[0031] The above can periodically display smaller objects and larger objects.

[0032] The technical solution of the present application is that the smart glasses detect the duration of the user watching the electronic product; when the smart glasses determine that the duration is greater than the time threshold, the first image of the electronic product is collected, and when the first image is identified and determined not to be a timely processed document, the user is prompted to perform an eye relief operation; after the user confirms, the smart glasses perform an eye relief operation, which may specifically include: the smart glasses control the middle part of the lens to display a smaller object (such as text or scenery) for a first set time, and then control the middle part to display a larger object for a second set time. In this way, in the first set time, a smaller object can be used to simulate a distant scenery, and then a larger object can be used to simulate a nearby object. In this way, the change of distant and near objects causes the change of the curvature of the lens, thereby achieving the purpose of adjusting the user's visual fatigue, thereby improving the user's experience.

[0033] For example, the smart glasses may detect the duration of a user watching an electronic product, for example:

[0034] The smart glasses periodically collect n pictures, perform object recognition on the n pictures to determine whether there are electronic products. If there are m consecutive pictures with electronic products among the n pictures, the duration is determined to be (m-1)*t1; where t1 is the time value of one cycle of periodic collection.

[0035] The above-mentioned identification of whether an object is an electronic product can be performed by an AI chip. For example, the AI ​​chip can determine whether an electronic product is present by running a neural network model. The above-mentioned neural network model can also be other models, such as a deep model, etc.

[0036] For example, the smart glasses may detect the duration of a user watching an electronic product by:

[0037] The smart glasses establish a communication connection with the electronic product, the smart glasses periodically receive dynamic responses from the electronic product, and determine the total time of the received dynamic responses as the duration of the electronic product.

[0038] By way of example, the above-mentioned identifying the first image and determining that it does not belong to the timely processed document may specifically include:

[0039] The smart glasses perform text recognition on the first image to determine the first text contained therein, and input the first text into a classifier to identify and determine the type of the first text. If the type belongs to a timely processed document, the first image is determined to be a timely processed document; if the type does not belong to a timely processed document, the first image is determined to be a non-timely processed document.

[0040] For example, if the type of the first text is code, speech, etc., it is a timely processed document. If the type of the first text is novel, subtitle, etc., it is determined not to be a timely processed document.

[0041] By way of example, the above-mentioned identifying the first image and determining that it does not belong to the timely processed document may specifically include:

[0042] The smart glasses form a first input matrix based on the RGB values ​​of the pixels of the first image, obtain a second input matrix of the zeroth image before the first image (similar to the first input matrix), perform a difference operation on the first input matrix and the second input matrix to obtain a difference matrix, and count the number x of non-zero element values ​​in the difference matrix. If the number x is greater than a quantity threshold, it is determined that the document is to be processed in a timely manner; if the number x is less than the quantity threshold, it is determined that the document is not to be processed in a timely manner.

[0043] The principle of this method is that for timely processed documents, generally the documents being processed by the user, the rate of change of the images of the two frames will be relatively high, so the number of non-zero elements will be relatively large. Conversely, for non-timely processed documents, the user stays on a page for a long time, then the number of non-zero element values ​​of the same page will be relatively large, and then determine whether it is a non-timely processed document.

[0044] See also Figure 3 , Figure 3 A display control system for smart glasses is provided, the system comprising:

[0045] A detection unit, used to detect the duration of a user viewing the electronic product;

[0046] The processing unit is used to control the camera to capture the first image of the electronic product when it is determined that the duration is greater than the time threshold, and when the first image is identified and determined not to be a document to be processed in a timely manner, prompt the user to perform an eye relief operation; after the user confirms, the eye relief operation is performed, and the operation specifically includes: the smart glasses control the middle part of the lens to display a smaller object for a first set time, and then control the middle part to display a larger object for a second set time.

[0047] Optional,

[0048] The processing unit is specifically used to periodically collect n pictures, perform object recognition on the n pictures to determine whether there are electronic products, and if there are m consecutive pictures among the n pictures with electronic products, determine the duration to be (m-1)*t1; wherein t1 is the time value of one cycle of periodic collection.

[0049] Optional,

[0050] The processing unit is specifically used for the smart glasses to establish a communication connection with the electronic product. The smart glasses periodically receive the dynamic response of the electronic product and determine the total time of the received dynamic response as the duration of the electronic product.

[0051] Optional,

[0052] The processing unit is specifically used to perform text recognition on the first image to determine the first text contained therein, and input the first text into a classifier to identify and determine the type of the first text. If the type belongs to a timely processed document, the first image is determined to be a timely processed document; if the type does not belong to a timely processed document, the first image is determined to be a non-timely processed document.

[0053] like Figure 3 The processing unit shown can also be used to perform the following Figure 2 The example solutions or detailed solutions of the illustrated embodiments are not described in detail here.

[0054] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any method recorded in the above method embodiments.

[0055] The present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any method described in the above method embodiment. The computer program product may be a software installation package.

[0056] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0057] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0058] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0059] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0060] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A display control method for smart glasses, characterized in that: The method comprises the following steps: Smart glasses detect how long users are looking at electronic products; When the smart glasses determine that the duration is greater than the time threshold, the smart glasses collect a first image of the electronic product, and when the smart glasses identify the first image and determine that the document does not belong to timely processing, prompt the user to perform an eye relief operation; After the user confirms, the smart glasses perform an eye relief operation, which specifically includes: the smart glasses control the middle part of the lens to display a smaller object for a first set time, and then control the middle part to display a larger object for a second set time; The smart glasses detecting the duration of the user watching the electronic product specifically includes: Smart glasses periodically collect n pictures, perform object recognition on n pictures to determine whether there are electronic products. If there are m consecutive pictures with electronic products among n pictures, determine the duration ; Wherein, t1 is the time value of one cycle of periodic collection; The smart glasses detecting the duration of the user watching the electronic product specifically includes: The smart glasses establish a communication connection with the electronic product, the smart glasses periodically receive dynamic responses from the electronic product, and determine the total time of the received dynamic responses as the duration of the electronic product.

2. The method according to claim 1, characterized in that The identifying the first image to determine that the document does not belong to timely processing specifically includes: The smart glasses perform text recognition on the first image to determine the first text contained therein, and input the first text into a classifier to identify and determine the type of the first text. If the type belongs to a timely processed document, the first image is determined to be a timely processed document; if the type does not belong to a timely processed document, the first image is determined to be a non-timely processed document.

3. A display control system for smart glasses, characterized in that: The system comprises: A detection unit, used to detect the duration of a user viewing the electronic product; The processing unit is configured to control the camera to capture a first image of the electronic product when it is determined that the duration is greater than a time threshold, and to prompt the user to perform an eye relief operation when the first image is identified and determined not to be a timely processed document; after the user confirms, the eye relief operation is performed, and the operation specifically includes: the smart glasses control the middle part of the lens to display a smaller object for a first set time, and then control the middle part to display a larger object for a second set time; The processing unit is specifically used to periodically collect n pictures, perform object recognition on the n pictures to determine whether there are electronic products, and if there are m consecutive pictures with electronic products among the n pictures, determine the duration ; Wherein, t1 is the time value of one cycle of periodic collection; The processing unit is specifically used for the smart glasses to establish a communication connection with the electronic product. The smart glasses periodically receive the dynamic response of the electronic product and determine the total time of the received dynamic response as the duration of the electronic product.

4. The system according to claim 3, characterized in that The processing unit is specifically used to perform text recognition on the first image to determine the first text contained therein, and input the first text into a classifier to identify and determine the type of the first text. If the type belongs to a timely processed document, the first image is determined to be a timely processed document; if the type does not belong to a timely processed document, the first image is determined to be a non-timely processed document.

5. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program causes a computer to execute the method as claimed in claim 1.

Images