Intelligent glasses, control method and system

Abstract

The invention provides intelligent glasses and a control method and system. The intelligent glasses at least comprise a glasses main body, an optical machine, a first camera, a second camera and a controller, the glasses main body consists of a left glasses leg, a right glasses leg and a glasses frame provided with display lenses. Acquiring a first image acquired by the first camera and a second image acquired by the second camera; calculating the distance between the target object and the intelligent glasses by using the first image and the second image to obtain a target image distance; according to the target image distance, the ray machine is controlled to project multimedia content related to the target object into the display lens, and the projection image distance of the multimedia content is equal to the target image distance. According to the scheme, the layout of the optical machine and the camera is optimized on the premise of ensuring the 3D perception precision, so that the intelligent glasses can accommodate more functional components, the functional diversity of the intelligent glasses is improved, and the user experience is improved by realizing adaptive projection of multimedia contents with different image distances.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent wearable devices, and in particular to an intelligent glasses, a control method and system. BACKGROUND

[0002] With the development of science and technology, augmented reality (AR) glasses gradually popularize and apply. The mainstream AR glasses currently adopt a scheme of "dual optical engine + multiple cameras", but this scheme leads to a large volume of the AR glasses on the one hand, and the lens legs cannot accommodate more functional components, the function is relatively single, on the other hand, the calibration of multiple cameras and dual optical engine is difficult, and parallax deviation is prone to occur, resulting in low 3D perception accuracy. SUMMARY

[0003] Therefore, the embodiments of the present application provide an intelligent glasses, a control method and system to solve the problems of single function and low 3D perception accuracy of the existing AR glasses scheme.

[0004] To achieve the above object, the embodiments of the present application provide the following technical scheme:

[0005] The first aspect of the embodiments of the present application discloses an intelligent glasses, which at least comprises: a glasses main body, an optical engine, a first camera, a second camera and a controller; the glasses main body is composed of a left lens leg, a right lens leg and a frame provided with a display lens;

[0006] The optical engine is arranged in the beam region of the frame, and the optical engine is connected with the display lens.

[0007] The first camera and the second camera are arranged at the front ends of the left lens leg and the right lens leg, respectively.

[0008] The controller is arranged in the interior of the left lens leg or the right lens leg, and the controller is connected with the optical engine, the first camera and the second camera, respectively.

[0009] Preferably, the optical axes of the first camera and the second camera are parallel, and the distance between the optical axes of the first camera and the second camera is adapted to the interpupillary distance of the human body.

[0010] Preferably, the optical engine is arranged in the middle part of the beam region of the frame.

[0011] Preferably, the optical engine is connected with the display lens through a light splitting component.

[0012] The second aspect of the embodiments of the present application discloses a control method, which is applicable to the intelligent glasses disclosed in the first aspect of the embodiments of the present application, and the method comprises:

[0013] Acquire a first image captured by a first camera and a second image captured by a second camera;

[0014] Using the first image and the second image, the distance between the target object and the smart glasses is calculated to obtain the target image distance. The target object is any object within the field of vision of the user wearing the smart glasses.

[0015] Based on the target image distance, the optical engine is controlled to project multimedia content related to the target object onto the display lens, and the projection image distance of the multimedia content is equal to the target image distance.

[0016] Preferably, based on the target image distance, controlling the optical engine to project multimedia content related to the target object onto the display lens includes:

[0017] Based on the target image distance, adjust the focal length of the lens of the optical engine and / or adjust the distance between the objective lens of the optical engine and the display lens so that the projected image distance of the optical engine is equal to the target image distance;

[0018] The optical engine is controlled to project multimedia content related to the target object onto the display lens according to the projection distance.

[0019] Preferably, before controlling the optical engine to project multimedia content related to the target object onto the display lens, the method further includes:

[0020] Based on the brightness of the target object, the optical engine is controlled to adjust the projection brightness of multimedia content related to the target object.

[0021] Preferably, before controlling the optical engine to project multimedia content related to the target object onto the display lens, the method further includes:

[0022] Based on the multimedia content, the optical engine is controlled to adjust the projection resolution of the multimedia content related to the target object.

[0023] Preferably, using the first image and the second image, calculating the distance between the target object and the smart glasses to obtain the target image distance includes:

[0024] Feature matching is performed on the first image and the second image to obtain the three-dimensional coordinates and depth information of the target object;

[0025] Using the three-dimensional coordinates and depth information of the target object, the distance between the target object and the smart glasses is calculated to obtain the target image distance.

[0026] A third aspect of this invention discloses a control system applicable to the smart glasses disclosed in the first aspect of this invention, the system comprising:

[0027] The acquisition unit is used to acquire a first image captured by a first camera and a second image captured by a second camera;

[0028] The calculation unit is used to calculate the distance between the target object and the smart glasses using the first image and the second image to obtain the target image distance, wherein the target object is any object within the field of vision of the user wearing the smart glasses;

[0029] The projection unit is used to control the optical engine to project multimedia content related to the target object onto the display lens according to the target image distance, wherein the projection image distance of the multimedia content is equal to the target image distance.

[0030] Based on the above embodiments of the present invention, a smart glasses, control method, and system are provided. The smart glasses include at least: a glasses body, an optical engine, a first camera, a second camera, and a controller. The glasses body consists of a left temple, a right temple, and a frame with display lenses. A first image captured by the first camera and a second image captured by the second camera are acquired. Using the first and second images, the distance between the target object and the smart glasses is calculated to obtain the target image distance. Based on the target image distance, the optical engine is controlled to project multimedia content related to the target object onto the display lenses, where the projection image distance of the multimedia content is equal to the target image distance. This solution optimizes the layout of the optical engine and cameras while ensuring 3D perception accuracy, enabling the smart glasses to accommodate more functional components, improving the functional diversity of the smart glasses, and achieving adaptive projection of multimedia content with different image distances to improve the user experience. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0032] Figure 1 A structural block diagram of a smart glasses provided in an embodiment of the present invention;

[0033] Figure 2 A flowchart of a control method provided in an embodiment of the present invention;

[0034] Figure 3 Another flowchart of a control method provided in an embodiment of the present invention;

[0035] Figure 4 A structural block diagram of a control system provided in an embodiment of the present invention;

[0036] Figure 5 Another structural block diagram of a control system provided in an embodiment of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0039] With the development of technology, augmented reality (AR) glasses are gradually becoming more widely used. Currently, the mainstream AR glasses adopt a "dual optical engine + multiple camera" solution. Specifically, the "dual optical engine + multiple camera" solution uses a binocular display architecture with one optical engine for each eye, and multiple (more than 3) external or internal cameras to achieve 3D perception.

[0040] However, the "dual optical engine + multiple camera" solution leads to high equipment cost and large size (two optical engines require a lot of space on both sides of the frame), and the temples can only accommodate basic components such as batteries. The temples cannot accommodate more functional components, resulting in relatively simple functions. On the other hand, the calibration of multiple cameras and dual optical engines is more difficult, which can easily lead to parallax deviation, resulting in lower 3D perception accuracy and poor 3D display effect.

[0041] Furthermore, the inventors discovered through research that current AR glasses also employ solutions such as "single optical engine + single camera" and "external camera + AR glasses," but these solutions also have the following drawbacks:

[0042] The "single-optical-engine + single-camera" solution uses a single-optical-engine in conjunction with a beam splitter to achieve binocular display, and only one camera is equipped to collect environmental information. Although this reduces costs and size, the single camera can only acquire two-dimensional image information and cannot calculate depth through the principle of binocular vision. This results in the lack of 3D perception function, and can only achieve simple image overlay, which cannot meet the needs of scenarios that require three-dimensional environmental perception, such as navigation and virtual interaction.

[0043] For the "external camera + AR glasses" solution, the AR glasses are connected to an external binocular camera via a data cable to achieve 3D perception (the external binocular camera transmits the perception data to the AR glasses host). However, the external camera compromises the portability of the AR glasses, and the data cable transmission is prone to signal delay (the delay time is usually >50ms), which affects the real-time performance of the AR display. At the same time, the calibration and storage of external devices are more difficult.

[0044] In summary, whether it's the "dual optical engine + multiple cameras" solution, the "single optical engine + single camera" solution, or the "external camera + AR glasses" solution, none of them can simultaneously meet the requirements of high-precision 3D perception, small size, and high convenience.

[0045] To address this, this solution proposes a smart glasses, a control method, and a system. The smart glasses include at least: a main body, an optical engine, a first camera, a second camera, and a controller. The main body consists of a left temple, a right temple, and a frame with display lenses. The system acquires a first image captured by the first camera and a second image captured by the second camera. Using the first and second images, the distance between the target object and the smart glasses is calculated to obtain the target image distance. Based on the target image distance, the optical engine is controlled to project multimedia content related to the target object onto the display lenses, where the projection image distance of the multimedia content is equal to the target image distance.

[0046] This solution optimizes the layout of the optical engine and camera while ensuring 3D perception accuracy, thereby reducing the size of the smart glasses (high portability) or enabling them to accommodate more functional components (improving the functionality of the smart glasses), and achieving adaptive projection of multimedia content at different image distances to improve the user experience.

[0047] See Figure 1 The diagram shows a structural block diagram of a smart glasses provided by an embodiment of the present invention. The smart glasses include at least: a glasses body 101, an optical engine 102 (e.g., a single optical engine), a first camera 103, a second camera 104, and a controller 105. The glasses body 101 is composed of a left temple 1011, a right temple 1012, and a frame 1013 with display lenses.

[0048] Specifically, the optical engine 102 is located in the beam area of ​​the lens frame 1013, and is connected to the display lens on the lens frame 1013. The first camera 103 and the second camera 104 are respectively located at the front end of the left temple 1011 and the right temple 1012. The controller 105 is located inside the left temple 1011 or the right temple 1012, and is connected to the optical engine 102, the first camera 103, and the second camera 104.

[0049] In some embodiments, the frame 1013 is provided with at least one set of display lenses, which are used to receive multimedia content (such as images, text, etc.) projected by the optical engine 102.

[0050] For example, the frame 1013 is provided with two sets of display lenses, namely: the display lens on the left side of the frame 1013 (referred to as the left display lens) and the display lens on the right side of the frame 1013 (referred to as the right display lens).

[0051] In some embodiments, the optical engine 102 is disposed in the middle of the beam region of the lens frame 1013, and the optical engine 102 is connected to the display lens via a beam-splitting assembly. For example, the optical engine 102 connects the left display lens and the right display lens via a beam-splitting film.

[0052] It should be noted that a beam splitter specifically refers to an optical element with beam splitting characteristics. For example, a beam splitter can be an optical element such as a beam splitting plane mirror, a beam splitting prism, or a beam splitting thin film.

[0053] The output of the optical engine 102 is coupled into the display lens through two layers of polarizer holographic gratings, and coupled into the human eye through the output region, so as to project the multimedia content generated by the optical engine 102 onto the display lens.

[0054] For example, assuming that the frame 1013 is equipped with two sets of display lenses, the output end of the optical engine 102 is coupled into the two-layer polarizer holographic grating coupling area, and the light output from the optical engine 102 is split into two parts and transmitted to the left and right display lenses. It is then coupled to the human eye through the coupling area, so as to realize the synchronous projection of the multimedia content generated by the optical engine 102 onto the left and right display lenses, thereby realizing binocular display.

[0055] It should be noted that the polarizing holographic grating coupling region of this solution is equivalent to a high-efficiency polarization beam splitter, which can transmit light with different polarizations to the left and right display lenses respectively. Furthermore, the beam splitter is very thin (e.g., the thickness of the beam splitting film is 1~3µm), which is beneficial for the integration of smart glasses.

[0056] In some embodiments, the first camera 103 is built into the front end of the left temple (e.g., near the side of the frame 1013), and the second camera 104 is built into the front end of the right temple (e.g., near the side of the frame 1013). The optical axes of the first camera 103 and the second camera 300 are parallel, and the distance between the optical axes of the first camera 103 and the second camera 300 is adapted to the human interpupillary distance.

[0057] Specifically, the distance between the optical axes of the first camera 103 and the second camera 300 can be adjusted to match the human interpupillary distance, and the adjustable range can be 50mm-75mm (for example only). The image acquisition direction of the first camera 103 and the second camera 300 is consistent with the line of sight of the user wearing smart glasses.

[0058] In some embodiments, the controller 105 is integrated inside the left temple 1011 or the right temple 1012, and the controller 105 is connected to the optical engine 102, the first camera 103 and the second camera 104 respectively via FPC cables.

[0059] The controller 105 can receive image data (i.e., 3D perception data) collected by the first camera 103 and the second camera 104. The controller 105 calculates 3D perception information based on the image data. Based on the 3D perception information, the controller 105 sends an image distance adjustment command to the optical engine 102, so that the optical engine 102 adjusts relevant parameters according to the image distance adjustment command to project multimedia content onto the display lens. How the multimedia content is projected will be described in detail in subsequent embodiments.

[0060] above Figure 1 This document describes the hardware architecture of smart glasses. In practical applications, the smart glasses in this solution can be AR glasses. The smart glasses in this solution adopt a "single-optical-engine + dual-camera" approach.

[0061] In the above Figure 1 Based on the given hardware architecture of smart glasses, see [link / reference]. Figure 2 This invention provides a flowchart of a control method applicable to the smart glasses provided in the above embodiments. The control method includes:

[0062] Step S201: Acquire a first image captured by the first camera and a second image captured by the second camera.

[0063] In the specific implementation step S201, the controller acquires a first image captured by the first camera and a second image captured by the second camera.

[0064] Specifically, the first camera and the second camera simultaneously acquire two-dimensional images of the surrounding environment. The first camera acquires a first image, and the second camera acquires a second image. The first camera transmits the acquired first image to the controller, and the second camera transmits the acquired second image to the controller.

[0065] Step S202: Using the first image and the second image, calculate the distance between the target object and the smart glasses to obtain the target image distance. The target object is any object within the field of vision of the user wearing the smart glasses.

[0066] In the specific implementation step S202, the controller performs feature matching on the first image and the second image to obtain the three-dimensional coordinates and depth information of the target object, which is any object within the field of vision of the "user wearing smart glasses".

[0067] The controller uses the three-dimensional coordinates and depth information of the target object to calculate the distance between the target object and the smart glasses to obtain the target image distance.

[0068] Understandably, based on the numerical value of the target image distance, the target image distance can be divided into three types: close range (0.3m-1m), medium range (1m-5m), and long range (more than 5m).

[0069] It should be further explained that the target image distance is the distance between the "target object" and the "smart glasses", so the target image distance is also equivalent to the distance between the "target object" and the "eyes of the user wearing the smart glasses".

[0070] In practical applications, after a user wears smart glasses, the user will see one or more target objects. In order to improve the 3D display effect of smart glasses, it is desirable to project multimedia content (such as pictures, text information, etc.) related to the target object into the vicinity of the target object. This will make the display effect of multimedia content very harmonious. Therefore, it is necessary to calculate the target image distance using the first image and the second image, and then project the multimedia content according to the target image distance.

[0071] Based on this, one application scenario of this solution is: after a user wears smart glasses, whatever object the user sees (the object seen is the target object) will have its multimedia content projected and displayed in the vicinity of the target object in the smart glasses.

[0072] For example, when a user wears smart glasses and sees a flower on the roadside at close range, the flower is the target object. The distance between the flower and the smart glasses is calculated to obtain the target image distance. Based on the target image distance, the text description of the flower (i.e., multimedia content) is projected. At this time, the user can see the text description of the flower displayed near the flower in the smart glasses.

[0073] For example, when a user wears smart glasses and sees a distant airplane, the airplane becomes the target object. The distance between the airplane and the smart glasses is calculated to obtain the target image distance. Based on the target image distance, the airplane's graphic information (i.e., multimedia content) is projected. At this time, the user can see the airplane's graphic information displayed near the airplane in the smart glasses, and the graphic information moves as the airplane moves.

[0074] Step S203: Based on the target image distance, control the optical engine to project multimedia content related to the target object onto the display lens, where the projection image distance of the multimedia content is equal to the target image distance.

[0075] It should be noted that the projection image distance (also known as the projection distance) represents how far away the content projected by the optical engine is from the user.

[0076] In the specific implementation step S203, the focal length of the lens of the optical engine and / or the distance from the objective lens of the optical engine to the display lens are adjusted according to the target image distance so that the projection image distance of the optical engine is equal to the target image distance. The optical engine is controlled to project multimedia content related to the target object onto the display lens according to the projection distance (equal to the target image distance), so that the user can see the multimedia content projected and displayed near the target object in the smart glasses.

[0077] It is understandable that optical-mechanical projection involves using lenses to image the contents of the display lens onto a certain distance. The relationship between the "focal length of the optical-mechanical lens," the "projection image distance," and the "distance from the optical-mechanical objective lens to the display lens" is as follows: 1 / L2 + 1 / L1 = 1 / f. Where L1 is the distance from the display lens to the optical-mechanical objective lens, L2 is the projection image distance, and f is the focal length of the optical-mechanical lens.

[0078] When it is desired that L2 equals the target image distance, this can be achieved by adjusting f and L1. Specifically, there are three adjustment methods:

[0079] The first adjustment method involves keeping L1 unchanged and changing f to make L2 equal to the target image distance.

[0080] The second adjustment method involves keeping f constant and changing L1 to make L2 equal to the target image distance.

[0081] The third adjustment method involves jointly adjusting f and L1 to make L2 equal to the target image distance.

[0082] In practical applications, an image distance adjustment command is generated based on the target image distance. This image distance adjustment command carries at least the "focal length f of the lens of the optical engine" and / or the "distance L1 from the objective lens of the optical engine to the display lens". The "focal length f of the lens of the optical engine" and / or the "distance L1 from the objective lens of the optical engine to the display lens" carried in the image distance adjustment command are used to adjust the projected image distance to the target image distance.

[0083] The image distance adjustment command is sent to the optical engine, which then performs the following actions: according to the content carried by the image distance adjustment command, the parameters of the internal microdisplay chip and optical lens are adjusted to generate multimedia content with a corresponding projection image distance (equal to the target image distance). The multimedia content is then projected onto the display lens through a beam splitting component (e.g., a beam splitting film), thereby achieving an AR display that matches the target image distance.

[0084] It should be noted that, for image distance adjustment commands that at least include "focal length f of the optical engine's lens" and / or "distance L1 between the optical engine's objective lens and the display lens", if the first adjustment method is used, then the image distance adjustment command must at least include "focal length f of the optical engine's lens"; if the second adjustment method is used, then the image distance adjustment command must at least include "distance L1 between the optical engine's objective lens and the display lens"; if the third adjustment method is used, then the image distance adjustment command must at least include "focal length f of the optical engine's lens" and "distance L1 between the optical engine's objective lens and the display lens".

[0085] In some embodiments, in order to further make the multimedia content and the target object appear visually matched, the projection brightness of the multimedia can be adjusted to match the brightness of the target object. That is, according to the brightness of the target object, the optical engine is controlled to adjust the projection brightness of the multimedia content related to the target object. At this time, in addition to carrying the "focal length f of the lens of the optical engine" and / or the "distance L1 from the objective lens of the optical engine to the display lens", the image distance adjustment command also carries the projection brightness.

[0086] Specifically, the brightness of the target object (or the brightness around the target object) is detected by the first and second cameras. Based on the brightness of the target object, an appropriate brightness is calculated as the projection brightness for the multimedia content. An image distance adjustment command carrying the projection brightness is sent to the optical engine, which then adjusts the projection power according to the projection brightness, so that the projected multimedia content and the target object appear visually matched.

[0087] In some embodiments, if the multimedia content contains a lot of information (e.g., a lot of words), it is necessary to increase the projection resolution of the multimedia content to improve the display clarity of the multimedia content. That is, according to the multimedia content, the optical engine is controlled to adjust the projection resolution of the multimedia content related to the target object. At this time, in addition to carrying the "focal length f of the lens of the optical engine" and / or the "distance L1 from the objective lens of the optical engine to the display lens", the image distance adjustment command also carries the projection resolution.

[0088] Specifically, based on the information density of the multimedia content, a suitable resolution is calculated to serve as the projection resolution for the multimedia content. An image distance adjustment command carrying the projection resolution is then sent to the optical engine, which adjusts the display sharpness of the multimedia content according to this projection resolution.

[0089] As can be seen from the above embodiments, in addition to carrying the "focal length f of the optical engine lens" and / or the "distance L1 from the objective lens of the optical engine to the display lens", the image distance adjustment command can also carry the projection brightness and / or projection resolution. That is, in addition to projecting multimedia content near the target object, the projection brightness and / or projection resolution of the multimedia content can also be adjusted.

[0090] To better understand how multimedia content is projected onto a display lens, the following example illustrates the process using an application scenario. (See attached image.) Figure 3 The diagram illustrates another flowchart of a control method provided by an embodiment of the present invention. In this application scenario embodiment, the distance L1 from the objective lens of the optical engine to the display lens remains unchanged, while the focal length f of the lens of the optical engine is changed to make the projected image distance equal to the target image distance. Figure 3 Includes the following steps:

[0091] Step S301: The controller acquires a first image captured by the first camera and a second image captured by the second camera.

[0092] Step S302: The controller performs feature matching on the first image and the second image to obtain the three-dimensional coordinates and depth information of the target object, and calculates the target image distance using the three-dimensional coordinates and depth information of the target object.

[0093] Step S303: The controller generates an image distance adjustment command, which carries the focal length, projection brightness and projection resolution.

[0094] Step S304: The controller sends the image distance adjustment command to the optical engine.

[0095] Step S305: The optical engine adjusts the parameters of the internal micro-display chip and optical lens according to the content carried by the image distance adjustment command, generates multimedia content with the corresponding projection image distance, and projects the multimedia content onto the display lens through the beam splitter.

[0096] It should be noted that the execution principle of steps S301 to S305 can be found in the above embodiments of the present invention. Figure 2 The details will not be repeated here.

[0097] As can be seen from the above embodiments, this solution has the following advantages:

[0098] High 3D perception accuracy: The dual cameras are integrated with the temples, and the distance between the optical axes of the dual cameras is adapted to the human interpupillary distance, which conforms to the principle of binocular vision and results in a smaller error in depth calculation.

[0099] Size and cost optimization: The single-optical-engine design, with the optical-engine centrally located, frees up space in the temples, allowing the temples to accommodate dual cameras and a larger capacity battery, resulting in an overall device weight of less than 80g.

[0100] Improved portability: Dual cameras are built into the temples, eliminating the need for external cameras. The smart glasses are highly integrated, making them easy to store and use.

[0101] Projection image distance adaptive: Through the linkage of 3D perception data and optical engine, it can switch between near, medium and far distance multimedia content projection in real time, adapting to scenarios such as reading, navigation, and virtual interaction.

[0102] Corresponding to the control method provided in the above embodiments of the present invention, see also... Figure 4 The present invention also provides a structural block diagram of a control system, which is applicable to the smart glasses provided in the above embodiments. The system includes: an acquisition unit 401, a calculation unit 402, and a projection unit 403.

[0103] The acquisition unit 401 is used to acquire a first image captured by the first camera and a second image captured by the second camera.

[0104] The calculation unit 402 is used to calculate the distance between the target object and the smart glasses using the first image and the second image to obtain the target image distance. The target object is any object within the field of vision of the user wearing the smart glasses.

[0105] The projection unit 403 is used to control the optical engine to project multimedia content related to the target object onto the display lens according to the target image distance, wherein the projection image distance of the multimedia content is equal to the target image distance.

[0106] In some embodiments, the projection unit 403 is specifically used to: adjust the focal length of the lens of the optical engine and / or adjust the distance between the objective lens of the optical engine and the display lens according to the target image distance, so that the projection image distance of the optical engine is equal to the target image distance; and control the optical engine to project multimedia content related to the target object onto the display lens according to the projection distance.

[0107] Preferably, the projection unit 403 is also used to: control the optical engine to adjust the projection brightness of multimedia content related to the target object according to the brightness of the target object.

[0108] Preferably, the projection unit 403 is also used to: control the optical engine to adjust the projection resolution of the multimedia content related to the target object according to the multimedia content.

[0109] Preferred, combined Figure 4 See Figure 5 This diagram illustrates another structural block diagram of a control system provided by an embodiment of the present invention. The computing unit 402 includes:

[0110] The matching subunit 4021 is used to perform feature matching on the first image and the second image to obtain the three-dimensional coordinates and depth information of the target object.

[0111] The calculation subunit 4022 is used to calculate the distance between the target object and the smart glasses using the three-dimensional coordinates and depth information of the target object to obtain the target image distance.

[0112] In summary, the embodiments of the present invention provide a smart glasses, a control method, and a system that optimize the layout of the optical engine and camera while ensuring 3D perception accuracy. This allows the smart glasses to accommodate more functional components, improve their functional diversity, and enable adaptive projection of multimedia content at different image distances to enhance the user experience.

[0113] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0114] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0115] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A smart glass, characterized by, The intelligent glasses at least include a glasses main body, a light machine, a first camera, a second camera and a controller; the glasses main body is composed of a left temple, a right temple and a frame provided with a display lens; The light machine is arranged in the crossbeam region of the frame, and the light machine is connected with the display lens; The first camera and the second camera are respectively arranged at the front ends of the left temple and the right temple; The controller is arranged in the interior of the left temple or the right temple, and the controller is connected with the light machine, the first camera and the second camera respectively.

2. The smart glasses of claim 1, wherein, The optical axes of the first camera and the second camera are parallel, and the distance between the optical axes of the first camera and the second camera is adapted to the interpupillary distance of a human body.

3. The smart glasses of claim 1, wherein, The light machine is arranged in the middle part of the crossbeam region of the frame.

4. The smart glasses of any one of claims 1-3, wherein, The light machine is connected with the display lens through a light splitting component.

5. A control method characterized by, The method is suitable for the intelligent glasses of any one of claims 1-4, and the method comprises: acquiring a first image collected by a first camera and a second image collected by a second camera; calculating the distance between a target object and the intelligent glasses by using the first image and the second image to obtain a target image distance, the target object being any object in the field of view of a user wearing the intelligent glasses; controlling a light machine to project multimedia content related to the target object into a display lens according to the target image distance, the projection image distance of the multimedia content being equal to the target image distance.

6. The method of claim 5, wherein, controlling a light machine to project multimedia content related to the target object into a display lens according to the target image distance, comprising: adjusting the focal length of a lens of the light machine and / or adjusting the distance between an objective lens of the light machine and the display lens according to the target image distance, so that the projection image distance of the light machine is equal to the target image distance; controlling the light machine to project the multimedia content related to the target object into the display lens according to the projection distance.

7. The method of claim 5, wherein, Before controlling the light machine to project the multimedia content related to the target object into the display lens, further comprising: controlling the light machine to adjust the projection brightness of the multimedia content related to the target object according to the brightness of the target object.

8. The method of claim 5, wherein, Before controlling the light machine to project the multimedia content related to the target object into the display lens, further comprising: controlling the light machine to adjust the projection resolution of the multimedia content related to the target object according to the multimedia content.

9. The method of claim 5, wherein, calculating the distance between a target object and the intelligent glasses by using the first image and the second image to obtain a target image distance, comprising: performing feature matching on the first image and the second image to obtain the three-dimensional coordinates and depth information of the target object; calculating the distance between the target object and the intelligent glasses by using the three-dimensional coordinates and depth information of the target object to obtain a target image distance.

10. A control system characterized by, The system is suitable for the intelligent glasses of any one of claims 1-4, and the system comprises: an acquisition unit, configured to acquire a first image collected by a first camera and a second image collected by a second camera; a computing unit configured to calculate a distance between an object and the smart glasses using the first image and the second image, the object being any object within a field of view of a user wearing the smart glasses; a projecting unit configured to control a light engine to project multimedia content related to the object into a display lens according to the distance, a projection image distance of the multimedia content being equal to the distance.

Images