System and method for adaptively controlling intelligent glasses in combination with Internet of Things

Abstract

The invention, which relates to the technical field of intelligent control, discloses a system and method for adaptive control of intelligent glasses in combination with the Internet of Things, comprising: monitoring the light intensity of the current environment, and obtaining the light change rate; when the light change rate is greater than a preset threshold value, triggering an auxiliary display mode, and obtaining a fused display image; displaying the fused display image as the display content of an auxiliary display mode, and performing adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set; and performing environment fitness analysis according to the real-time control parameter set, and if the preset environment fitness is met, releasing the auxiliary display mode, and switching to the self-adaptively controlled lens display mode. The technical problems that in the prior art, when light changes drastically, the lens is easily adjusted excessively, instant visual shielding is caused, and the visual experience and the visual stability of a user are affected are solved, and the technical effect of ensuring smooth visual transition and the visual stability of the user under the light change is achieved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to a system and method for adaptively controlling intelligent glasses in combination with the Internet of Things. Background Art

[0002] In the use scenarios of smart glasses, the automatic adjustment function of the lens is designed to adjust parameters such as brightness, focal length, and filter mode in real time according to changes in external light to enhance the user's visual experience. However, when the light changes drastically, such as going from a dim indoor environment to a bright outdoor environment or vice versa, the automatic adjustment mechanism of the lens may produce instantaneous over-adjustment. Although this adjustment speed can respond quickly to light changes, due to the drastic adjustment process, it often causes the lens parameters to fluctuate too much in a short period of time, resulting in instantaneous occlusion or blurring of visual information. Summary of the invention

[0003] The present application provides a system and method for adaptively controlling smart glasses in combination with the Internet of Things, which is used to solve the technical problem that the prior art easily causes excessive adjustment of lenses when light changes drastically, resulting in instantaneous visual occlusion, affecting the user's visual experience and visual stability.

[0004] In view of the above problems, the present application provides a system and method for adaptively controlling smart glasses in combination with the Internet of Things.

[0005] In a first aspect of the present application, a system for adaptively controlling smart glasses in combination with the Internet of Things is provided, the system comprising: a light intensity monitoring unit, the light intensity monitoring unit is used to monitor the light intensity of the current environment in real time through a light sensor to obtain the light change rate, wherein the light sensor is integrated on the smart glasses; an auxiliary display unit, the auxiliary display unit is used to trigger the auxiliary display mode of the smart glasses when the light change rate is greater than a preset threshold, and obtain a fused display image, wherein the fused display image is a fused display image of an infrared auxiliary mode and a frame display mode; an adaptive control unit, the adaptive control unit is used to display the fused display image as the display content of the auxiliary display mode after triggering the auxiliary display mode, and at the same time, the lens control module of the smart glasses performs adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set, wherein the control parameter set includes the brightness, focal length and filtering mode of the lens; a fitness analysis unit, the fitness analysis unit is used to perform environmental fitness analysis according to the real-time control parameter set, and if the preset environmental fitness is met, the auxiliary display mode is released, and the lens display mode after adaptive control is switched.

[0006] A second aspect of the present application provides a method for adaptively controlling smart glasses in combination with the Internet of Things, the method comprising: The light intensity of the current environment is monitored in real time by a light sensor to obtain the light change rate, wherein the light sensor is integrated on the smart glasses; when the light change rate is greater than a preset threshold, the auxiliary display mode of the smart glasses is triggered to obtain a fused display image, wherein the fused display image is a fused display image of an infrared auxiliary mode and a frame display mode; after the auxiliary display mode is triggered, the fused display image is displayed as the display content of the auxiliary display mode, and at the same time, the lens control module of the smart glasses performs adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set, wherein the control parameter set includes the brightness, focal length and filtering mode of the lens; an environmental adaptability analysis is performed according to the real-time control parameter set, and if the preset environmental adaptability is met, the auxiliary display mode is released, and the lens display mode after adaptive control is switched.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: The present application monitors the light intensity of the current environment in real time through a light sensor to obtain the light change rate, wherein the light sensor is integrated on the smart glasses; when the light change rate is greater than a preset threshold, the auxiliary display mode of the smart glasses is triggered to obtain a fused display image, wherein the fused display image is a fused display image of an infrared auxiliary mode and a frame display mode; after the auxiliary display mode is triggered, the fused display image is displayed as the display content of the auxiliary display mode, and at the same time, the lens control module of the smart glasses performs adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set, wherein the control parameter set includes the brightness, focal length and filtering mode of the lens; an environmental adaptability analysis is performed according to the real-time control parameter set, and if the preset environmental adaptability is met, the auxiliary display mode is released, and the lens display mode after adaptive control is switched. The present invention solves the technical problem in the prior art that when the light changes dramatically, it is easy to cause excessive adjustment of lenses, resulting in instantaneous visual occlusion, affecting the user's visual experience and visual stability. The present invention uses a built-in light sensor in the smart glasses to monitor light changes in real time, trigger an auxiliary display mode, fuse infrared and visible light images, adaptively adjust brightness, focal length and filter mode, generate a real-time control parameter set, and ensure visual stability through environmental adaptability analysis. When the adaptability is met, it switches to the adaptive display mode, thereby achieving the technical effect of ensuring a smooth visual transition and visual stability of the user under light changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0009] Figure 1 A schematic diagram of the system structure for adaptive control of smart glasses in combination with the Internet of Things provided in an embodiment of the present application; Figure 2 A schematic flow chart of a method for adaptively controlling smart glasses in combination with the Internet of Things provided in an embodiment of the present application.

[0010] Description of the reference numerals: light intensity monitoring unit 11 , auxiliary display unit 12 , adaptive control unit 13 , fitness analysis unit 14 . DETAILED DESCRIPTION

[0011] The present application provides a system and method for adaptively controlling smart glasses in combination with the Internet of Things, aiming to solve the technical problem that the prior art easily causes excessive adjustment of lenses when light changes drastically, resulting in instantaneous visual occlusion, affecting the user's visual experience and visual stability. The system and method use the built-in light sensor of the smart glasses to monitor light changes in real time, trigger an auxiliary display mode, fuse infrared and visible light images, adaptively adjust brightness, focal length and filter mode, generate a real-time control parameter set, and ensure visual stability through environmental adaptability analysis. When the adaptability is met, the system switches to the adaptive display mode, thereby achieving the technical effect of ensuring a smooth visual transition and visual stability for users under light changes.

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0013] It should be noted that any variations of the terms "include" and "have" are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or modules that are not explicitly listed or inherent to these processes, methods, products or devices.

[0014] Embodiment 1, as Figure 1 As shown, the embodiment of the present application provides a system for adaptively controlling smart glasses in combination with the Internet of Things, the system comprising: The light intensity monitoring unit 11 is used to monitor the light intensity of the current environment in real time through a light sensor and obtain the light change rate, wherein the light sensor is integrated on the smart glasses.

[0015] In an embodiment of the present application, when the smart glasses are started, the light sensor starts initialization work. The light sensor is a micro sensor integrated on the smart glasses, which is used to capture the light intensity of the environment irradiated on the surface of the glasses. This light sensor uses technologies such as photodiodes or photoresistors to convert the received light energy into electrical signals for output. After the initialization is completed, the light sensor enters the working state and begins to monitor the light intensity in the environment in real time. In order to determine the dynamic changes of the ambient light, the light change rate is calculated regularly. This step is achieved by comparing the light intensity values ​​at the current moment and the previous moment. Specifically, the light change rate is obtained by subtracting the difference between the current light intensity and the light intensity at the previous moment and removing the time interval. Among them, the time interval is the time difference of the sensor sampling. The calculation is completed by the built-in microprocessor of the glasses, and the calculated light change rate reflects the speed of change of the light over time.

[0016] The auxiliary display unit 12 is used to trigger the auxiliary display mode of the smart glasses when the light change rate is greater than a preset threshold value, and obtain a fused display image, wherein the fused display image is a fused display image of the infrared auxiliary mode and the frame display mode.

[0017] In an embodiment of the present application, when the light intensity monitoring unit in the smart glasses detects that the rate of change of light exceeds a preset threshold, that is, the light changes too fast and too drastically, the auxiliary display mode is triggered immediately. The preset threshold is a critical value of the rate of change of light set in advance. Exceeding this value indicates that the drastic change of ambient light may affect the user's visual experience. After triggering the auxiliary display mode, the smart glasses will enter the interval control interval. The purpose of this interval is to provide a smooth visual transition and avoid discomfort to the user caused by instantaneous occlusion of the lens due to drastic changes in light. In the interval control interval, the infrared auxiliary mode and the frame display mode are enabled, and a fused display image is generated.

[0018] The infrared assist mode uses the infrared sensor on the smart glasses to provide low-brightness, low-stimulation infrared image support in low-light or drastic changes, helping users maintain basic visual information and avoid sudden brightness changes that irritate the eyes. At the same time, the frame display mode temporarily locks the edge area of ​​the lens, stabilizes the edge of the user's field of vision through the frame image, and ensures that the central field of vision is not affected by drastic changes in lens brightness, focal length, etc. Then, the images of these two modes are fused to generate a fused display image. Furthermore, in the system provided in the embodiment of the application, the auxiliary display unit 12 is also used for: According to the infrared auxiliary mode in the auxiliary display mode, image capture is performed on the first visual area of ​​the smart glasses to obtain an infrared display image of the current environment, wherein the first visual area is the main visual area of ​​the user; according to the frame display mode in the auxiliary display mode, image capture is performed on the first visual area of ​​the smart glasses to obtain an outline display image of the current environment; image fusion is performed based on the infrared display image and the outline display image, and a fused display image is output.

[0019] In the embodiment of the present application, when the light changes dramatically, the infrared auxiliary mode is activated, which focuses on the first visual area of ​​the smart glasses, that is, the user's main visual area. This main visual area is the core part of the user's field of view and provides the most important visual information. The infrared auxiliary mode uses the infrared sensor in the smart glasses to collect images of the main visual area and obtain the infrared display image of the current environment.

[0020] At the same time, the frame display mode is started synchronously. The frame display mode uses the visible light camera on the smart glasses to collect images, focusing on the contour information of the first visual area. In this mode, the primary visual area is collected again, but this time the visible light camera is used to capture the visible light signal in the environment. In order to extract the contour information, an edge detection algorithm, such as the Sobel operator or the Canny edge detection algorithm, is used to identify the edges of objects in the visible light image and generate a contour display image.

[0021] After the acquisition of infrared images and contour images is completed, the image fusion stage begins. Image fusion relies on image registration technology and feature point matching algorithms, such as SIFT. Specifically, the infrared image and contour image are first spatially registered to ensure that the two images are accurately aligned in space. Then the SIFT algorithm is used to detect key feature points in the image and identify important detail information in the image. Through feature point matching, similar areas in the infrared image and contour image are found, aligned and fused. Finally, a fused display image is generated through weighted averaging or other fusion algorithms, such as image fusion based on wavelet transform.

[0022] Furthermore, in the system provided in the embodiment of the application, the auxiliary display unit 12 is also used for: According to the infrared auxiliary mode in the auxiliary display mode, image capture is performed on the first visual area of ​​the smart glasses to obtain an infrared display image of the current environment; according to the frame display mode in the auxiliary display mode, image capture is performed on the second visual area of ​​the smart glasses to obtain an outline display image of the current environment, wherein the second visual area is a neighborhood expansion area of ​​the first visual area; image fusion is performed based on the outline display image and the infrared display image, and a fused display image is output.

[0023] In an embodiment of the present application, in order to cope with the situation of drastic changes in light, visual acquisition of the first visual area and the second visual area is achieved through multi-sensor collaboration, and images from different sources are fused to provide a stable visual experience.

[0024] First, the infrared auxiliary mode is started, and the long-wave infrared sensor integrated in the smart glasses is used to capture images of the user's inner ring area, that is, the main visual area. The infrared sensor can sense the thermal radiation emitted by objects, especially in low light or drastic changes, and generates an infrared display image by capturing thermal signals. This infrared image contains the thermal radiation information of objects in the user's main visual area. At the same time, the frame display mode is started, and the visible light camera in the smart glasses is used to capture images of the user's outer ring area, that is, the second visual area. The outer ring area is an extended neighborhood part of the main visual area, covering the edge of the user's field of view. In order to maintain a stable perception of the environment when the light changes drastically, the edge detection algorithm is used to process the outer ring area to generate a contour display image.

[0025] Next, the infrared image from the inner ring and the contour image of the outer ring are fused through image registration technology and feature point matching algorithms such as SIFT or SURF algorithms. In the image fusion process, the weighted average method or wavelet transform algorithm is used to effectively merge the features of the two images, and finally the fused display image is output.

[0026] Furthermore, the system provided in the application embodiment is also used for: Acquire a display density feature of the current environment; if the display density feature is a central display density feature, perform image capture on a first visual area of ​​the smart glasses according to a frame display mode in the auxiliary display mode; if the display density feature is a non-central display density feature, perform image capture on a second visual area of ​​the smart glasses according to the frame display mode in the auxiliary display mode.

[0027] In the embodiment of the present application, the display density feature of the current environment is first obtained. To this end, an object detection algorithm based on a convolutional neural network, such as YOLO and an image segmentation model, such as U-Net, is used. These models are trained by supervised learning. A large number of annotated image data sets are used during training, and these data sets contain images in various environments and their corresponding object annotation information. These models adjust weight parameters through multiple rounds of forward propagation and back propagation, and gradually optimize their ability to identify object bounding boxes, object categories, and distribution features of objects in images. Specifically, the environment image is first collected in real time through a built-in visible light camera. The collected image is input into the trained YOLO model, and the model detects the category and position of all objects in the image. Then, the U-Net model further refines the boundaries of the object through pixel-level classification. Finally, the distribution of these objects in the visual area is analyzed. If the objects appear densely in the center of the field of view, it is judged as a central display density feature; if the objects are mainly concentrated in the peripheral area, it is judged as a non-central display density feature.

[0028] When the display density feature in the environment is detected as central, the frame display mode is started, and the image of the first visual area of ​​the user is focused on, and the image of the first visual area of ​​the smart glasses is collected according to the frame display mode in the auxiliary display mode. When the display density feature is judged to be non-central, the collection focus is shifted from the main visual area to the second visual area, and the image of the second visual area of ​​the smart glasses is collected according to the frame display mode in the auxiliary display mode.

[0029] Furthermore, in the system provided in the embodiment of the application, the auxiliary display unit 12 is also used for: The infrared display image and the contour display image are subjected to histogram equalization processing, and the processed infrared display image and the contour display image are output; feature point detection is performed on the processed infrared display image and the contour display image using a feature point matching algorithm, and SIFT feature points and SURF feature points are generated; SIFT feature descriptors and SURF feature descriptors are generated according to the SIFT feature points and the SURF feature points; image registration and fusion are performed based on the SIFT feature descriptors and the SURF feature descriptors, and a fused display image is output.

[0030] In the embodiment of the present application, the infrared display image and the contour display image are firstly subjected to histogram equalization processing, and the distribution of the brightness of each pixel is obtained by analyzing the pixel brightness distribution of each image and calculating the image histogram. Then, the brightness value of the pixel is redistributed based on the cumulative distribution function (CDF) of the image to enhance the contrast of the image, and finally the infrared display image and the contour display image after equalization processing are output.

[0031] Next, feature point detection is performed on the processed infrared display image and contour display image, using SIFT and SURF. The SIFT algorithm can detect key points in the image, and these points are highly robust to image rotation, scaling, and illumination changes; and the SURF algorithm, as an accelerated version of SIFT, can detect feature points faster and maintain high matching accuracy. In this step, the SIFT algorithm is used to detect multi-scale feature points, and the main direction of each feature point is calculated to make it rotation invariant; at the same time, the SURF algorithm quickly detects feature points in the image through the Hessian matrix, and finally outputs SIFT and SURF feature points in the infrared and contour images.

[0032] After the feature point detection is completed, a feature descriptor is generated for each feature point. These descriptors are used to describe the local image information around the feature point, which is convenient for subsequent image matching. For SIFT feature points, the gradient direction histogram in its neighborhood is calculated to generate a 128-dimensional SIFT feature descriptor; for SURF feature points, the Haar wavelet transform is performed on the pixel blocks around the feature point to generate a 64-dimensional SURF feature descriptor.

[0033] The infrared display image and the contour display image are then aligned using image registration technology. The core of image registration is to find corresponding points in the two images by matching feature points, and to calculate the geometric transformation between the images based on these point pairs, so that the two images are aligned in the same coordinate system. Specifically, the nearest neighbor search is first used to match the SIFT and SURF feature descriptors in the infrared image and the contour image, and the closest matching point pair is found by calculating the Euclidean distance of each pair of descriptors. At the same time, the nearest neighbor ratio test is used to improve the matching accuracy and eliminate poor matches. The RANSAC algorithm is then used to estimate the geometric transformation model between the images, such as affine transformation or perspective transformation, by randomly sampling point pairs to ensure the robustness of the image alignment. Finally, the transformation matrix is ​​applied to align the infrared and contour display images so that they are aligned in the same space.

[0034] After image registration is completed, image fusion is performed. In order to fuse the information in the infrared image and the contour image, the weighted average method or wavelet transform is used. The weighted average method performs a weighted average of the pixel values ​​of the two images to ensure that the fused image retains both the thermal information in the infrared image and the edge information in the contour image. The wavelet transform decomposes the image into different frequency levels through multi-scale decomposition technology, and performs information fusion at each level to ensure that both the details and global information of the image can be fully retained.

[0035] Through the above steps, a fused display image is generated, which contains both the thermal radiation information of the infrared display image and the edge structure information of the contour display image.

[0036] Furthermore, in the system provided in the embodiment of the application, the auxiliary display unit 12 is also used for: Determine whether the light change rate is a positive light change. If the light change rate is a positive light change, assign the preset threshold to be a first preset threshold; if the light change rate is a negative light change, assign the preset threshold to be a second preset threshold; wherein, a positive light change is a change in light intensity from low to high, and a negative light change is a change in light intensity from high to low.

[0037] In the embodiment of the present application, first, the smart glasses monitor the light intensity of the current environment in real time through the integrated light sensor. Once the light change rate is calculated, the type of light change is determined based on the positive or negative sign of the rate. The light change type is divided into positive light change and negative light change.

[0038] When the light change rate is positive, that is, the light intensity changes from low to high. This usually happens when the user walks from indoors to outdoors, such as from a low-brightness indoor environment to a high-brightness outdoor environment. At this time, the light intensity increases rapidly, which is judged as a positive light change. When the light change rate is negative, that is, the light intensity changes from high to low. This usually happens when the user walks from outdoors to indoors, such as from a bright outdoor environment to a darker indoor environment. At this time, the light intensity decreases rapidly, which is judged as a negative light change.

[0039] According to the type of light change, the corresponding preset threshold is adjusted. The threshold is the critical point used to determine when the smart glasses start or adjust their display mode. In different light change scenarios, the threshold is dynamically adjusted according to the magnitude of the change rate. If a positive light change is detected, the preset threshold is set to the first preset threshold due to the sharp increase in light intensity. This threshold is usually higher because in this case, it is necessary to respond to a larger change in light intensity to prevent the user from experiencing visual discomfort due to sudden strong light exposure. If a negative light change is detected, the preset threshold is set to the second preset threshold due to the rapid decrease in light intensity. This threshold is relatively low because in the case of reduced light, a quick adjustment is required to adapt to the dim environment to avoid visual impairment.

[0040] The adaptive control unit 13 is used to display the fused display image as the display content of the auxiliary display mode after triggering the auxiliary display mode. At the same time, the lens control module of the smart glasses performs adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set, wherein the control parameter set includes the brightness, focal length and filtering mode of the lens.

[0041] In the embodiment of the present application, when it is detected that the light change rate exceeds a set threshold, the adaptive control unit starts the auxiliary display mode. In this mode, the fused display image is displayed as the display content.

[0042] While the auxiliary display mode is triggered, the light sensor of the smart glasses continuously detects the ambient light intensity and transmits the data to the adaptive control unit in real time. The control unit uses a series of control algorithms, such as fuzzy control algorithm or PID control algorithm, to generate a control parameter set containing multiple lens adjustment parameters based on the real-time light data provided by the sensor. This parameter set mainly includes three key variables: brightness adjustment, focal length adjustment, and filter mode selection.

[0043] Specifically, when adjusting the brightness, the fuzzy control algorithm is used to adjust the brightness of the lens. The fuzzy control algorithm processes the input light intensity data to obtain the lens brightness suitable for the current environment. For example, when the light intensity increases sharply, the lens brightness will automatically decrease to reduce the stimulation of strong light to the user's eyes; conversely, when the light weakens, the lens brightness will increase to ensure that the user can still see the surrounding objects clearly in a low-light environment. To help users switch their field of view between objects at different distances, an autofocus algorithm is used. Based on the changes in light and focal distance, the algorithm adjusts the focal length of the lens through a micro-electric drive or liquid crystal lens to ensure that the user maintains visual clarity when quickly switching the focus of the object. The process is dynamically adjusted according to the user's line of sight, the distance of the object in the environment, or the focal length requirement. When selecting the filter mode, the scene recognition algorithm is used to identify the current light source type, such as sunlight, artificial light, etc., and automatically switch the filter mode of the lens. For example, when strong sunlight is detected, switch to polarized or UV protection mode to reduce the impact of glare; in a dim environment, enable the blue light protection mode to reduce eye fatigue.

[0044] The adaptive control unit ensures that each adjustment of the lens parameters can adapt to changes in external light through a closed-loop feedback control system. Closed-loop feedback control is a dynamic adjustment process that makes feedback adjustments based on the light intensity collected in real time. The control system continuously receives sensor data and compares the current state with the expected state through a feedback control algorithm to automatically adjust the lens brightness, focal length, and filter mode to reduce errors and ensure that the adjustment results meet expectations. After each adjustment, the sensor will re-collect the light intensity data in the current state and feed it back to the adaptive control unit. Through the feedback control mechanism, the error between the adjusted state and the expected state is corrected to ensure that the final lens parameters can meet the needs of the current ambient light.

[0045] Through the above steps, the smart glasses adjust the display content and lens parameters in real time when the light changes drastically, providing users with a smooth and stable visual experience.

[0046] The fitness analysis unit 14 is used to perform environmental fitness analysis according to the real-time control parameter set. If the preset environmental fitness is met, the auxiliary display mode is released and switched to the lens display mode after adaptive control.

[0047] In an embodiment of the present application, the fitness analysis unit performs an environmental fitness analysis based on the real-time control parameter set, specifically, analyzes visual clarity, brightness adaptability and light filtering effect. When analyzing visual clarity, the user's visual clarity in the current environment is analyzed by combining focal length adjustment with image processing algorithms. The measurement of visual clarity depends on image sharpness and the user's ability to distinguish the edges of objects. The visual data fed back by the camera and the focal length information detected by the sensor are used to evaluate whether the current visual clarity meets the set standards. When performing brightness adaptability analysis, the brightness adjustment of the lens is evaluated to meet the lighting requirements of the current environment based on the feedback data from the real-time light intensity sensor. When analyzing the light filtering effect, the scene recognition algorithm and the filter mode adjustment are used to evaluate whether the current light filtering effect can effectively reduce light interference.

[0048] The fitness analysis unit generates a real-time environmental fitness value based on the analysis results of the above three indicators. The fitness value represents the comprehensive performance of the current smart glasses in the current environment, ensuring that they can adapt to external light and user needs. The fitness value is calculated by combining the scores of different indicators through a weighted average algorithm to generate a comprehensive environmental fitness. Each indicator is assigned a different weight according to its importance.

[0049] Once the real-time environmental fitness value is calculated, the fitness analysis unit will compare it with the preset environmental fitness threshold. If the real-time environmental fitness is greater than or equal to the preset environmental fitness, it means that the current control parameter set has fully adapted to the current environmental light conditions, and the current visual experience is considered to have reached the optimal state. Therefore, the auxiliary display mode is released and switched to the lens display mode after adaptive control.

[0050] Furthermore, in the system provided in the embodiment of the application, the fitness analysis unit 14 is also used for: Perform an environmental adaptability analysis according to the real-time control parameter set, wherein the environmental adaptability analysis includes visual clarity, brightness adaptability and light filtering effect; output real-time environmental adaptability according to the visual clarity, the brightness adaptability and the light filtering effect; determine whether the real-time environmental adaptability is greater than or equal to the preset environmental adaptability, and if so, release the auxiliary display mode.

[0051] In the embodiment of the present application, the environmental adaptability is analyzed based on the real-time control parameter set to ensure that the smart glasses can provide a suitable visual experience in various environments. The core of the adaptability analysis is to evaluate the current visual clarity, brightness adaptability and light filtering effect, so as to determine whether the auxiliary display mode can be released.

[0052] Specifically, the acquired real-time control parameter set is used in combination with the current environmental conditions to perform environmental adaptability analysis to ensure the best visual experience in various light environments. The core of the adaptability analysis is to evaluate visual clarity, brightness adaptability, and light filtering effects, and to score these factors.

[0053] Visual clarity is achieved by evaluating the effect of focus adjustment. By using the built-in camera and image processing algorithm of smart glasses, it is evaluated whether the objects seen by the user in the current environment are clear. When performing the evaluation, the image in the user's field of view is first captured by the camera. The captured image is then processed using the Sobel algorithm or the Canny edge detection algorithm to identify the edges of the objects in the image. Edge detection evaluates the clarity of the object by calculating the brightness difference between the object and the background. Next, based on the edge detection results, the sharpness of the object edge is calculated using the gradient statistics method and quantified into a clarity score of 0 to 100 points. If the edge contrast is high and the detected object clarity reaches more than 80%, the clarity score is 90 to 100 points; if the sharpness is low, the score is less than 50 points.

[0054] The brightness adaptability score is based on the match between the lens brightness and the ambient light intensity. The ambient light intensity is detected in real time by a light sensor, and the adaptability is evaluated based on the adjusted lens brightness. When evaluating, the light intensity of the current environment is first detected by the light sensor, and the lens brightness is automatically adjusted based on the sensor data. The adjusted brightness value is compared with the light intensity of the current environment to calculate the match between the two. A match close to 1 indicates a good brightness adjustment, while a match close to 0 indicates poor adaptability. Finally, the match is converted into a brightness adaptability score ranging from 0 to 100 points. When the match is above 0.9, the score is 90 to 100 points; when it is below 0.5, the score is below 50 points.

[0055] The light filtering effect is evaluated through the filter mode of the smart glasses. When evaluating, the light filtering effect is evaluated by comparing the light intensity before and after the filter mode is enabled, that is, the light intensity before filtering is subtracted from the light intensity after filtering, and then the light intensity before filtering is removed to obtain the green light effect. If the light filtering effect is significant, that is, the light filtering effect reaches more than 80%, it means that the light filtering effect is good. The light filtering effect is quantified into a score of 0 to 100. When the light filtering effect exceeds 80%, the score is 90 to 100 points; when it is less than 50%, the score is below 50 points.

[0056] The comprehensive real-time environmental adaptability value is calculated by scoring the visual clarity, brightness adaptability and light filtering effect individually. The scores of each indicator are weighted using the weighted average method to obtain the final adaptability value, where the weights of visual clarity, brightness adaptability and light filtering effect are pre-set by technical experts.

[0057] Finally, the calculated real-time environmental adaptability will be compared with the preset adaptability threshold. If the real-time adaptability is greater than or equal to the preset threshold, the auxiliary display mode is released and switched to the lens display mode after adaptive control; if it is lower than the threshold, the auxiliary display mode is maintained.

[0058] In the embodiments of the present application, in summary, the embodiments of the present application have at least the following technical effects: The present application monitors the light intensity of the current environment in real time through a light sensor to obtain the light change rate, wherein the light sensor is integrated on the smart glasses; when the light change rate is greater than a preset threshold, the auxiliary display mode of the smart glasses is triggered to obtain a fused display image, wherein the fused display image is a fused display image of an infrared auxiliary mode and a frame display mode; after the auxiliary display mode is triggered, the fused display image is displayed as the display content of the auxiliary display mode, and at the same time, the lens control module of the smart glasses performs adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set, wherein the control parameter set includes the brightness, focal length and filtering mode of the lens; an environmental adaptability analysis is performed according to the real-time control parameter set, and if the preset environmental adaptability is met, the auxiliary display mode is released, and the lens display mode after adaptive control is switched. The present invention solves the technical problem in the prior art that when the light changes dramatically, it is easy to cause excessive adjustment of lenses, resulting in instantaneous visual occlusion, affecting the user's visual experience and visual stability. The present invention uses a built-in light sensor in the smart glasses to monitor light changes in real time, trigger an auxiliary display mode, fuse infrared and visible light images, adaptively adjust brightness, focal length and filter mode, generate a real-time control parameter set, and ensure visual stability through environmental adaptability analysis. When the adaptability is met, it switches to the adaptive display mode, thereby achieving the technical effect of ensuring a smooth visual transition and visual stability of the user under light changes.

[0059] Embodiment 2 is based on the same inventive concept as the system for adaptively controlling smart glasses in combination with the Internet of Things in the above embodiment. Figure 2 As shown, the embodiment of the present application provides a method for adaptively controlling smart glasses in combination with the Internet of Things, the method comprising: The light intensity of the current environment is monitored in real time by a light sensor to obtain the light change rate, wherein the light sensor is integrated on the smart glasses; when the light change rate is greater than a preset threshold, the auxiliary display mode of the smart glasses is triggered to obtain a fused display image, wherein the fused display image is a fused display image of an infrared auxiliary mode and a frame display mode; after the auxiliary display mode is triggered, the fused display image is displayed as the display content of the auxiliary display mode, and at the same time, the lens control module of the smart glasses performs adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set, wherein the control parameter set includes the brightness, focal length and filtering mode of the lens; an environmental adaptability analysis is performed according to the real-time control parameter set, and if the preset environmental adaptability is met, the auxiliary display mode is released, and the lens display mode after adaptive control is switched.

[0060] Furthermore, the method for obtaining a fused display image further includes: According to the infrared auxiliary mode in the auxiliary display mode, image capture is performed on the first visual area of ​​the smart glasses to obtain an infrared display image of the current environment, wherein the first visual area is the main visual area of ​​the user; according to the frame display mode in the auxiliary display mode, image capture is performed on the first visual area of ​​the smart glasses to obtain an outline display image of the current environment; image fusion is performed based on the infrared display image and the outline display image, and a fused display image is output.

[0061] Furthermore, the method for obtaining a fused display image further includes: According to the infrared auxiliary mode in the auxiliary display mode, image capture is performed on the first visual area of ​​the smart glasses to obtain an infrared display image of the current environment; according to the frame display mode in the auxiliary display mode, image capture is performed on the second visual area of ​​the smart glasses to obtain an outline display image of the current environment, wherein the second visual area is a neighborhood expansion area of ​​the first visual area; image fusion is performed based on the outline display image and the infrared display image, and a fused display image is output.

[0062] Furthermore, the method further comprises: Get the display density characteristics of the current environment; If the display density feature is a central display density feature, image capture is performed on the first visual area of ​​the smart glasses according to the frame display mode in the auxiliary display mode; if the display density feature is a non-central display density feature, image capture is performed on the second visual area of ​​the smart glasses according to the frame display mode in the auxiliary display mode.

[0063] Furthermore, performing image fusion according to the infrared display image and the contour display image, and outputting a fused display image, the method further includes: The infrared display image and the contour display image are subjected to histogram equalization processing, and the processed infrared display image and the contour display image are output; feature point detection is performed on the processed infrared display image and the contour display image using a feature point matching algorithm, and SIFT feature points and SURF feature points are generated; SIFT feature descriptors and SURF feature descriptors are generated according to the SIFT feature points and the SURF feature points; image registration and fusion are performed based on the SIFT feature descriptors and the SURF feature descriptors, and a fused display image is output.

[0064] Further, performing environmental adaptability analysis according to the real-time control parameter set, the method further includes: Perform an environmental adaptability analysis according to the real-time control parameter set, wherein the environmental adaptability analysis includes visual clarity, brightness adaptability and light filtering effect; output real-time environmental adaptability according to the visual clarity, the brightness adaptability and the light filtering effect; determine whether the real-time environmental adaptability is greater than or equal to the preset environmental adaptability, and if so, release the auxiliary display mode.

[0065] Furthermore, when the light change rate is greater than a preset threshold, the method further includes: Determine whether the light change rate is a positive light change. If the light change rate is a positive light change, assign the preset threshold to be a first preset threshold; if the light change rate is a negative light change, assign the preset threshold to be a second preset threshold; wherein, a positive light change is a change in light intensity from low to high, and a negative light change is a change in light intensity from high to low.

[0066] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. The processes depicted in the accompanying drawings do not necessarily require the specific order and continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0067] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0068] This specification and drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A system for adaptive control of smart glasses combined with the Internet of Things, characterized in that: The method comprises: A light intensity monitoring unit, the light intensity monitoring unit is used to monitor the light intensity of the current environment in real time through a light sensor to obtain the light change rate, wherein the light sensor is integrated on the smart glasses; An auxiliary display unit, wherein the auxiliary display unit is used to trigger the auxiliary display mode of the smart glasses and obtain a fused display image when the light change rate is greater than a preset threshold, wherein the fused display image is a fused display image of an infrared auxiliary mode and a frame display mode; An adaptive control unit, wherein after the auxiliary display mode is triggered, the adaptive control unit is used to display the fused display image as the display content of the auxiliary display mode, and at the same time, the lens control module of the smart glasses performs adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set, wherein the control parameter set includes the brightness, focal length and filter mode of the lens; The fitness analysis unit is used to perform environmental fitness analysis according to the real-time control parameter set. If the preset environmental fitness is met, the auxiliary display mode is released and switched to the lens display mode after adaptive control.

2. The system for adaptively controlling smart glasses in combination with the Internet of Things as claimed in claim 1, characterized in that: Acquire fused display images, the system includes: Performing image acquisition on a first visual area of ​​the smart glasses according to the infrared auxiliary mode in the auxiliary display mode to obtain an infrared display image of the current environment, wherein the first visual area is a main visual area of ​​the user; Capturing an image of a first visual area of ​​the smart glasses according to a frame display mode in the auxiliary display mode to obtain a contour display image of a current environment; Image fusion is performed based on the infrared display image and the contour display image, and a fused display image is output.

3. The system for adaptively controlling smart glasses in combination with the Internet of Things as claimed in claim 1, characterized in that: Acquire the fused display image, the system also includes: Performing image acquisition on the first visual area of ​​the smart glasses according to the infrared auxiliary mode in the auxiliary display mode to obtain an infrared display image of the current environment; Performing image acquisition on a second visual area of ​​the smart glasses according to the frame display mode in the auxiliary display mode to obtain a contour display image of the current environment, wherein the second visual area is a neighborhood expansion area of ​​the first visual area; Image fusion is performed based on the contour display image and the infrared display image, and a fused display image is output.

4. The system for adaptively controlling smart glasses in combination with the Internet of Things as claimed in claim 1, characterized in that: The system further comprises: Get the display density characteristics of the current environment; If the display density feature is a central display density feature, performing image acquisition on a first visual area of ​​the smart glasses according to a frame display mode in the auxiliary display mode; If the display density feature is a non-center display density feature, image capture is performed on the second visual area of ​​the smart glasses according to the frame display mode in the auxiliary display mode.

5. The system for adaptively controlling smart glasses in combination with the Internet of Things as claimed in any one of claims 2 or 3, characterized in that: Performing image fusion according to the infrared display image and the contour display image, and outputting a fused display image, the system comprises: Performing histogram equalization processing on the infrared display image and the contour display image, and outputting the processed infrared display image and contour display image; The feature point matching algorithm is used to detect the feature points of the processed infrared display image and contour display image to generate SIFT feature points and SURF feature points; Generate a SIFT feature descriptor and a SURF feature descriptor according to the SIFT feature points and the SURF feature points; Image registration and fusion are performed based on the SIFT feature descriptor and the SURF feature descriptor, and a fused display image is output.

6. The system for adaptively controlling smart glasses in combination with the Internet of Things as claimed in claim 1, characterized in that: According to the real-time control parameter set, the environmental adaptability analysis is performed, and the system includes: Performing an environmental adaptability analysis according to the real-time control parameter set, wherein the environmental adaptability analysis includes visual clarity, brightness adaptability, and light filtering effect; Outputting real-time environmental adaptability according to the visual clarity, the brightness adaptability and the light filtering effect; It is determined whether the real-time environmental adaptability is greater than or equal to the preset environmental adaptability. If the real-time environmental adaptability is greater than or equal to the preset environmental adaptability, the auxiliary display mode is released.

7. The system for adaptively controlling smart glasses in combination with the Internet of Things as claimed in claim 1, characterized in that: When the light change rate is greater than a preset threshold, the system includes: Determine whether the light change rate is a positive light change, if the light change rate is a positive light change, assign the preset threshold to be a first preset threshold, if the light change rate is a negative light change, assign the preset threshold to be a second preset threshold; Among them, the positive example light change is the change of light intensity from low to high, and the negative example light change is the change of light intensity from high to low.

8. A method for adaptively controlling smart glasses in combination with the Internet of Things, characterized in that: The method is performed by a system for adaptively controlling smart glasses in combination with the Internet of Things according to any one of claims 1 to 7, comprising: Monitor the light intensity of the current environment in real time through a light sensor to obtain the light change rate, wherein the light sensor is integrated on the smart glasses; When the light change rate is greater than a preset threshold, the auxiliary display mode of the smart glasses is triggered to obtain a fused display image, wherein the fused display image is a fused display image of the infrared auxiliary mode and the frame display mode; After the auxiliary display mode is triggered, the fused display image is displayed as the display content of the auxiliary display mode, and at the same time, the lens control module of the smart glasses performs adaptive control according to the light intensity of the current environment to obtain a real-time control parameter set, wherein the control parameter set includes the brightness, focal length and filter mode of the lens; An environmental adaptability analysis is performed according to the real-time control parameter set. If the preset environmental adaptability is met, the auxiliary display mode is released and switched to the lens display mode after adaptive control.

Images