Control method and system based on wearable intelligent equipment

Abstract

The invention discloses a control method and system based on wearable intelligent equipment, and relates to the technical field of intelligent control. Static complex factors of a current scene are analyzed based on a scene recognition model, and dynamic complex factors of the current scene are analyzed after video stream data are acquired in a monitoring time period; and the processing end substitutes the static complex factor and the dynamic complex factor into the fusion model for calculation to obtain the overall complexity of the current scene, dynamically adjusts a complexity threshold by combining the motion posture data of the user to obtain a corrected complexity threshold, and controls the operation of the intelligent glasses according to a comparison result of the overall complexity and the corrected complexity threshold. The control system analyzes the scene where the user is located in real time, so that the intelligent glasses are intelligently controlled when the user is in the complex scene, the intelligent control content is adjusted in combination with the current motion state of the user, the user is effectively helped to improve the concentration degree, and the risk of traffic accidents is reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to a control method and system based on a wearable intelligent device. Background Art

[0002] Smart glasses are a type of wearable device that integrates electronic technology and eyewear functions. They are designed to provide functions such as information display, augmented reality (AR), virtual reality (VR), and voice control by embedding intelligent modules such as display screens, sensors, cameras, and audio outputs. The design goal of smart glasses is usually to enable users to obtain information, interact, and improve user experience without interrupting their daily activities.

[0003] The prior art has the following deficiencies: The functions of smart glasses, such as voice commands, touch operations, screen displays, etc., all require users to switch their attention to a certain extent. When riding or driving, the user's attention should be focused on road conditions, traffic signals and other traffic participants. Frequent switching of attention may lead to slow reactions, especially in complex scenarios such as intersections of people and vehicles and pedestrians. Since the brain needs time to switch attention and process information, users may fail to respond in time, increasing the risk of traffic accidents.

[0004] Based on this, the present invention proposes a control method and system based on wearable smart devices, which conducts real-time analysis of the scene in which the user is located, thereby intelligently controlling the smart glasses when the user is in a complex scene, and adjusts the intelligent control content in combination with the user's current motion state, effectively helping the user improve concentration and reduce the risk of traffic accidents. Summary of the invention

[0005] The purpose of the present invention is to provide a control method and system based on a wearable smart device to solve the deficiencies in the background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a control method based on a wearable smart device, the control method comprising the following steps: The acquisition end is started when the smart glasses are turned on and runs, and multiple frames of real-time images are obtained through the sensor devices on the smart glasses; The static complexity factor of the current scene is analyzed based on the scene recognition model. After acquiring the video stream data during the monitoring period, the dynamic complexity factor of the current scene is analyzed. The processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model for calculation to obtain the overall complexity of the current scene. The complexity threshold is dynamically adjusted in combination with the user's motion posture data to obtain a corrected complexity threshold, and the operation of the smart glasses is controlled based on the comparison result between the overall complexity and the corrected complexity threshold.

[0007] In a preferred embodiment, the processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model calculation to obtain the overall complexity of the current scene. The fusion model expression is: , where is the overall complexity, is the dynamic complexity factor, is the static complexity factor, , is the adjustment coefficient, and , Both are greater than 0.

[0008] In a preferred embodiment, dynamically adjusting the complexity threshold in combination with the user's motion posture data to obtain a modified complexity threshold includes the following steps: The user's motion posture data is obtained, and the motion posture data includes the user's risk coefficient. The user's risk coefficient is compared with the preset first risk threshold and the second risk threshold. If the user's risk coefficient is greater than the second risk threshold, the complexity threshold needs to be reduced to increase the monitoring intensity. If the user's risk coefficient is less than the first risk threshold, the complexity threshold needs to be increased to reduce the monitoring intensity. If the user's risk coefficient is greater than or equal to the first risk threshold, and the user's risk coefficient is less than or equal to the second risk threshold, there is no need to adjust the complexity threshold. The adjustment algorithm is: , where To correct the complexity threshold, is the initial complexity threshold, is the user risk factor, is the first risk threshold, is the second risk threshold.

[0009] In a preferred embodiment, controlling the operation of the smart glasses according to the comparison result between the overall complexity and the modified complexity threshold comprises the following steps: The obtained overall complexity is compared with the modified complexity threshold. If the overall complexity is less than or equal to the modified complexity threshold, the operation of the smart glasses will not be intervened. If the overall complexity is greater than the modified complexity threshold, the operation of the smart glasses will be intervened. The control includes: controlling the display of the smart glasses to turn off.

[0010] In a preferred embodiment, the user's motion posture data includes a user risk coefficient, and the calculation logic of the user risk coefficient is: obtain the user's motion speed and the user's turning frequency, normalize the user's motion speed and the user's turning frequency, map the value range of the user's motion speed and the user's turning frequency to between [0,1], obtain the normalized value of the user's motion speed and the normalized value of the user's turning frequency, and sum the normalized value of the user's motion speed and the normalized value of the user's turning frequency to obtain the user risk coefficient.

[0011] In a preferred embodiment, after acquiring the video stream data during the monitoring period, analyzing the dynamic complexity factor of the current scene includes the following steps: After acquiring the video stream data, the frame difference method is used to identify dynamic objects in the scene, the movement of objects is detected by comparing the changes between consecutive frames, the change amount of each frame is calculated and the moving area of ​​the dynamic object is determined, and the object tracking algorithm is used to track the movement path of each dynamic object in the video; Based on the position changes of objects in consecutive frames, the movement speed of each object is calculated, the number of times each object appears in the time window is recorded, the frequency of object appearance in the scene is calculated, and the object complexity factor is calculated based on the object's movement speed and object appearance frequency.

[0012] In a preferred embodiment, the calculation logic of the object complexity factor is: normalize the object movement speed and the object appearance frequency, and sum the normalized object movement speed and object appearance frequency to obtain the dynamic complexity factor.

[0013] In a preferred embodiment, the acquisition end is started when the smart glasses are turned on and running, and multiple frames of real-time images are acquired through the sensor device on the smart glasses, and the static complex factors of the current scene are analyzed based on the scene recognition model, including the following steps: The built-in camera of smart glasses collects multi-frame image data of the current environment in real time, uses image denoising algorithm to remove sensor noise, and uses image enhancement technology to enhance image details; Use scene recognition models to perform object detection and scene segmentation to identify and label different elements in the image; Based on the identified scene elements and the spatial distribution information of the scene, the static complexity factor is calculated.

[0014] In a preferred embodiment, the static complexity factor is quantified by the following dimensions: Spatial density: including the density of objects on the road, the number and arrangement of buildings; Object types and layout: including the types of objects in the environment, including vehicles, pedestrians, traffic signs and layout complexity, including intersections and traffic facilities; Obstacles and visual range: including obstructions in the environment and the wearer's field of view.

[0015] A control system based on a wearable smart device, including an acquisition module, an analysis module, a calculation module, a correction module, and a control module: Acquisition module: starts when the smart glasses are turned on and running, obtains multiple frames of real-time images through the sensor devices on the smart glasses, obtains video stream data during the monitoring period, and sends multiple frames of real-time images and video stream data to the analysis module; Analysis module: Analyzes the static complexity factor of the current scene based on the scene recognition model, analyzes the dynamic complexity factor of the current scene based on the video stream data, and sends the static complexity factor and the dynamic complexity factor to the calculation module; Calculation module: Substitute the static complexity factor and the dynamic complexity factor into the fusion model for calculation to obtain the overall complexity of the current scene, and send the overall complexity to the control module; Correction module: dynamically adjusts the complexity threshold in combination with the user's motion posture data to obtain a corrected complexity threshold, and sends the corrected complexity threshold to the control module; Control module: controls the operation of smart glasses based on the comparison result between the overall complexity and the modified complexity threshold.

[0016] In the above technical solution, the technical effects and advantages provided by the present invention are: The present invention analyzes the static complexity factor of the current scene based on the scene recognition model, and after acquiring the video stream data in the monitoring time period, analyzes the dynamic complexity factor of the current scene, substitutes the static complexity factor and the dynamic complexity factor into the fusion model for calculation, obtains the overall complexity of the current scene, dynamically adjusts the complexity threshold in combination with the user's motion posture data to obtain the modified complexity threshold, and controls the operation of the smart glasses based on the comparison result between the overall complexity and the modified complexity threshold. The control system performs real-time analysis of the scene in which the user is located, thereby intelligently controlling the smart glasses when the user is in a complex scene, and adjusts the intelligent control content in combination with the user's current motion state, effectively helping the user to improve concentration and reduce the risk of traffic accidents.

[0017] The present invention helps to more accurately predict and perceive environmental changes by combining dynamic and static factors. For example, in a dynamic traffic scene, the frequent movement of objects (such as vehicles and pedestrians) may cause drastic changes in traffic conditions, while in a static environment (such as a shopping mall or a home scene), the changes in static factors may be more significant. The method can dynamically adjust the evaluation of complexity in real time, which helps real-time system decision-making and response. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0019] Figure 1 The present invention is a flow chart of the method. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Example 1: Please refer to Figure 1 As shown, the control method based on a wearable smart device described in this embodiment includes the following steps: The acquisition end is started when the smart glasses are turned on and run. It obtains multiple frames of real-time images through the sensor equipment on the smart glasses, analyzes the static complexity factor of the current scene based on the scene recognition model, and analyzes the dynamic complexity factor of the current scene after obtaining the video stream data during the monitoring period. The processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model for calculation to obtain the overall complexity of the current scene, dynamically adjusts the complexity threshold based on the user's motion posture data to obtain the corrected complexity threshold, and controls the operation of the smart glasses based on the comparison result of the overall complexity and the corrected complexity threshold.

[0022] This application analyzes the static complexity factor of the current scene based on the scene recognition model, and after obtaining the video stream data during the monitoring period, analyzes the dynamic complexity factor of the current scene, substitutes the static complexity factor and the dynamic complexity factor into the fusion model for calculation, obtains the overall complexity of the current scene, dynamically adjusts the complexity threshold in combination with the user's motion posture data to obtain the corrected complexity threshold, and controls the operation of the smart glasses based on the comparison result between the overall complexity and the corrected complexity threshold. The control system performs real-time analysis of the scene in which the user is located, thereby intelligently controlling the smart glasses when the user is in a complex scene, and adjusts the intelligent control content in combination with the user's current motion state, effectively helping users improve their concentration and reduce the risk of traffic accidents.

[0023] Embodiment 2: The acquisition end is started when the smart glasses are turned on and running, and multiple frames of real-time images are acquired through the sensor device on the smart glasses, and the static complex factors of the current scene are analyzed based on the scene recognition model, including the following steps: Smart glasses use built-in cameras or other visual sensors to collect multiple frames of image data in the current environment in real time. These images will be used for subsequent scene analysis to help identify the complexity of the surrounding environment, using image denoising algorithms (such as Gaussian filtering, bilateral filtering, etc.) to remove sensor noise; image enhancement techniques (such as histogram equalization, Laplace transform, etc.) are used to enhance image details to meet the needs of subsequent complex factor analysis.

[0024] Use scene recognition models (such as convolutional neural networks CNN, YOLO, Mask R-CNN, etc.) to perform object detection and scene segmentation, identify different elements in the image and label them. In this way, the system is able to extract objects in the environment.

[0025] Using the Mask R-CNN model for object detection and scene segmentation, identifying and labeling different elements in an image includes the following steps: Using the Mask R-CNN model for object detection and scene segmentation, it can identify different elements in the image and generate a corresponding label for each element. Mask R-CNN is an extension of Faster R-CNN. In addition to conventional object detection, it can also generate a pixel-level mask for each object. The following are the detailed steps to achieve object detection and scene segmentation: Input image preparation: First, you need to prepare the input image and perform appropriate preprocessing on the image. The image should be of high quality and the size of the image should be suitable for the input requirements of the model.

[0026] Image scaling and normalization: Scale the image to a suitable size (e.g. 800×800) and normalize it to ensure that the range of each pixel value is suitable for the network input requirements.

[0027] Select a pre-trained model: You can choose a pre-trained Mask R-CNN model, such as the one trained on the COCO dataset. This allows for fast object detection and segmentation on many common objects.

[0028] You can use deep learning frameworks like Detectron2, TensorFlow Object Detection API, or Keras to load the pre-trained Mask R-CNN model.

[0029] Model structure: Backbone: ResNet or ResNeXt is usually used as a feature extraction network to extract high-level features from images.

[0030] Region Proposal Network (RPN): Generates potential object regions (i.e. candidate boxes) and scans the image through a sliding window.

[0031] ROIAlign: Perform feature pooling on the generated candidate boxes to accurately extract features and perform subsequent processing.

[0032] Branch head: Mask R-CNN adds a branch to predict the segmentation mask for each candidate box.

[0033] Region Proposal Network (RPN): Mask R-CNN first uses RPN to generate candidate object regions (Region Proposals). RPN scans the image through a sliding window mechanism, generates multiple candidate boxes, and calculates a binary classification score of the object and the background for each candidate box.

[0034] Candidate box screening: The candidate boxes generated by RPN are screened, usually using non-maximum suppression (NMS) to remove boxes with too high overlap, and only retain the candidate boxes that are most likely to contain objects.

[0035] ROIAlign operation: For the candidate boxes filtered by RPN, use the ROIAlign operation to accurately map these areas to the feature map. ROIAlign can solve the accuracy problem in the traditional ROI Pooling method and obtain more refined features.

[0036] Feature extraction: Extract high-level features of each candidate region through a feature extraction network (such as ResNet). These features will be used for subsequent classification and segmentation tasks.

[0037] Object classification: Use the fully connected layer to classify the features of each candidate box to determine which object category the candidate box belongs to.

[0038] Mask prediction: Mask R-CNN adds a branch to the traditional object detection task to generate a binary mask for each object. For each candidate box, the network generates a pixel-level binary mask that represents the exact shape of the object in the area.

[0039] Classification result: For each candidate box, output the category label of the object.

[0040] Mask output: For each object, output the corresponding binary mask, which represents the exact shape of the object in the image.

[0041] Bounding box: The bounding box of each object is predicted and annotated in the image, usually represented by a rectangular box.

[0042] Non-Maximum Suppression (NMS): To remove duplicate boxes, the NMS algorithm is usually used to select the box with the highest score and remove those boxes that have a high overlap with other boxes.

[0043] Mask refinement: Further processing is performed to ensure the accuracy of the mask. The mask is clipped to match the shape of the object to ensure that the mask of each object is accurate.

[0044] Visualization results: Display the results of object detection on the image, including the object's category label, bounding box, and mask. The mask is usually superimposed on the object area in a semi-transparent way to facilitate comparison with the original image.

[0045] Bounding Box: The bounding box of each object is marked with a different color.

[0046] Mask: The mask of an object is usually displayed using transparent layers of different colors to highlight the shape of the object.

[0047] The above solution is implemented through Python tool code as follows: Import-cv2 Import-numpy-as-np Import-matplotlib.pyplot as plt From-maskrcnn_benchmark import MaskRCNN # Assume that MaskRCNN is used for implementation # Load the pre-trained model model = MaskRCNN.from_pretrained("path_to_pretrained_model") # Read the image image = cv2.imread('input_image.jpg') # Perform object detection and segmentation outputs = model(image) # Output includes categories, masks, bounding boxes, etc. class_ids = outputs['class_ids'] masks = outputs['masks'] boxes = outputs['boxes'] # Visualize the results For-i-in-range(len(masks)): mask = masks[i] box = boxes[i] class_id = class_ids[i] # Draw the bounding box cv2.rectangle(image, (box[0], box[1]), (box[2], box[3]), (0, 255,0), 2) # Draw the mask plt.imshow(mask, alpha=0.5) # semi-transparent mask plt.title(f'Class {class_id}') plt.show() # Display the image cv2.imshow('Detected Image', image) cv2.waitKey(0) cv2.destroyAllWindows() By using the Mask R-CNN model, the system can detect and classify objects in the image and generate accurate segmentation masks for each object. This not only helps to identify the elements in the scene, but also can finely segment the scene and provide higher quality environmental perception information, which is especially suitable for application scenarios that require high-precision object positioning and segmentation, such as autonomous driving, intelligent monitoring, augmented reality, etc.

[0048] Based on the segmentation and object detection results, the static complexity factor of the current scene is analyzed. Static complexity factors include but are not limited to factors such as buildings, road signs, object density, and site layout in the environment. The static complexity factor is calculated based on the identified scene elements and the spatial distribution information of the scene.

[0049] The calculation logic of the static complexity factor is as follows: obtain the spatial density, object type layout index and obstacle vision impact index of the current scene, normalize the spatial density, object type layout index and obstacle vision impact index, and then sum the normalized spatial density, object type layout index and obstacle vision impact index to obtain the static complexity factor. The larger the value of the static complexity factor, the more complex the scene is and the greater the impact on distracting user attention.

[0050] The calculation logic of spatial density is as follows: the number of all static objects (such as buildings, road signs, and vehicles parked on the road) identified within the camera's field of view is calculated using the field of view angle and the depth information of the image acquisition to obtain the visible area area. The number of all static objects is divided by the visible area area to obtain the spatial density. Spatial density reflects the density of static objects per unit area. The larger the value, the more objects there are in the environment, the more crowded the scene, the more restricted the user's field of vision, and the increased risk of distraction. If the spatial density value is low, it means that the objects in the environment are sparsely distributed, the scene is more open, the user's field of vision is clearer, the risk of distraction is smaller, and the complexity of the scene is lower. When the spatial density value is high, the objects in the environment are denser, the scene becomes more complex, visual interference and potential dangers increase, the user's attention may be distracted, and the complexity is higher.

[0051] The calculation logic of the object type layout index is as follows: after obtaining the number of different object types and the number of all static objects in the visible area, the object type ratio is obtained by dividing the number of different object types by the number of all static objects; after obtaining the average distance between objects, the object layout ratio is obtained by dividing the average distance between objects by the number of all static objects; the object type ratio is multiplied by the object layout ratio to obtain the object type layout index; The object type layout index combines the diversity of object types in the scene and the complexity of the layout between objects. It reflects the diversity of object types and the degree of chaos in the layout in the environment. If the index is low, it means that there are fewer types of objects in the scene, the layout is simple, the objects in the environment are relatively scattered and the spacing is large, the user's attention is less distracted, and the scene complexity is low. When the index is high, it means that the objects in the scene are rich in variety and the layout is complex. The distribution of objects is relatively close or there are many intersections, which may lead to increased visual interference, easy distraction of users, and high scene complexity.

[0052] The calculation logic of the obstacle vision impact index is as follows: obtain the number of occluders and the number of all static objects in the visible area, divide the number of occluders by the number of all static objects to obtain the occluder density, obtain the area of ​​the occluded area and the area of ​​the visible area, divide the area of ​​the occluded area by the area of ​​the visible area to obtain the restricted vision ratio, and add the occluder density to the restricted vision ratio to obtain the obstacle vision impact index; The obstacle vision impact index reflects the occluders in the field of vision and the degree of impact of these occluders on the user's field of vision. The higher the value, the higher the degree of obstruction of the field of vision, which affects the user's perception of the environment and increases the risk of distraction. If the index is low, it means that there are fewer obstructions in the environment or the field of vision is less restricted, the user can clearly see the surrounding objects and roads, the scene complexity is low, and the user's attention is not easily distracted. If the index is high, it means that the field of vision is blocked by more obstacles, or a large area is blocked, the user's perception of the surrounding environment is greatly affected, the attention is easily distracted, and the scene complexity is high.

[0053] Static complexity factors can be quantified along the following dimensions: Spatial density: such as the density of objects on the road, the number and arrangement of buildings, etc.

[0054] Object types and layout: The types of objects in the environment (such as vehicles, pedestrians, traffic signs) and the complexity of their layout (such as intersections, complex traffic facilities).

[0055] Obstacles and visual range: possible obstructions in the environment (such as buildings, trees, etc.) and the wearer's field of view.

[0056] After acquiring the video stream data during the monitoring period, the dynamic complexity factors of the current scene are analyzed, including the following steps: After acquiring the video stream data, use background subtraction, frame difference method or optical flow method to identify dynamic objects in the scene. Through these technologies, dynamic objects can be separated from static backgrounds to detect the movement of objects. The movement of objects can be detected by comparing the changes between consecutive frames. The change amount of each frame can be calculated and the moving area of ​​the dynamic object can be determined. Object tracking algorithms (such as Kalman filtering, Mean-shift, SORT, etc.) can be used to track the movement path of each dynamic object in the video.

[0057] Based on the position change of the object in consecutive frames, the movement speed of each object can be calculated. The common method is to calculate the displacement of the object between two frames by Euclidean distance, and then calculate the speed according to the time between frames. That is, the displacement of the object between two frames in the video stream is first calculated by the Euclidean distance algorithm, and then the time difference between the two frames in the video stream is calculated. Finally, the object's movement speed is obtained by dividing the displacement between the two frames in the video stream by the time difference. Record the number of times each object appears in a certain time window to measure the activity of the object. If the object appears or disappears frequently, the dynamic complexity of the scene is high. For each object detected in each frame, calculate its frequency of appearance in the scene. When the object appears frequently, the dynamic complexity of the scene is high. The object frequency is obtained by dividing the number of object appearances by the total number of image frames in the time period.

[0058] The object's movement speed and object's appearance frequency are normalized, and the normalized object's movement speed and object's appearance frequency are summed to obtain the dynamic complexity factor. If the dynamic complexity factor is high, it means the scene is complex. The system can increase the alarm or provide more detailed information to the user. In a dynamic and complex scene, the smart glasses can reduce the display of information, or remind the user through sound and vibration to avoid visual overload.

[0059] The processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model calculation to obtain the overall complexity of the current scene, including the following steps: The processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model calculation to obtain the overall complexity of the current scene. The fusion model expression is: , where is the overall complexity, is the dynamic complexity factor, is the static complexity factor, , is the adjustment coefficient, and , Both are greater than 0.

[0060] By combining the two types of factors, the present application can more accurately reflect the complexity of the actual scene, especially in dynamic scenes, where the impact of static factors may be small, but dynamic factors can significantly change the complexity.

[0061] The weighting coefficients α and β in the model make this method flexible. According to actual needs, the influence of dynamic complexity factor and static complexity factor on the overall complexity can be controlled by adjusting these two parameters. If dynamic changes have a greater impact on scene complexity, α can be increased and β can be reduced; conversely, if static factors are more important, β can be adjusted to be larger.

[0062] This weighting method can be adapted to different application scenarios. For example, in traffic monitoring, dynamic complexity factors (such as traffic flow, vehicle speed, etc.) may be more important; while in indoor environment monitoring, static complexity factors (such as furniture layout, space density, etc.) may be more influential. By adjusting the coefficients α and β, we can flexibly respond to complexity assessments in different environments and optimize the system's responsiveness.

[0063] Combining dynamic and static factors helps to more accurately predict and perceive environmental changes. For example, in dynamic traffic scenarios, the frequent movement of objects (such as vehicles and pedestrians) may cause drastic changes in traffic conditions, while in static environments (such as shopping malls or home scenes), the changes in static factors may be more significant. This method can dynamically adjust the evaluation of complexity in real time, which helps real-time system decision-making and response.

[0064] Dynamically adjusting the complexity threshold in combination with the user's motion posture data to obtain a modified complexity threshold includes the following steps: The user's motion posture data is obtained, the motion posture data includes a user risk coefficient, the user's motion speed and the user's turning frequency are obtained, the user's motion speed and the user's turning frequency are normalized, the user's motion speed and the user's turning frequency are mapped to a value range between [0,1], the user's motion speed normalized value and the user's turning frequency normalized value are obtained, and the user's motion speed normalized value and the user's turning frequency normalized value are summed to obtain the user risk coefficient.

[0065] The larger the user risk factor, the higher the risk the user may face during the current exercise and appropriate intervention or reminder is needed.

[0066] Moreover, the greater the overall complexity of the current scene, the more dynamic and static objects there are in the current scene, and the more complex the scene is; Normally, after obtaining the overall complexity of the current scene, the overall complexity is compared with the preset complexity threshold to determine whether it is necessary to control the operation of the smart glasses. However, in actual applications, if the user is in a slower movement speed or a low turning frequency (such as walking), even in a complex scene, there is no need to turn off the display prompts of the smart glasses (unless the scene is very complex). If the user is in a faster movement speed or a low turning frequency (such as cycling, running, etc.), it is necessary to increase the monitoring of user reminders and shut down the smart glasses.

[0067] Therefore, after obtaining the user risk coefficient, we adjust the preset complexity threshold according to the user risk coefficient. The adjustment steps are as follows: The user risk factor is compared with the preset first risk threshold and the second risk threshold. If the user risk factor is greater than the second risk threshold, the complexity threshold needs to be reduced to increase the monitoring intensity. If the user risk factor is less than the first risk threshold, the complexity threshold needs to be increased to reduce the monitoring intensity. If the user risk factor is greater than or equal to the first risk threshold and the user risk factor is less than or equal to the second risk threshold, there is no need to adjust the complexity threshold. The specific adjustment algorithm is: , where To correct the complexity threshold, is the initial complexity threshold, is the user risk factor, is the first risk threshold, is the second risk threshold.

[0068] The operation of the smart glasses is controlled according to the comparison result between the overall complexity and the modified complexity threshold, including the following steps: The obtained overall complexity is compared with the corrected complexity threshold. If the overall complexity is less than or equal to the corrected complexity threshold, the operation of the smart glasses will not be intervened. If the overall complexity is greater than the corrected complexity threshold, the operation of the smart glasses will be intervened. The control includes: controlling the smart glasses to turn off the display, and turning on the display for a period of time (for example, 10 seconds) and then turning it off (for example, 2 seconds) at intervals, or continuously detecting the overall complexity of the current scene, and combining it with the navigation information, if the overall complexity continues to be greater than the corrected complexity threshold and the current scene is straight driving, then continue to turn off the smart glasses display; if the overall complexity continues to be greater than the corrected complexity threshold and the current scene requires turning, then turn on the smart glasses display.

[0069] Embodiment 3: The control system based on a wearable smart device described in this embodiment includes a collection module, an analysis module, a calculation module, a correction module, and a control module: Acquisition module: starts when the smart glasses are turned on and running, obtains multiple frames of real-time images through the sensor devices on the smart glasses, obtains video stream data during the monitoring period, and sends multiple frames of real-time images and video stream data to the analysis module; Analysis module: Analyzes the static complexity factor of the current scene based on the scene recognition model, analyzes the dynamic complexity factor of the current scene based on the video stream data, and sends the static complexity factor and the dynamic complexity factor to the calculation module; Calculation module: Substitute the static complexity factor and the dynamic complexity factor into the fusion model for calculation to obtain the overall complexity of the current scene, and send the overall complexity to the control module; Correction module: dynamically adjusts the complexity threshold in combination with the user's motion posture data to obtain a corrected complexity threshold, and sends the corrected complexity threshold to the control module; Control module: controls the operation of smart glasses based on the comparison result between the overall complexity and the modified complexity threshold.

[0070] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0071] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0072] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0073] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0074] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A control method based on a wearable smart device, characterized in that: The control method comprises the following steps: The acquisition end is started when the smart glasses are turned on and runs, and multiple frames of real-time images are obtained through the sensor devices on the smart glasses; The static complexity factor of the current scene is analyzed based on the scene recognition model. After acquiring the video stream data during the monitoring period, the dynamic complexity factor of the current scene is analyzed. The processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model for calculation to obtain the overall complexity of the current scene. The complexity threshold is dynamically adjusted in combination with the user's motion posture data to obtain a corrected complexity threshold, and the operation of the smart glasses is controlled based on the comparison result between the overall complexity and the corrected complexity threshold.

2. A control method based on a wearable smart device according to claim 1, characterized in that: The processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model calculation to obtain the overall complexity of the current scene. The fusion model expression is: , where is the overall complexity, is the dynamic complexity factor, is the static complexity factor, , is the adjustment coefficient, and , Both are greater than 0.

3. A control method based on a wearable intelligent device according to claim 2, characterized in that: Dynamically adjusting the complexity threshold in combination with the user's motion posture data to obtain a modified complexity threshold includes the following steps: The user's motion posture data is obtained, and the motion posture data includes the user's risk coefficient. The user's risk coefficient is compared with the preset first risk threshold and the second risk threshold. If the user's risk coefficient is greater than the second risk threshold, the complexity threshold needs to be reduced to increase the monitoring intensity. If the user's risk coefficient is less than the first risk threshold, the complexity threshold needs to be increased to reduce the monitoring intensity. If the user's risk coefficient is greater than or equal to the first risk threshold, and the user's risk coefficient is less than or equal to the second risk threshold, there is no need to adjust the complexity threshold. The adjustment algorithm is: , where To correct the complexity threshold, is the initial complexity threshold, is the user risk factor, is the first risk threshold, is the second risk threshold.

4. A control method based on a wearable intelligent device according to claim 3, characterized in that: The operation of the smart glasses is controlled according to the comparison result between the overall complexity and the modified complexity threshold, including the following steps: The obtained overall complexity is compared with the modified complexity threshold. If the overall complexity is less than or equal to the modified complexity threshold, the operation of the smart glasses will not be intervened. If the overall complexity is greater than the modified complexity threshold, the operation of the smart glasses will be intervened. The control includes: controlling the display of the smart glasses to turn off.

5. A control method based on a wearable intelligent device according to claim 4, characterized in that: The user's motion posture data includes a user risk coefficient. The calculation logic of the user risk coefficient is as follows: obtain the user's motion speed and the user's turning frequency, normalize the user's motion speed and the user's turning frequency, map the value range of the user's motion speed and the user's turning frequency to between [0,1], obtain the normalized value of the user's motion speed and the normalized value of the user's turning frequency, and sum the normalized value of the user's motion speed and the normalized value of the user's turning frequency to obtain the user risk coefficient.

6. A control method based on a wearable intelligent device according to claim 5, characterized in that: After acquiring the video stream data during the monitoring period, the dynamic complexity factors of the current scene are analyzed, including the following steps: After acquiring the video stream data, the frame difference method is used to identify dynamic objects in the scene, the movement of objects is detected by comparing the changes between consecutive frames, the change amount of each frame is calculated and the moving area of ​​the dynamic object is determined, and the object tracking algorithm is used to track the movement path of each dynamic object in the video; Based on the position changes of objects in consecutive frames, the movement speed of each object is calculated, the number of times each object appears in the time window is recorded, the frequency of object appearance in the scene is calculated, and the object complexity factor is calculated based on the object's movement speed and object appearance frequency.

7. A control method based on a wearable intelligent device according to claim 6, characterized in that: The calculation logic of the object complexity factor is: normalize the object movement speed and the object appearance frequency, and sum the normalized object movement speed and object appearance frequency to obtain the dynamic complexity factor.

8. A control method based on a wearable intelligent device according to claim 7, characterized in that: The acquisition end is started when the smart glasses are turned on and running, and multiple frames of real-time images are obtained through the sensor devices on the smart glasses. The static complex factors of the current scene are analyzed based on the scene recognition model, including the following steps: The built-in camera of smart glasses collects multi-frame image data of the current environment in real time, uses image denoising algorithm to remove sensor noise, and uses image enhancement technology to enhance image details; Use scene recognition models to perform object detection and scene segmentation to identify and label different elements in the image; Based on the identified scene elements and the spatial distribution information of the scene, the static complexity factor is calculated.

9. A control method based on a wearable smart device according to claim 8, characterized in that: The static complexity factor is quantified by the following dimensions: Spatial density: including the density of objects on the road, the number and arrangement of buildings; Object types and layout: including the types of objects in the environment, including vehicles, pedestrians, traffic signs and layout complexity, including intersections and traffic facilities; Obstacles and visual range: including obstructions in the environment and the wearer's field of view.

10. A control system based on a wearable smart device, used to implement the control method according to any one of claims 1 to 9, characterized in that: Including acquisition module, analysis module, calculation module, correction module and control module: Acquisition module: starts when the smart glasses are turned on and running, obtains multiple frames of real-time images through the sensor devices on the smart glasses, obtains video stream data during the monitoring period, and sends multiple frames of real-time images and video stream data to the analysis module; Analysis module: Analyzes the static complexity factor of the current scene based on the scene recognition model, analyzes the dynamic complexity factor of the current scene based on the video stream data, and sends the static complexity factor and the dynamic complexity factor to the calculation module; Calculation module: Substitute the static complexity factor and the dynamic complexity factor into the fusion model for calculation to obtain the overall complexity of the current scene, and send the overall complexity to the control module; Correction module: dynamically adjusts the complexity threshold in combination with the user's motion posture data to obtain a corrected complexity threshold, and sends the corrected complexity threshold to the control module; Control module: controls the operation of smart glasses based on the comparison result between the overall complexity and the modified complexity threshold.

Images