A control method and system based on a wearable smart device

Abstract

The application discloses a control method and system based on a wearable smart device, relates to the technical field of intelligent control, and is characterized in that a static complex factor of a current scene is analyzed based on a scene recognition model, and after video stream data is acquired in a monitoring time period, a dynamic complex factor of the current scene is analyzed, the static complex factor and the dynamic complex factor are substituted into a fusion model for calculation by a processing end, the overall complexity of the current scene is acquired, the complexity threshold value is dynamically adjusted in combination with motion posture data of a user to acquire a corrected complexity threshold value, and the smart glasses are controlled to operate according to a comparison result of the overall complexity and the corrected complexity threshold value. The control system performs real-time analysis on a scene in which a user is located, thereby intelligently controlling the smart glasses when the user is in a complex scene, and adjusting intelligent control content in combination with a current motion state of the user, so that the user can effectively improve concentration and reduce the risk of traffic accidents.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent control, in particular to a control method and system based on a wearable smart device. BACKGROUND

[0002] Smart glasses are a wearable device that integrates electronic technology and glasses functions, aiming to provide information display, augmented reality (AR), virtual reality (VR), voice control and other functions by embedding display screens, sensors, cameras, audio outputs and other smart modules. The design goal of smart glasses is usually to enable users to obtain information, interact and improve user experience without interrupting daily activities.

[0003] The prior art has the following disadvantages:

[0004] The functions of smart glasses such as voice commands, touch operations, screen displays, etc. all require users to switch attention to a certain extent. When riding or driving, the user's attention should be focused on road conditions, traffic signals and other road users. Frequent attention switching may lead to delayed reactions, especially in complex scenarios such as people and vehicles intersecting and pedestrians crossing, as the brain needs time to switch attention and process information, which may result in the user failing to react in time, increasing the risk of traffic accidents.

[0005] Based on this, the present application proposes a control method and system based on a wearable smart device, which analyzes the scene in which the user is located in real time, thereby intelligently controlling the smart glasses when the user is in a complex scene, and adjusting the intelligent control content in combination with the current motion state of the user, effectively helping the user to improve concentration and reduce the risk of traffic accidents. SUMMARY

[0006] The purpose of the present application is to provide a control method and system based on a wearable smart device to solve the problems in the background art.

[0007] In order to achieve the above-mentioned purpose, the present application provides the following technical solution: a control method based on a wearable smart device, the control method comprising the following steps:

[0008] The acquisition end is started when the smart glasses are powered on and run, and a plurality of real-time images are obtained through the sensing device on the smart glasses;

[0009] Based on the scene recognition model, the static complexity factors of the current scene are analyzed, and after the video stream data is obtained in the monitoring period, the dynamic complexity factors of the current scene are analyzed. The processing end calculates the overall complexity of the current scene by substituting the static complexity factors and the dynamic complexity factors into the fusion model;

[0010] The complexity threshold is dynamically adjusted according to the motion posture data of the user to obtain a revised complexity threshold, and the smart glasses are controlled according to the comparison result of the overall complexity and the revised complexity threshold.

[0011] In a preferred embodiment, the processing end inputs the static complexity factor and the dynamic complexity factor into a fusion model to calculate the overall complexity of the current scene, and the expression of the fusion model is:

[0012] In the formula, C is the overall complexity, F is the dynamic complexity factor, S is the static complexity factor, and a and b are both greater than 0.

[0013] In a preferred embodiment, the complexity threshold is dynamically adjusted according to the motion posture data of the user to obtain a revised complexity threshold, including the following steps:

[0014] The motion posture data of the user is obtained, and the motion posture data includes a user risk coefficient. The user risk coefficient is compared with a preset first risk threshold and a second risk threshold. If the user risk coefficient is greater than the second risk threshold, the complexity threshold needs to be reduced to increase the monitoring intensity. If the user risk coefficient is less than the first risk threshold, the complexity threshold needs to be increased to reduce the monitoring intensity. If the user risk coefficient is greater than or equal to the first risk threshold and less than or equal to the second risk threshold, the complexity threshold does not need to be adjusted, and the adjustment algorithm is:

[0015] In the formula, C is the revised complexity threshold, C0 is the initial complexity threshold, R is the user risk coefficient, R1 is the first risk threshold, and R2 is the second risk threshold.

[0016] In a preferred embodiment, the smart glasses are controlled according to the comparison result of the overall complexity and the revised complexity threshold, including the following steps:

[0017] The obtained overall complexity is compared with the revised complexity threshold. If the overall complexity is less than or equal to the revised complexity threshold, the smart glasses are not intervened to run. If the overall complexity is greater than the revised complexity threshold, the smart glasses are intervened to run, and the control includes controlling the smart glasses to display off.

[0018] ​​​​​​​​​​​​In a preferred embodiment, the motion posture data of the user includes a user risk coefficient, and the calculation logic of the user risk coefficient is: obtaining a user motion speed and a user steering frequency, performing normalization processing on the user motion speed and the user steering frequency, mapping the value range of the user motion speed and the user steering frequency to [0, 1], obtaining a user motion speed normalized value and a user steering frequency normalized value, and summing the user motion speed normalized value and the user steering frequency normalized value to obtain the user risk coefficient.

[0019] In a preferred embodiment, after obtaining the video stream data in the monitoring time period, the dynamic complexity factor of the current scene is analyzed, including the following steps:

[0020] After obtaining the video stream data, the frame difference method technology is used to identify the dynamic objects in the scene, the motion of the objects is detected by comparing the changes between consecutive frames, the change amount of each frame is calculated and the motion area of the dynamic objects is determined, and the object tracking algorithm is used to track the motion path of each dynamic object in the video;

[0021] Based on the position changes of the objects in consecutive frames, the motion speed of each object is calculated, the number of times each object appears in a time window is recorded, the object appearance frequency in the scene is calculated, and the object complexity factor is calculated according to the object motion speed and the object appearance frequency.

[0022] In a preferred embodiment, the calculation logic of the object complexity factor is: performing normalization processing on the object motion speed and the object appearance frequency, and summing the normalized object motion speed and the normalized object appearance frequency to obtain the dynamic complexity factor.

[0023] In a preferred embodiment, the collection end is started when the smart glasses are turned on and run, a plurality of real-time images are obtained through the sensing devices on the smart glasses, the static complexity factor of the current scene is analyzed based on a scene recognition model, and the following steps are included:

[0024] Real-time image data in the current environment is collected through the built-in camera of the smart glasses, image denoising algorithm is used to remove sensor noise, and image enhancement technology is used to enhance the details of the image;

[0025] Object detection and scene segmentation are performed using the scene recognition model, different elements in the image are identified and labeled;

[0026] According to the identified scene elements, the static complexity factor is calculated in combination with the spatial distribution information of the scene.

[0027] In a preferred embodiment, the static complexity factor is quantified in the following several dimensions:

[0028] Spatial density: including the density of objects on the road, the number and arrangement of buildings;

[0029] Object type and layout: including the type of objects in the environment, including vehicles, pedestrians, traffic signs, and layout complexity, including intersections and traffic facilities;

[0030] Obstacles and visual range: including the presence of obstacles in the environment and the wearer's field of view.

[0031] A control system based on a wearable smart device, comprising a collection module, an analysis module, a calculation module, a correction module and a control module:

[0032] The collection module: start when the smart glasses are turned on, acquire multiple real-time images through the sensing device on the smart glasses, acquire video stream data in the monitoring period, and send the multiple real-time images and the video stream data to the analysis module;

[0033] The analysis module: based on the scene recognition model to analyze the static complex factors of the current scene, based on the video stream data to analyze the dynamic complex factors of the current scene, and send the static complex factors and the dynamic complex factors to the calculation module;

[0034] The calculation module: the static complex factors and the dynamic complex factors are substituted into the fusion model to calculate the overall complexity of the current scene, and the overall complexity is sent to the control module;

[0035] The correction module: dynamically adjust the complexity threshold value to obtain the corrected complexity threshold value in combination with the user's motion posture data, and send the corrected complexity threshold value to the control module;

[0036] The control module: controls the operation of the smart glasses according to the comparison result of the overall complexity and the corrected complexity threshold value.

[0037] In the above technical solution, the technical effects and advantages provided by the present application are:

[0038] The present application analyzes the static complex factors of the current scene based on the scene recognition model, and after acquiring the video stream data in the monitoring period, analyzes the dynamic complex factors of the current scene, substitutes the static complex factors and the dynamic complex factors into the fusion model to calculate the overall complexity of the current scene, dynamically adjusts the complexity threshold value in combination with the user's motion posture data to obtain the corrected complexity threshold value, and controls the operation of the smart glasses according to the comparison result of the overall complexity and the corrected complexity threshold value. The control system analyzes the scene in which the user is located in real time, thereby intelligently controlling the smart glasses when the user is in a complex scene, and adjusting the intelligent control content in combination with the current motion state of the user, effectively helping the user to improve the concentration and reduce the risk of traffic accidents.

[0039] The present application helps to more accurately predict and perceive environmental changes by using dynamic and static factors in combination. For example, in a dynamic traffic scene, the frequent movement of objects (such as vehicles, pedestrians) can cause dramatic changes in traffic conditions, while in a static environment (such as a shopping mall or home scene), changes in static factors can be more significant. This method can dynamically adjust the complexity assessment in real time, helping real-time system decision-making and response. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0041] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0042] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0043] Embodiment 1: Please refer to Figure 1 As shown in the figure, the control method based on a wearable smart device described in this embodiment includes the following steps:

[0044] The acquisition end is started when the smart glasses are turned on and run, acquires multiple real-time images through the sensing device on the smart glasses, analyzes the static complex factors of the current scene based on a scene recognition model, and after acquiring video stream data in a monitoring time period, analyzes the dynamic complex factors of the current scene. The processing end substitutes the static complex factors and the dynamic complex factors into a fusion model to calculate the overall complexity of the current scene, dynamically adjusts the complexity threshold value in combination with the user's motion posture data to obtain a corrected complexity threshold value, and controls the smart glasses to run according to the comparison result of the overall complexity and the corrected complexity threshold value.

[0045] The application analyzes the static complexity factors of the current scene based on a scene recognition model, and after obtaining video stream data in a monitoring time period, analyzes the dynamic complexity factors of the current scene. The static complexity factors and the dynamic complexity factors are put into a fusion model for calculation to obtain the overall complexity of the current scene. The complexity threshold is dynamically adjusted in combination with the user's motion posture data to obtain a corrected complexity threshold, and the intelligent glasses are controlled according to the comparison result of the overall complexity and the corrected complexity threshold. The control system analyzes the scene in which the user is located in real time, thereby intelligently controlling the intelligent glasses when the user is in a complex scene, and adjusting the intelligent control content in combination with the current motion state of the user, effectively helping the user to improve concentration and reduce the risk of traffic accidents.

[0046] Embodiment 2: The collection end starts when the intelligent glasses are turned on and run, obtains multiple frames of real-time images through the sensing device on the intelligent glasses, analyzes the static complexity factors of the current scene based on a scene recognition model, including the following steps:

[0047] The intelligent glasses collect multiple frames of image data in the current environment in real time through the built-in camera or other visual sensors. These images will be used for subsequent scene analysis to help identify the complexity of the surrounding environment. Image denoising algorithms such as Gaussian filtering and bilateral filtering are used to remove sensor noise; image enhancement techniques such as histogram equalization and Laplace transform are used to enhance the details of the image to adapt to the needs of subsequent complexity factor analysis.

[0048] Object detection and scene segmentation are performed using a scene recognition model such as Convolutional Neural Network (CNN), YOLO, Mask R-CNN, etc. Different elements in the image are identified and labeled. In this way, the system can extract objects in the environment.

[0049] Object detection and scene segmentation are performed using a Mask R-CNN model to identify different elements in the image and label them, including the following steps:

[0050] Object detection and scene segmentation are performed using a Mask R-CNN model to identify different elements in the image and generate a corresponding label for each element. Mask R-CNN is an extension of Faster R-CNN, which can not only perform regular object detection but also generate a pixel-level mask for each object. The following are the detailed steps for object detection and scene segmentation:

[0051] Input image preparation: First, the input image needs to be prepared and the image needs to be preprocessed appropriately. The image should be of high quality and the size of the image should be appropriate for the input requirements of the model.

[0052] Image Scaling and Normalization: Scale the image to a suitable size (e.g., 800x800) and normalize it to ensure that each pixel value falls within the range required by the network.

[0053] Selecting a Pre-trained Model: Choose a pre-trained Mask R-CNN model, such as one trained on the COCO dataset. This allows for fast object detection and segmentation on many common objects.

[0054] Loading the Pre-trained Model: Use a deep learning framework like Detectron2, TensorFlow Object Detection API, or Keras to load the pre-trained Mask R-CNN model.

[0055] Model Architecture:

[0056] Backbone: Typically uses ResNet or ResNeXt as the feature extraction network to extract high-level features from the image.

[0057] Region Proposal Network (RPN): Generates potential object regions (i.e., candidate boxes) by scanning the image with a sliding window mechanism.

[0058] ROIAlign: Performs feature pooling on the generated candidate boxes to accurately extract features for further processing.

[0059] Branch Head: Mask R-CNN adds a branch to predict a segmentation mask for each candidate box.

[0060] Region Proposal Network (RPN): Mask R-CNN first uses the RPN to generate candidate object regions (Region Proposals). The RPN scans the image with a sliding window mechanism, generates multiple candidate boxes, and computes a binary classification score for each candidate box indicating whether it contains an object or not.

[0061] Candidate Box Filtering: Filters the candidate boxes generated by the RPN, typically using Non-Maximum Suppression (NMS) to remove highly overlapping boxes and keep only the most likely ones containing objects.

[0062] ROIAlign Operation: For the filtered candidate boxes from the RPN, use the ROIAlign operation to accurately map these regions to the feature maps. ROIAlign addresses the precision issues of traditional ROI Pooling methods, resulting in finer features.

[0063] Feature Extraction: High-level features are extracted from each candidate region using a feature extraction network such as ResNet. These features will be used for subsequent classification and segmentation tasks.

[0064] Object Classification: A fully connected layer is used to classify the features of each candidate box, determining which object category the candidate box belongs to.

[0065] Mask Prediction: Mask R-CNN adds a branch to the traditional object detection task to generate a binary mask for each object. For each candidate box, the network generates a pixel-level binary mask representing the precise shape of the object within that region.

[0066] Classification Results: For each candidate box, the class label of the object is outputted.

[0067] Mask Output: For each object, the corresponding binary mask is outputted, which represents the precise shape of the object in the image.

[0068] Bounding Box: The bounding box of each object is predicted and labeled in the image, usually represented by a rectangular frame.

[0069] Non-Maximum Suppression (NMS): To remove duplicate boxes, the NMS algorithm is often used to select the highest scoring box and delete those with high overlap with other boxes.

[0070] Mask Refinement: Through further processing, the accuracy of the mask is ensured. The mask is cropped to match the shape of the object to ensure that each object's mask is accurate.

[0071] Visualization Results: The results of object detection are displayed on the image, including the class label of the object, the bounding box, and the mask. The mask is usually overlaid on the object's area in a semi-transparent manner to contrast with the original image.

[0072] Bounding Box: The bounding box of each object is labeled with different colors.

[0073] Mask: The mask of the object is usually displayed with different colored transparent layers, highlighting the shape of the object.

[0074] The above scheme is implemented through python tool code as follows:

[0075] Import-cv2

[0076] Import-numpy-as-np

[0077] Import-matplotlib.pyplot as plt

[0078] From-maskrcnn_benchmark import MaskRCNN # Assuming MaskRCNN is used for implementation

[0079] # Load the pre-trained model

[0080] model = MaskRCNN.from_pretrained("path_to_pretrained_model")

[0081] # Read Image

[0082] image = cv2.imread('input_image.jpg')

[0083] # Perform object detection and segmentation

[0084] outputs = model(image)

[0085] # Output includes category, mask, bounding box, etc.

[0086] class_ids = outputs['class_ids']

[0087] masks = outputs['masks']

[0088] boxes = outputs['boxes']

[0089] # Visualization Results

[0090] For i-in-range(len(masks)):

[0091] mask = masks[i]

[0092] box = boxes[i]

[0093] class_id = class_ids[i]

[0094] # Draw bounding box

[0095] cv2.rectangle(image, (box[0], box[1]), (box[2], box[3]), (0, 255,0), 2)

[0096] # Draw a mask

[0097] plt.imshow(mask, alpha=0.5) # Semi-transparent mask

[0098] plt.title(f'Class {class_id}')

[0099] plt.show()

[0100] # Display image

[0101] cv2.imshow('Detected Image', image)

[0102] cv2.waitKey(0)

[0103] cv2.destroyAllWindows()

[0104] By using the Mask R-CNN model, the system can detect and classify objects in images and generate accurate segmentation masks for each object. This not only helps identify individual elements in a scene but also enables fine-grained scene segmentation, providing higher-quality environmental perception information. It is particularly suitable for applications requiring high-precision object localization and segmentation, such as autonomous driving, intelligent surveillance, and augmented reality.

[0105] Based on the segmentation and object detection results, the static complexity factors of the current scene are analyzed. Static complexity factors include, but are not limited to, factors such as buildings, road signs, object density, and site layout in the environment. The static complexity factors are calculated based on the identified scene elements and the spatial distribution information of the scene.

[0106] The calculation logic of the static complexity factor is as follows: obtain the spatial density, object type layout index, and obstacle vision impact index of the current scene; normalize the spatial density, object type layout index, and obstacle vision impact index; and sum the normalized spatial density, object type layout index, and obstacle vision impact index to obtain the static complexity factor. The larger the value of the static complexity factor, the more complex the scene is, and the greater the impact on the distraction of user attention.

[0107] The calculation logic for spatial density is as follows: within the camera's field of view, the number of all identified static objects (such as buildings, road signs, vehicles parked on the road, etc.) is used to calculate the visible area using the field of view and the depth information acquired from the image. The spatial density is obtained by dividing the number of all static objects by the visible area.

[0108] The spatial density reflects the density of static objects in a unit area. The greater the value, the more objects in the environment, the more crowded the scene, the more limited the user's field of view, and the greater the risk of distraction. If the spatial density value is low, it means that the objects in the environment are distributed sparsely, the scene is open, the user's field of view is clearer, and the risk of distraction is smaller. When the spatial density value is high, the objects in the environment are more dense, the scene becomes more complex, the visual interference and potential danger increase, the user's attention may be distracted, and the complexity is higher.

[0109] The calculation logic of the object category layout index is: after obtaining the number of different object categories in the visible area and the number of all static objects, the object category proportion is obtained by dividing the number of different object categories by the number of all static objects, after obtaining the average distance between objects, the object layout proportion is obtained by dividing the average distance between objects by the number of all static objects, and the object category layout index is obtained by multiplying the object category proportion by the object layout proportion.

[0110] The object category layout index combines the diversity of object categories in the scene and the layout complexity of objects. It reflects the diversity of object categories and the degree of layout confusion in the environment. If the index is low, it means that the scene has fewer object categories, the layout is simple, the objects in the environment are relatively dispersed and have a large distance, the user's attention is less distracted, and the scene complexity is lower. When the index is high, it means that the scene has rich object categories and complex layout, the distribution of objects is relatively close or has many intersection points, which may increase visual interference and easily distract the user's attention, and the scene complexity is higher.

[0111] The calculation logic of the obstacle field of view influence index is: after obtaining the number of occluding objects in the visible area and the number of all static objects, the occluding object density is obtained by dividing the number of occluding objects by the number of all static objects, after obtaining the area of the occluded area and the area of the visible area, the field of view limited proportion is obtained by dividing the area of the occluded area by the area of the visible area, and the obstacle field of view influence index is obtained by adding the occluding object density to the field of view limited proportion.

[0112] The obstacle field of view influence index reflects the occluding objects in the field of view and the degree of influence of these occluding objects on the user's field of view. The higher the value, the higher the degree of occlusion of the field of view, which affects the user's perception of the environment and increases the risk of distraction. If the index is low, it means that there are fewer occluding objects in the environment or the degree of field of view limitation is low, the user can clearly see the surrounding objects and roads, the scene complexity is low, and the user's attention is not easily distracted. If the index is high, it means that the field of view is more occluded by obstacles, or a large area is occluded, the user's perception of the surrounding environment is greatly affected, the attention is easily distracted, and the scene complexity is higher.

[0113] Static complexity factors can be quantified by the following dimensions:

[0114] Spatial density: For example, the density of objects on the road, the number and arrangement of buildings, etc.

[0115] Object type and layout: The complexity of the types of objects in the environment (such as vehicles, pedestrians, traffic signs) and the layout (such as intersections, complex traffic facilities).

[0116] Obstacles and visibility range: The presence of obstructions in the environment (such as buildings, trees, etc.) and the wearer's field of view.

[0117] After obtaining the video stream data in the monitoring period, analyze the dynamic complexity factors of the current scene, including the following steps:

[0118] After obtaining the video stream data, use background subtraction, frame difference or optical flow technology to identify dynamic objects in the scene. Through these technologies, dynamic objects can be separated from static backgrounds, so as to detect the motion of objects, and the motion of objects can be detected by comparing the changes between consecutive frames. The amount of change in each frame can be calculated and the motion area of dynamic objects can be determined, and object tracking algorithms (such as Kalman filter, Mean-shift, SORT, etc.) can be used to track the motion path of each dynamic object in the video.

[0119] Based on the position change of the object in the consecutive frames, the motion speed of each object can be calculated. The common method is to calculate the displacement of the object between two frames through Euclidean distance, and then calculate the speed according to the time difference between frames, that is, first calculate the displacement of the object between two frames of video stream through Euclidean distance algorithm, then calculate the time difference between two frames of video stream, and finally calculate the motion speed of the object by dividing the displacement of the object between two frames of video stream by the time difference;

[0120] Record the number of times each object appears within a certain time window to measure the activity of the object. If the object frequently appears or disappears, the dynamic complexity of the scene is high. The frequency of the object appearing in the scene is calculated, and the frequency of the object appearing in the scene is calculated. If the frequency of the object appearing in the scene is high, the dynamic complexity of the scene is high, and the frequency of the object appearing in the scene is calculated by dividing the number of times the object appears by the total number of frames in the time period.

[0121] Normalize the object motion speed and object appearance frequency, sum the normalized object motion speed and object appearance frequency to obtain the dynamic complexity factor. If the dynamic complexity factor is high, it indicates that the scene is complex, and the system can increase the alarm or provide more detailed information to the user. In a dynamic complex scene, the smart glasses can reduce the display of information, or remind the user through sound and vibration to avoid visual overload.

[0122] The processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model to calculate the overall complexity of the current scene, including the following steps:

[0123] The processing end substitutes the static complexity factor and the dynamic complexity factor into the fusion model to calculate the overall complexity of the current scene, and the expression of the fusion model is:

[0124] In the formula, C is the overall complexity, Cdyn is the dynamic complexity factor, Cstat is the static complexity factor, α and β are adjustment coefficients, and , both α and β are greater than 0. ,

[0125] The present application can more accurately reflect the complexity of the actual scene by combining the two types of factors. In particular, in a dynamic scene, the influence of static factors may be small, but dynamic factors can significantly change the complexity.

[0126] The weighting coefficients α and β in the model make the method flexible. According to actual needs, the influence of the dynamic complexity factor and the static complexity factor on the overall complexity can be controlled by adjusting these two parameters. If the dynamic change has a greater impact on the complexity of the scene, α can be increased and β can be decreased; conversely, if static factors are more important, β can be adjusted to be larger.

[0127] This weighting method can be adapted in different application scenarios. For example, in traffic monitoring, dynamic complexity factors (such as traffic flow, vehicle speed, etc.) may be more important; while in indoor environment monitoring, static complexity factors (such as furniture arrangement, space density, etc.) may have more influence. By adjusting the coefficients α and β, the complexity evaluation in different environments can be flexibly responded to, and the response ability of the system can be optimized.

[0128] The combination of dynamic and static factors helps to more accurately predict and perceive environmental changes. For example, in a dynamic traffic scene, the frequent movement of objects (such as vehicles, pedestrians) can cause dramatic changes in traffic conditions, while in a static environment (such as a shopping mall or home scene), changes in static factors may be more significant. This method can dynamically adjust the complexity evaluation in real time, which helps real-time system decision-making and response.

[0129] The user's motion posture data is combined to dynamically adjust the complexity threshold to obtain a corrected complexity threshold, including the following steps:

[0130] ​Obtaining the motion posture data of the user, the motion posture data including a user risk coefficient, obtaining the user motion speed and the user turning frequency, normalizing the user motion speed and the user turning frequency, mapping the value range of the user motion speed and the user turning frequency to [0, 1], obtaining the user motion speed normalized value and the user turning frequency normalized value, and summing the user motion speed normalized value and the user turning frequency normalized value to obtain the user risk coefficient.

[0131] The greater the user risk coefficient, the more likely the user faces higher risk in the current motion process, and appropriate intervention or reminder is needed.

[0132] Moreover, the greater the overall complexity of the current scene, the more dynamic and static objects in the current scene, and the more complex the scene is.

[0133] Generally, after obtaining the overall complexity of the current scene, the overall complexity is compared with a preset complexity threshold to determine whether the smart glasses need to be controlled to run. However, in actual application, if the user is at a slow motion speed or a small turning frequency (for example, walking), even in a complex scene, there is no need to turn off the display prompt of the smart glasses (unless the scene is very complex). If the user is at a fast motion speed or a small turning frequency (for example, cycling, running, etc.), the monitoring intensity of the user reminder and the smart glasses needs to be increased.

[0134] Therefore, after obtaining the user risk coefficient, the preset complexity threshold is adjusted according to the user risk coefficient, and the adjustment steps are as follows:

[0135] The user risk coefficient is compared with a preset first risk threshold and a second risk threshold. If the user risk coefficient is greater than the second risk threshold, the complexity threshold needs to be adjusted smaller to increase the monitoring intensity. If the user risk coefficient is less than the first risk threshold, the complexity threshold needs to be adjusted larger to decrease the monitoring intensity. If the user risk coefficient is greater than or equal to the first risk threshold and less than or equal to the second risk threshold, the complexity threshold does not need to be adjusted. The specific adjustment algorithm is as follows: , wherein, is the corrected complexity threshold, is the initial complexity threshold, is the user risk coefficient, is the first risk threshold, is the second risk threshold.

[0136] The smart glasses are controlled to run according to the comparison result of the overall complexity and the corrected complexity threshold, including the following steps:

[0137] The obtained overall complexity is compared with the modified complexity threshold value. If the overall complexity is less than or equal to the modified complexity threshold value, the operation of the smart glasses is not intervened. If the overall complexity is greater than the modified complexity threshold value, the operation of the smart glasses is intervened, and the control includes: controlling the smart glasses to display off, and then turning on the display for a period of time (for example, 2s) every interval of a period of time (for example, 10s), or continuously detecting the overall complexity of the current scene, and combining the navigation information. If the overall complexity is continuously greater than the modified complexity threshold value, and the current scene is straight, the smart glasses display is continuously turned off. If the overall complexity is continuously greater than the modified complexity threshold value, and the current scene needs to turn, the smart glasses display is turned on.

[0138] Embodiment 3: The control system based on a wearable smart device described in this embodiment includes a collection module, an analysis module, a calculation module, a correction module, and a control module.

[0139] The collection module is started when the smart glasses are turned on and runs, and acquires multiple frames of real-time images through a sensing device on the smart glasses. Video stream data is acquired in a monitoring time period. The multiple frames of real-time images and the video stream data are sent to the analysis module.

[0140] The analysis module analyzes static complex factors of the current scene based on a scene recognition model, and analyzes dynamic complex factors of the current scene based on the video stream data. The static complex factors and the dynamic complex factors are sent to the calculation module.

[0141] The calculation module calculates the overall complexity of the current scene by substituting the static complex factors and the dynamic complex factors into a fusion model. The overall complexity is sent to the control module.

[0142] The correction module dynamically adjusts the complexity threshold value to obtain a modified complexity threshold value in combination with user motion posture data. The modified complexity threshold value is sent to the control module.

[0143] The control module controls the operation of the smart glasses according to the comparison result of the overall complexity and the modified complexity threshold value.

[0144] The above formulas are dimensionless values calculated. The formulas are obtained by software simulation of a large amount of data to obtain a formula of the nearest real situation. The preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0145] It should be understood that the term "and / or" in this document is only used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after it, but it can also represent an "and / or" relationship. The specific meaning can be understood according to the context before and after it.

[0146] It should be understood that the size of the sequence number of each process described above in various embodiments of the present application does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0147] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0148] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited to this. Any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A control method based on a wearable smart device, the method comprising: The control method comprises the following steps: The acquisition end is started when the intelligent glasses are turned on, and a plurality of real-time images are acquired through a sensing device on the intelligent glasses; Based on a scene recognition model, static complexity factors of the current scene are analyzed, and after video stream data is acquired in a monitoring time period, dynamic complexity factors of the current scene are analyzed, and the processing end substitutes the static complexity factors and the dynamic complexity factors into a fusion model to calculate the overall complexity of the current scene; The complexity threshold is dynamically adjusted in combination with the motion posture data of the user to obtain a corrected complexity threshold, and the intelligent glasses are controlled according to the comparison result of the overall complexity and the corrected complexity threshold; The processing end substitutes the static complexity factors and the dynamic complexity factors into a fusion model to calculate the overall complexity of the current scene, and the expression of the fusion model is: wherein is the overall complexity, is the dynamic complexity factor, is the static complexity factor, , is the adjustment coefficient, and , are all greater than 0; The complexity threshold is dynamically adjusted in combination with the motion posture data of the user to obtain a corrected complexity threshold, and the intelligent glasses are controlled according to the comparison result of the overall complexity and the corrected complexity threshold; The motion posture data of the user is acquired, the motion posture data of the user includes a user risk coefficient, the user risk coefficient is compared with a preset first risk threshold and a second risk threshold, if the user risk coefficient is greater than the second risk threshold, the complexity threshold needs to be reduced to increase the monitoring intensity, if the user risk coefficient is less than the first risk threshold, the complexity threshold needs to be increased to reduce the monitoring intensity, if the user risk coefficient is greater than or equal to the first risk threshold and the user risk coefficient is less than or equal to the second risk threshold, the complexity threshold does not need to be adjusted, and the adjustment algorithm is: wherein is a revised complexity threshold, is an initial complexity threshold, is a user risk factor, is a first risk threshold, is a second risk threshold. 2.The control method based on a wearable smart device of claim 1, wherein: The intelligent glasses are controlled according to the comparison result of the overall complexity and the corrected complexity threshold, and the control comprises the following steps: The acquired overall complexity is compared with the corrected complexity threshold, if the overall complexity is less than or equal to the corrected complexity threshold, the intelligent glasses are not intervened and controlled to run, if the overall complexity is greater than the corrected complexity threshold, the intelligent glasses are intervened and controlled to run, and the control includes controlling the intelligent glasses to display off. 3.The control method based on a wearable smart device of claim 2, wherein: The motion posture data of the user includes a user risk coefficient, and the calculation logic of the user risk coefficient is that the user motion speed and the user turning frequency are acquired, the user motion speed and the user turning frequency are normalized to map the value range of the user motion speed and the user turning frequency to [0, 1], the user motion speed normalized value and the user turning frequency normalized value are acquired, and the user risk coefficient is obtained by summing the user motion speed normalized value and the user turning frequency normalized value.

4. The control method based on a wearable smart device according to claim 3, characterized in that: After the video stream data is acquired in the monitoring time period, the dynamic complexity factors of the current scene are analyzed, and the control comprises the following steps: After the video stream data is acquired, a frame difference method is used to identify dynamic objects in the scene, the motion of the objects is detected by comparing the changes between continuous frames, the change amount of each frame is calculated and the motion area of the dynamic objects is determined, and an object tracking algorithm is used to track the motion path of each dynamic object in the video; Based on the position changes of the objects in the continuous frames, the motion speed of each object is calculated, the number of times that each object appears in a time window is recorded, the object appearance frequency in the scene is calculated, and the object complexity factor is calculated according to the object motion speed and the object appearance frequency.

5. The control method based on a wearable smart device according to claim 4, characterized in that: The object complexity factor calculation logic is: normalizing the object movement speed and the object appearance frequency, summing the normalized object movement speed and the object appearance frequency to obtain a dynamic complexity factor. 6.The control method based on a wearable smart device of claim 5, wherein: The acquisition end is started when the smart glasses are powered on, and multiple frames of real-time images are acquired through the sensing devices on the smart glasses. The static complexity factor of the current scene is analyzed based on a scene recognition model, including the following steps: Real-time acquisition of multiple frames of image data in the current environment through the built-in camera of the smart glasses, removal of sensor noise using an image denoising algorithm, and enhancement of image details using image enhancement techniques; Object detection and scene segmentation using a scene recognition model to identify and label different elements in the image; According to the identified scene elements, combined with the spatial distribution information of the scene, the static complexity factor is calculated.

7. The control method based on a wearable smart device according to claim 6, characterized in that: The static complexity factor is quantified in the following dimensions: Spatial density: including the density of objects on the road, the number and arrangement of buildings; Object type and layout: including the types of objects in the environment, including vehicles, pedestrians, traffic signs, and layout complexity, including intersections and traffic facilities; Obstacles and visual range: including the presence of obstructions in the environment and the wearer's field of view.

8. A control system based on a wearable smart device, for implementing the control method according to any one of claims 1-7, characterized in that: The system includes an acquisition module, an analysis module, a calculation module, a correction module, and a control module: The acquisition module is started when the smart glasses are powered on, and multiple frames of real-time images are acquired through the sensing devices on the smart glasses. Video stream data is acquired during the monitoring period, and the multiple frames of real-time images and the video stream data are sent to the analysis module; The analysis module analyzes the static complexity factor of the current scene based on a scene recognition model, and analyzes the dynamic complexity factor of the current scene based on the video stream data. The static complexity factor and the dynamic complexity factor are sent to the calculation module; The calculation module calculates the overall complexity of the current scene by substituting the static complexity factor and the dynamic complexity factor into a fusion model, and sends the overall complexity to the control module; The correction module dynamically adjusts the complexity threshold value to obtain a corrected complexity threshold value based on the user's motion posture data, and sends the corrected complexity threshold value to the control module; The control module controls the operation of the smart glasses based on the comparison result of the overall complexity and the corrected complexity threshold value.

Images