Control method and device of smart glasses, smart glasses and medium

Abstract

This application provides a control method, device, smart glasses, and medium for smart glasses. It uses a camera device mounted on the smart glasses frame to capture environmental images of the wearer, determine the presence of dangerous targets in the images, and if dangerous targets are present, determine their location information and confidence level. Based on the location information and / or confidence level, it determines the corresponding danger level and performs security actions corresponding to that level. This method requires no active operation from the wearer. It uses the smart glasses as a moving sensing carrier, combined with the camera device on the frame, to form a continuous image input of the wearer's actual activity environment. This allows danger identification, level determination, and security response to be continuously implemented around the same usage scenario, providing safety protection services to the wearer before danger occurs and offering a more timely and targeted safety foundation for traffic and personal risks during travel.

Description

Technical Field

[0001] This application relates to the field of smart glasses technology, and in particular to a control method, device, smart glasses and medium for smart glasses. Background Technology

[0002] In the modern urban traffic environment, individuals face increasingly complex safety risks during their daily commutes. On the one hand, the high speed of motor vehicles, violations (such as running red lights, driving against traffic, and dangerous driving), and dangerous behaviors of pedestrians (such as jaywalking and running red lights) may lead to traffic accidents; on the other hand, potential threats from individuals in public places (such as suspicious persons following for extended periods, people with obscured faces approaching abnormally, fights, etc.) may cause personal safety issues.

[0003] Current personal security measures typically rely on fixed security cameras and security applications installed on smart devices. Fixed security cameras, due to their fixed installation locations, cannot flexibly adjust their monitoring coverage and cannot provide dynamic protection by tracking an individual's movement. Furthermore, their core function is video recording and archiving, only usable for tracing and verifying incidents after they occur; they cannot provide real-time warnings before or during danger, resulting in extremely poor timeliness. Security applications installed on smart devices heavily depend on manual user intervention to trigger warnings and protective functions. When users are focused on walking, distracted, or unaware of potential threats, the devices cannot autonomously identify risks and activate protective measures, easily missing the optimal time for avoidance.

[0004] Therefore, the current problem to be solved is how to achieve proactive early warning of security protection without requiring active user operation, so as to remind users before danger occurs. Summary of the Invention

[0005] This application provides a control method, device, smart glasses, and medium for smart glasses, in order to solve the shortcomings of existing technologies that cannot provide proactive early warning for safety protection before danger occurs.

[0006] In a first aspect, this application provides a control method for smart glasses, applied to smart glasses, wherein a camera device is disposed on the frame of the smart glasses for acquiring environmental images of the wearer, and the method includes:

[0007] Acquire an image of the wearer's environment and determine if any dangerous targets are present in the image.

[0008] In the case of dangerous targets in environmental images, determine the location information and confidence level of the dangerous targets;

[0009] Based on location information and / or confidence level, determine the corresponding hazard level;

[0010] Perform security actions corresponding to the hazard level.

[0011] In one possible implementation, the corresponding hazard level is determined based on location information and / or confidence level, including:

[0012] Identify the hazard type of the hazardous target;

[0013] Based on location information, determine the orientation and distance of the dangerous target relative to the wearer;

[0014] The hazard level is determined based on the type of hazard, confidence level, and the orientation and / or distance of the hazard target relative to the wearer.

[0015] In one possible implementation, the hazard types include: Type 1, Type 2, and Type 3. The hazard level is determined based on the hazard type, confidence level, and the orientation and / or distance of the hazardous target relative to the wearer, including:

[0016] If a dangerous target meets the first condition, the danger level is determined to be the first danger level. The first condition includes at least one of the following: the danger type is the first type, the distance is within the first distance interval, and the confidence level is less than the preset confidence level.

[0017] If the dangerous target meets the second condition, the danger level is determined to be the second danger level. The second condition includes at least one of the following: the danger type is the second type, the distance is within the second distance interval, the dangerous target is in front of the wearer, and the second distance interval is smaller than the first distance interval.

[0018] If a dangerous target meets the third condition, the danger level is determined to be the third danger level. The third condition includes at least one of the following: the danger type is the third type, the distance is within the third distance interval, the confidence level is greater than the preset confidence level, the third distance interval is smaller than the second distance interval, and the threat level of the first type is less than the threat level of the second type, which is less than the threat level of the third type.

[0019] In one possible implementation, the smart glasses also include: a bone conduction speaker and a vibrator, which perform security actions corresponding to the level of danger, including:

[0020] Identify hazardous targets and the corresponding alert messages based on their hazard levels;

[0021] When the danger level is the highest level, the vibrator is controlled to vibrate at the first vibration frequency to display the warning message in augmented reality.

[0022] When the danger level is the second danger level, the vibrator is controlled to vibrate at the second vibration frequency to display the reminder message in augmented reality and to play the reminder message through the bone conduction speaker. The second vibration frequency is higher than the first vibration frequency.

[0023] When the danger level is the third danger level, the vibrator is controlled to vibrate at the third vibration frequency, the bone conduction speaker is controlled to play a reminder message, and a danger message is sent to the user terminal corresponding to the smart glasses. The third vibration frequency is higher than the second vibration frequency.

[0024] In one possible implementation, the environmental image includes: multiple consecutive image frames; determining whether a hazardous target exists in the environmental image includes:

[0025] Identify multiple candidate objects in the starting image frame of multiple consecutive image frames;

[0026] For any one of multiple candidate objects, the motion trajectory of the candidate object is determined based on multiple consecutive image frames;

[0027] Based on the motion trajectory, determine the behavioral state of the candidate object;

[0028] If the behavior status indicates that the candidate object is abnormal, the candidate object is identified as a dangerous target.

[0029] In one possible implementation, the environmental image includes: multiple consecutive image frames; determining whether a hazardous target exists in the environmental image includes:

[0030] An image recognition model is used to identify multiple consecutive image frames to determine whether there are dangerous targets in the environmental image.

[0031] Secondly, this application provides a control device for smart glasses, which is applied to smart glasses. A camera device is installed on the frame of the smart glasses to capture environmental images of the wearer. The device includes:

[0032] The acquisition module is used to acquire images of the wearer's environment.

[0033] The processing module is used to determine whether there are dangerous targets in the environmental image; if there are dangerous targets in the environmental image, to determine the location information and confidence level of the dangerous targets; and to determine the corresponding hazard level based on the location information and / or confidence level.

[0034] The control module is used to execute security actions corresponding to the hazard level.

[0035] In one possible implementation, the processing module is used to determine the hazard type of the hazardous target; determine the orientation and distance of the hazardous target relative to the wearer based on location information; and determine the hazard level based on the hazard type, confidence level, orientation and / or distance of the hazardous target relative to the wearer.

[0036] In one possible implementation, the hazard types include: a first type, a second type, and a third type. The processing module is used to determine the hazard level as the first hazard level when the hazard target meets a first condition. The first condition includes at least one of the following: the hazard type is the first type, the distance is within a first distance interval, and the confidence level is less than a preset confidence level.

[0037] If the dangerous target meets the second condition, the danger level is determined to be the second danger level. The second condition includes at least one of the following: the danger type is the second type, the distance is within the second distance interval, the dangerous target is in front of the wearer, and the second distance interval is smaller than the first distance interval.

[0038] If a dangerous target meets the third condition, the danger level is determined to be the third danger level. The third condition includes at least one of the following: the danger type is the third type, the distance is within the third distance interval, the confidence level is greater than the preset confidence level, the third distance interval is smaller than the second distance interval, and the threat level of the first type is less than the threat level of the second type, which is less than the threat level of the third type.

[0039] In one possible implementation, the smart glasses also include: a bone conduction speaker and a vibrator, and a processing module, which is also used to determine dangerous targets and warning messages corresponding to the level of danger;

[0040] The control module is used to control the vibrator to vibrate at a first vibration frequency when the hazard level is the highest level, and to display the reminder message in augmented reality.

[0041] When the danger level is the second danger level, the vibrator is controlled to vibrate at the second vibration frequency to display the reminder message in augmented reality and to play the reminder message through the bone conduction speaker. The second vibration frequency is higher than the first vibration frequency.

[0042] When the danger level is the third danger level, the vibrator is controlled to vibrate at the third vibration frequency, the bone conduction speaker is controlled to play a reminder message, and a danger message is sent to the user terminal corresponding to the smart glasses. The third vibration frequency is higher than the second vibration frequency.

[0043] In one possible implementation, the environmental image includes: multiple consecutive image frames; a processing module for identifying multiple candidate objects in the starting image frame of the multiple consecutive image frames; for any one of the multiple candidate objects, determining the motion trajectory of the candidate object based on the multiple consecutive image frames; determining the behavioral state of the candidate object based on the motion trajectory; and determining the candidate object as a dangerous target if the behavioral state indicates that the candidate object is abnormal.

[0044] In one possible implementation, the environmental image includes: multiple consecutive image frames, and a processing module for recognizing and processing the multiple consecutive image frames using an image recognition model to determine whether there are dangerous targets in the environmental image.

[0045] Thirdly, this application provides a smart glasses, including: a frame, temples, lenses, a camera device, and a processor;

[0046] The camera device is mounted on the frame of the glasses to capture images of the wearer's surroundings;

[0047] The processor is mounted on the temple for implementing the methods shown in the first aspect and / or various possible implementations of the first aspect.

[0048] In one possible implementation, the number of camera devices is one or three;

[0049] When there is only one camera device, the camera device is set on the front of the frame to capture the wearer's first-person perspective image;

[0050] When there are three cameras, the cameras are respectively located on the front of the frame, on the side near the left temple, and on the side near the right temple.

[0051] Fourthly, this application provides a smart glasses, including: a memory and a processor;

[0052] The memory stores the instructions executed by the computer.

[0053] The processor executes computer execution instructions stored in the memory to implement the method shown in the first aspect and / or various possible implementations of the first aspect above.

[0054] Fifthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods shown in the first aspect and / or various possible implementations of the first aspect.

[0055] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the methods shown in the first aspect and / or various possible implementations of the first aspect.

[0056] The control method, device, smart glasses, and medium provided in this application for smart glasses acquire environmental images of the wearer through a camera device mounted on the frame of the smart glasses. The method determines whether dangerous targets exist in the environmental images, and if so, determines the location information and confidence level of the dangerous targets. Based on the location information and / or confidence level, the corresponding danger level is determined, and security actions corresponding to the danger level are performed. This method requires no active operation from the wearer. It uses the smart glasses as a moving sensing carrier, combined with the camera device on the frame, to form a continuous image input facing the wearer's actual activity environment. This allows danger identification, level determination, and security response to be continuously carried out around the same usage scenario, providing safety protection services to the wearer before danger occurs, and providing a more timely and targeted safety protection foundation for traffic and personal risks during travel. Attached Figure Description

[0057] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0058] Figure 1 A schematic diagram of the structure of smart glasses provided in this application embodiment. Figure 1 ;

[0059] Figure 2 A flowchart illustrating a control method for smart glasses provided in an embodiment of this application;

[0060] Figure 3 This is a schematic diagram of the structure of a control device for smart glasses provided in an embodiment of this application;

[0061] Figure 4 A schematic diagram of the structure of smart glasses provided in this application embodiment. Figure 2 .

[0062] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0065] In this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0066] In the modern urban traffic environment, individuals face increasingly complex safety risks during their daily commutes. On the one hand, the high speed of motor vehicles, violations (such as running red lights, driving against traffic, and dangerous driving), and dangerous behaviors of pedestrians (such as jaywalking and running red lights) may lead to traffic accidents; on the other hand, potential threats from individuals in public places (such as suspicious persons following for extended periods, people with obscured faces approaching abnormally, fights, etc.) may cause personal safety issues.

[0067] Existing personal security measures typically include fixed security systems, mobile security applications, and wearable alarm devices.

[0068] Fixed security systems typically collect video footage by deploying cameras at intersections, streets, or public areas, which is then reviewed and analyzed by a back-end system or manually. Because the cameras are fixed in location, their monitoring coverage cannot be flexibly adjusted, and they cannot provide dynamic protection by tracking individual movement. Furthermore, their core function is video recording and archiving for tracing and verification after accidents or incidents, failing to provide real-time warnings before or during danger, resulting in extremely poor timeliness of protection.

[0069] Mobile security applications typically rely on the phone's camera, positioning module, or manual trigger button to achieve risk reporting, location sharing, or simple detection. However, such solutions usually require users to actively turn on, click, or view the screen. When crossing the road, walking with their heads down, or when danger suddenly occurs, users often find it difficult to operate in time, and the device cannot autonomously identify risks and activate protection, resulting in missing the best time to avoid danger.

[0070] Wearable alarm devices focus on emergency assistance, requiring users to actively press buttons after noticing danger, and lack the ability to detect potential threats in advance.

[0071] The common shortcoming of the above solutions is that they are difficult to continuously and automatically identify dangerous targets in the environment during individual movement. They are prone to problems such as delayed alerts, generalized warnings, or inaccurate alarms, thus failing to meet the actual needs for real-time proactive perception and effective early warning in complex scenarios.

[0072] Therefore, the current problem to be solved is how to achieve proactive early warning for security protection without requiring active user intervention, so as to alert users before danger occurs.

[0073] To address the aforementioned issues, this application provides a control method for smart glasses. This method uses a camera device mounted on the smart glasses frame to capture environmental images of the wearer. Based on these images, it determines whether the wearer is currently in danger. When danger is determined, it determines the corresponding danger level based on the location information and confidence level of the dangerous target; then, it executes security actions corresponding to that danger level. This method requires no active operation from the wearer. It uses the smart glasses as a moving sensing carrier, combined with the camera device on the frame, to form a continuous image input of the wearer's actual activity environment. This allows danger identification, level determination, and security response to be continuously implemented around the same usage scenario, providing safety protection services to the wearer before danger occurs, and providing a more timely and targeted safety foundation for traffic and personal risks during travel.

[0074] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0075] First, the structure of the smart glasses provided in the embodiments of this application will be explained. Figure 1 A schematic diagram of the structure of smart glasses provided in this application embodiment. Figure 1 .

[0076] like Figure 1As shown, the smart glasses include: a frame 101, a left temple 103, a right temple 102, a lens 104, and a camera device 105.

[0077] Among them, the frame 101 refers to the frame component that serves as the main supporting structure of the smart glasses, which is used to support the lenses 104 and maintain the stability of the smart glasses wearing posture.

[0078] The frame 101 can be a full-rimmed, semi-rimless, or rimless design, or it can be designed as a modular, detachable frame. The frame 101 can be made of metal, engineering plastic, or composite materials, or it can use a combination of aluminum alloy and polymer coating to balance structural strength, lightweight, and wearing comfort.

[0079] The camera device 105 is mounted on the frame 101 and is used to capture environmental images of the wearer. The camera device 105 can continuously capture environmental images of the area in front of, to the side of, or around the wearer.

[0080] Environmental images are used to carry information about the scene around the wearer and are the basic data source for subsequent identification of dangerous targets, location extraction, and level determination.

[0081] Dangerous targets are potential threats identified in environmental images. These may include approaching motor vehicles, non-motor vehicles, individuals moving abnormally fast, road obstructions, targets that suddenly enter the field of vision under low visibility conditions at night, and other objects that may affect the wearer's safety.

[0082] In one possible implementation, the number of camera devices 105 can be one or more.

[0083] When there is only one camera device 105, the camera device 105 can be set on the front of the frame 101 to capture the wearer's first-person perspective image.

[0084] Understandably, the camera device 105 on the front of the frame 101 can be positioned, for example, at the front edge of the frame 101, near the left temple 103 or near the right temple 102, or at any position that can continuously capture environmental images of the area in front of, to the side of, or around the wearer in a near-first-person perspective. This application does not limit the placement of the camera device 105.

[0085] The camera device 105 can be installed in an embedded, exposed, or semi-embedded manner to balance field coverage, structural compactness, and impact resistance. The lens of the camera device 105 can be oriented in a manner that is basically consistent with the wearer's line of sight, or it can have preset pitch and yaw angles to extend the effective monitoring range.

[0086] When there are multiple camera devices 105, the camera devices 105 can be respectively located on the front of the frame 101, on the side near the left temple 103 facing the external environment, and / or on the side near the right temple 102 facing the external environment.

[0087] The camera 105 positioned on the front is used to continuously capture environmental images of the area in front of, to the side of, or around the wearer from a near first-person perspective. The camera 105 positioned on the side facing the external environment is used to capture environmental images of the wearer's side.

[0088] In one possible implementation, the camera device 105 may be any one of a micro CMOS (Complementary Metal-Oxide-Semiconductor) camera, a wide-angle camera, or a binocular / multi-lens micro camera module. The external dimensions of the camera device 105 are generally smaller than the front width of the frame 101 so as to be embedded without significantly changing the overall appearance. Its field of view and installation height should meet the image coverage requirements of the wearer in the travel scenario for the road ahead, mixed pedestrian and vehicle areas, and close-range targets to the side.

[0089] The right temple 102 and the left temple 103 are extension components that connect to both sides of the frame 101 and are used to support and fix the frame 101 near the wearer's ears, so as to keep the frame 101 stably on the wearer's head.

[0090] In one possible implementation, a hollow cavity may be provided inside the right temple 102 and / or the left temple 103 to accommodate a processor (not shown), a storage module, a communication module, and necessary wires or flexible circuit boards. The right temple 102 and the left temple 103 may be connected to the frame 101 via hinges, connectors, or a one-piece structure.

[0091] Lens 104 refers to an optical component installed inside the frame 101 and located in front of the wearer's line of sight. Its function is to provide vision correction, perspective observation, or display of obstruction, and to cooperate with smart glasses to complete environmental observation without significantly affecting the wearer's normal field of vision.

[0092] In terms of physical position, the lens 104 is fixed to the lens mounting area at the front of the frame 101, forming a stable snap-fit, embedding, or pressing relationship with the frame 101. The light-transmitting area of ​​the lens 104 is located in the area directly covered by the wearer's line of sight. If necessary, a partial coating or transparent display layer can also be used to take into account visual comfort.

[0093] In one possible implementation, a processor may be installed in the central control cavity of the right temple 102 and / or the left temple 103 of the smart glasses to receive environmental images collected by the camera device 105 and perform dangerous target detection, location information extraction, confidence assessment and danger level determination within the glasses, thereby driving corresponding security actions.

[0094] Physically, the processor is housed in the storage space inside the temple and is electrically connected to the power module, storage module, and communication module. The processor can be connected to the camera device 105, the prompting module, or an external interactive interface via a flexible circuit board to achieve stable data transmission and command control.

[0095] The processor can be any one or a combination of microcontrollers, edge AI (Augmented Reality) chips, or system-on-chip (SoC). It can also be implemented using discrete chip solutions, modular computing board solutions, or hardware-software co-processing solutions. The chip carrier can be a multi-layer printed circuit board, and the package can be a plastic package or a metal shielded package.

[0096] Understandably, the processor can integrate an image recognition model, which could be, for example, a lightweight model extracted from a large detection model through knowledge distillation technology and optimized for traffic hazards and personnel threats, and can run in real time on the processor (such as an edge AI chip).

[0097] Figure 2 This is a flowchart illustrating a control method for smart glasses provided in an embodiment of this application. The executing entity in this embodiment can, for example, be the smart glasses shown in the above embodiment. Figure 2 As shown, the control method for the smart glasses includes:

[0098] S201. Obtain an environmental image of the wearer and determine whether there are dangerous targets in the environmental image.

[0099] Among them, the environmental image is used to carry information about the scene around the wearer and is the basic data source for subsequent dangerous target identification, location extraction and level determination. It can be, for example, multiple consecutive image frames.

[0100] Dangerous targets are potential threats identified in environmental images. These may include approaching motor vehicles, non-motor vehicles, individuals moving abnormally fast, road obstructions, targets that suddenly enter the field of vision under low visibility conditions at night, and other objects that may affect the wearer's safety.

[0101] In this step, the camera device 105 can continuously output image frames according to a preset sampling period, or it can output a short video stream, and then the processor inside the smart glasses extracts the image to be analyzed from the video stream.

[0102] To adapt to changes in lighting, image jitter, and rapid scene transitions during travel, preprocessing can be performed on the acquired environmental images. Preprocessing may include image denoising, brightness normalization, dynamic range adjustment, blur suppression, distortion correction, inter-frame stabilization, and region of interest cropping. In nighttime environments, low-light enhancement processing can be further performed, and in backlit environments, local contrast compensation processing can be performed to ensure the usability of the input data for subsequent recognition models.

[0103] In one possible implementation, when the environmental image comprises multiple consecutive image frames, multiple candidate objects in the starting image frame of the multiple consecutive image frames can be identified first; then, for any one of the multiple candidate objects, the motion trajectory of the candidate object can be determined based on the multiple consecutive image frames.

[0104] Then, based on the movement trajectory, the behavioral state of the candidate object is determined; if the behavioral state indicates that the candidate object is abnormal, the candidate object is determined to be a dangerous target.

[0105] Among them, multiple consecutive image frames are used to provide the basis for temporal changes, the starting image frame is used to determine the initial candidate object, multiple candidate objects are used as the target set for subsequent tracking analysis, the motion trajectory is used to reflect the positional changes of the candidate object in consecutive image frames, and the behavioral state is used to characterize whether the candidate object has abnormal motion features.

[0106] In real-world scenarios, the camera device 105 can collect environmental information about the wearer's current environment in real time and perform image recognition processing on image frames formed at adjacent moments.

[0107] For example, candidate objects can be extracted from the initial image frame. These candidate objects can be pedestrians, vehicles, non-motorized vehicles, or other objects that may affect the wearer's safety.

[0108] For any candidate object, its position in subsequent image frames is matched and associated. The motion trajectory of the candidate object is formed by using bounding box center point displacement, key point tracking, or feature vector association. The motion trajectory can be represented as a coordinate sequence that changes over time.

[0109] After obtaining the movement trajectory of the candidate, the candidate's behavioral state can be further analyzed. When the trajectory shows continuous following of the wearer, rapid approach to the wearer, lingering around the wearer, suddenly crossing the direction of movement, or abnormal acceleration, it can be identified as an abnormal behavioral state.

[0110] To improve the stability of the judgment, a comprehensive analysis can be conducted by combining the trajectory change rate, acceleration, magnitude of directional changes, and the trend of changes in the relative position with the wearer. In cases where the behavioral state indications are abnormal, the candidate object is identified as a dangerous target.

[0111] This step expands single-frame recognition to continuous dynamic recognition by first identifying initial candidate objects and then judging behavioral anomalies based on their trajectories. Compared to methods that rely solely on static images, this step can more accurately distinguish between ordinary passing targets and risky targets, reducing the probability of false alarms and false negatives, and improving the real-time proactive warning capabilities of smart glasses in travel scenarios.

[0112] In one possible implementation, when determining whether a dangerous target exists, an image recognition model can be used to identify multiple consecutive image frames to determine whether a dangerous target exists in the environmental image.

[0113] Among them, the image recognition model is used to perform target detection, target classification and target association analysis on continuously acquired image frames, and then output the identification result of whether there are dangerous targets in the environment.

[0114] The image recognition model is deployed in the local processing unit of the smart glasses. It can use convolutional neural networks, feature pyramid-based detection networks, or recognition networks containing temporal feature fusion structures to identify vehicles, pedestrians, cyclists, or other abnormally approaching targets in adjacent image frames, and output the corresponding category labels, target boxes, and hazard judgment results.

[0115] In practical applications, the model can also combine the feature differences between consecutive frames to perform stability verification on candidate targets, so as to reduce the probability of false detection caused by single-frame blurring, occlusion or illumination changes. This application does not limit this aspect.

[0116] In this step, the acquired continuous image frame sequence can be input into the image recognition model. The image recognition model performs joint analysis on the correlation information between each frame and adjacent frames to determine whether there are targets with collision risk, proximity risk or abnormal behavior characteristics in the current environment.

[0117] By deploying the image recognition model within the local processing unit of the smart glasses, the recognition and reasoning process is completed entirely within the smart glasses themselves, eliminating the need to upload the collected image data to the cloud. This ensures that the smart glasses can function normally even in environments without a network connection, and fundamentally avoids the privacy risks associated with image data leakage. Furthermore, because recognition is based on multiple consecutive image frames, it reduces false positives and false negatives caused by single-frame recognition, improving the accuracy and real-time performance of dangerous target detection and enhancing the ability to proactively detect potential threats in travel scenarios.

[0118] In one possible implementation, in order to reduce false alarms caused by occasional false detections, temporal consistency analysis can be performed by combining the detection results of multiple consecutive frames. When the same target satisfies the conditions of positional continuity, category consistency, or motion trend consistency in several consecutive frames, the target is determined to be a real dangerous target. If a candidate target only appears briefly in a single frame and lacks spatial continuity, it is filtered out.

[0119] For complex scenarios in areas where pedestrians and vehicles mix, scene semantic segmentation can be performed first to identify pedestrian walkways, motor vehicle lanes, non-motor vehicle lanes, intersection areas, and zebra crossing areas. Then, the accuracy of identifying dangerous targets can be enhanced based on the relative relationship between the target and these areas. For example, when a motor vehicle is identified as being located in a motor vehicle lane and moving in the direction the wearer may enter, it can be included in the set of dangerous targets; when the target is far from the wearer's direction of movement and there is no tendency to intersect, the subsequent high-level warning process will not be triggered.

[0120] In this step, when there are no dangerous targets in the environmental image, the smart glasses maintain the monitoring state and continue to collect subsequent environmental images in a loop; when there are dangerous targets, a target data record can be generated that includes at least the target identifier, detection category, initial location parameters, and initial recognition score.

[0121] S202. When there are dangerous targets in the environmental image, determine the location information and confidence level of the dangerous targets.

[0122] Location information is used to characterize the spatial distribution of dangerous targets in environmental images; confidence level is used to characterize the reliability of the identification result of the dangerous target, so as to avoid misjudgment caused by relying on a single identification result.

[0123] Location information may include, for example, the area coordinates, boundary range, center point position, pixel area, relative field of view position, and relative orientation information of the dangerous target in the environmental image, such as front, left front, right front, side, or rear.

[0124] In this step, the two-dimensional region coordinates of the dangerous target can be determined based on the environmental image. The region coordinates can include the upper left and lower right pixel coordinates, or they can be represented by the center point coordinates and width and height parameters.

[0125] Based on the offset between the image center position and the target center position, it can be determined whether the dangerous target is located in the central or peripheral area of ​​the wearer's field of vision; combined with the field of view parameters of the camera device, pixel offset can be mapped to angle offset to obtain relative direction information that is more consistent with the actual spatial orientation.

[0126] If smart glasses are equipped with an inertial measurement unit, they can also combine head posture angles with image coordinates to maintain consistency in orientation judgment as the wearer turns, lowers, or raises their head.

[0127] For determining the confidence level, for example, the target classification score output by the image recognition model in the above steps can be directly used as the initial confidence level; in another possible embodiment, the initial confidence level can be fused with the number of stable occurrences of the target in consecutive frames, the bounding box overlap, the consistency category ratio, and the motion rationality to obtain a comprehensive confidence level.

[0128] The fusion method can use weighted calculation. For example, the comprehensive confidence C can be expressed as C = α × C_recognition + β × C_temporal + γ × C_motion, where C_recognition represents the recognition score of the model in the current frame, C_temporal represents the stability index of the target in consecutive frames, C_motion represents the degree of matching between the target's motion trajectory and the real scene pattern, and α, β, and γ are weight parameters and satisfy α + β + γ = 1.

[0129] By adjusting the various weighting parameters, a balance can be achieved between real-time performance and stability. For example, in road crossing scenarios, to reduce instantaneous false detections, the temporal stability weight can be appropriately increased; in scenarios involving sudden close-range targets, to avoid missed detections, the current frame recognition score weight can be appropriately increased. If multiple hazardous targets exist in the environmental image, their location information and confidence level are determined for each hazardous target, and a multi-target list is established. This multi-target list can include target number, category, location coordinates, orientation label, distance parameter, velocity trend parameter, and confidence level parameter for subsequent hazard level calculation.

[0130] This step, by further extracting the location information and confidence level of the dangerous target after identification, can expand the rough judgment of "whether the target exists" into a detailed description that takes into account both spatial relationship and identification reliability. This provides a more sufficient basis for hazard level determination, reduces the probability of generalized warnings and false triggers, and improves the practicality and stability of the system in dynamic and complex scenarios.

[0131] S203. Determine the corresponding hazard level based on location information and / or confidence level.

[0132] The hazard level is used to classify dangerous targets so that different levels of risk correspond to different intensities of security response.

[0133] Since location information can reflect the spatial proximity, orientation, and potential intersection between the dangerous target and the wearer, and confidence level can reflect the reliability of the identification results, the use of either alone or in combination can constitute the main basis for judging the risk level.

[0134] The hazard level can be set in a multi-level structure, such as three levels: low hazard, medium hazard and high hazard, or it can be further refined into a hierarchical structure of one to five levels. The higher the level value, the more urgent the risk and the more necessary it is to take immediate warning or action.

[0135] In this step, a hazard level rule base can be pre-established. This rule base can record the hazard level mapping relationship corresponding to different location states and different confidence ranges.

[0136] For example, when a dangerous target is located at the edge of the field of vision and is far away with no obvious approach trend, it can be judged as a low-risk level; when a dangerous target is located near the center of the field of vision and has an intersection trend with the wearer's direction of travel or the distance is continuously decreasing, it can be judged as a high-risk level; when the target is close but the confidence level is insufficient, it can be temporarily maintained at an intermediate level and subsequent frames can be observed; when the target is in a critical area and the confidence level is higher than the preset threshold, it can be directly upgraded to a high-risk level.

[0137] For example, a location risk score can be calculated first based on location information, then a confidence correction score can be calculated based on confidence level, and finally a comprehensive risk score R can be generated.

[0138] The comprehensive risk score can be expressed as R = k1 × P position + k2 × P proximity + k3 × C, where P position represents the orientation risk score of the target in the image, P proximity represents the risk score corresponding to the target distance and approach speed, C represents the confidence level, and k1, k2, and k3 are proportional coefficients.

[0139] Then, based on a preset range, R is mapped to different risk levels. For example, when R is below a first threshold, it is determined to be a low-risk level; when R is between the first and second thresholds, it is determined to be a medium-risk level; and when R is above the second threshold, it is determined to be a high-risk level. This parameterized implementation method is beneficial for adjusting the strategy in different application environments. For example, in densely populated areas with mixed pedestrian and vehicle traffic, the location risk weight can be increased, and in low-light environments at night, the confidence threshold constraint can be increased to suppress misjudgments caused by low-quality images.

[0140] In one possible implementation, the hazard type of the dangerous target can be determined first, and the orientation and distance of the dangerous target relative to the wearer can be determined based on location information; then, the hazard level can be determined based on the hazard type, confidence level, orientation and / or distance of the dangerous target relative to the wearer.

[0141] Among them, the hazard type is used to characterize the category attribute to which the hazardous target belongs, so as to distinguish vehicles, non-motorized vehicles, pedestrians, obstacles or other objects that may cause risks.

[0142] Orientation is used to characterize the directional relationship between a dangerous target and the wearer, while distance is used to characterize the proximity of a dangerous target to the wearer. Both can be calculated from the position of the dangerous target in the environmental image combined with the wearer's orientation.

[0143] In this step, the environmental image can be processed first using a lightweight image recognition model to output the category of the dangerous target, the coordinates of the bounding box, and the corresponding confidence level. Then, based on the positional relationship between the center point of the bounding box and the center of the image, as well as the wearer's orientation information, the orientation information of the dangerous target located to the left, right, front, or back of the wearer can be calculated. At the same time, the distance between the target and the wearer can be estimated based on the target's imaging size, camera calibration parameters, or perspective relationship.

[0144] When the hazard type corresponds to a high-risk category and the target is located in close proximity to the wearer, the hazard level can be set to a higher level; when the hazard type corresponds to a low-risk category and the target is far away or the confidence level is low, the hazard level can be set to a lower level.

[0145] If there is a comprehensive conflict between the hazard type, confidence level, location, and distance, a weighted judgment can be made according to preset weights to output a hazard level that matches the actual threat level.

[0146] Understandably, the determination of the danger level no longer relies solely on a single confidence level or a single location information, but rather on a joint analysis of the target category, identification credibility, and spatial relationships, so that the alarm judgment is more consistent with the wearer's actual risk status during the journey.

[0147] This step can improve the response speed to close-range, high-threat targets and reduce false alarms to distant or low-confidence targets, thereby enhancing the accuracy and practicality of smart glasses in complex travel scenarios.

[0148] In one possible implementation, the hazard types may include, for example, a first type, a second type, and a third type. The first, second, and third types are used to classify identified hazardous targets into threat categories, each corresponding to a different level of risk urgency; the threat level of the first type is less than that of the second type, which is less than that of the third type.

[0149] The first type can indicate potential threat targets, such as motor vehicles with abnormal driving at a distance, and pedestrians engaging in dangerous behaviors such as crossing the road or running red lights.

[0150] The second type can indicate a clear threat target, such as: a vehicle moving fast and approaching from directly in front, a suspicious person loitering and following for a long time, a suspicious person approaching, a vehicle driving in the wrong direction, a vehicle running a red light at a close distance, and a person running fast towards them.

[0151] The third type can indicate imminent threats, such as the timing of a close-range fight or an impending vehicle collision.

[0152] If a dangerous target meets the first condition, its hazard level can be determined as the first hazard level.

[0153] The first condition includes at least one of the following: the hazard type is type 1, the distance is within the first distance interval, and the confidence level is less than the preset confidence level.

[0154] For example, if a dangerous target belongs to the first type and its distance falls within the first distance range, or its confidence level is lower than the preset confidence level, then the dangerous target is determined to be of the first danger level.

[0155] If a hazardous target meets the second condition, its hazard level can be determined as the second hazard level.

[0156] The second condition includes at least one of the following: the hazard type is the second type, the distance is within the second distance interval, the hazard target is in front of the wearer, and the second distance interval is smaller than the first distance interval.

[0157] For example, if a dangerous target belongs to the second type and is within the second distance range and located in front of the wearer, it is judged as the second level of danger.

[0158] A dangerous target relative to the wearer's front can be understood as the area in the wearer's direction of travel. Understandably, forward-facing, close-range targets have a higher warning priority.

[0159] If a hazardous target meets the third condition, its hazard level can be determined as the third hazard level.

[0160] The third condition includes at least one of the following: the hazard type is the third type, the distance is within the third distance interval, and the confidence level is greater than the preset confidence level.

[0161] Since the second distance interval is smaller than the first distance interval and the third distance interval is smaller than the second distance interval, the closer the distance, the higher the corresponding danger level, thereby achieving rapid upgrading and classification of high-risk targets at close range.

[0162] Understandably, the first, second, and third hazard levels are used to represent the hazard classification results from low to high.

[0163] The first, second, and third distance intervals are used to characterize the spatial proximity between the dangerous target and the wearer, with the interval range decreasing sequentially.

[0164] This step incorporates target category, identification confidence level, and spatial location into the judgment, avoiding alerts based on a single factor. For high-threat types or close-range forward targets, a higher level of danger can be directly output; for targets with low confidence but still potential risks, a lower level can be initially set to reduce the spread of false alarms. By combining threat level ranking with distance interval stratification, more granular risk identification and more timely early warning output can be achieved in complex mobile scenarios, thereby improving the wearer's safety protection during travel.

[0165] For multi-target scenarios, the hazard level of each dangerous target can be calculated separately, and the highest level among multiple levels can be selected as the current global hazard level; alternatively, the hazard level information of multiple targets can be retained at the same time and output after being sorted by priority.

[0166] If two or more targets are simultaneously in a high-risk state, their location distribution can be further considered to determine the final warning form. For example, if there are high-risk targets in front and to the side at the same time, a stronger combined security action can be triggered.

[0167] S204. Perform security actions corresponding to the hazard level.

[0168] Different levels of danger correspond to different security actions.

[0169] Security actions may include, for example, playing voice prompts, providing alerts via vibration, and providing alerts via augmented reality technology.

[0170] In one possible implementation, the smart glasses may also include, for example, a bone conduction speaker and a vibrator.

[0171] Bone conduction speakers deliver voice prompts to the wearer without significantly obscuring ambient sound, making them suitable for pedestrian crossings, intersection crossings, and mixed pedestrian-vehicle traffic. Vibrators transmit hazard warnings to the wearer through tactile means, with their vibration frequency characterizing the intensity of the warning.

[0172] For example, a bone conduction speaker can play voice messages by placing the speaker close to the temporal bone, which can both remind the wearer and reduce interference with ambient noise.

[0173] The vibrator can be installed on the inside of the temple or near the nose pad, and its output end forms a stable coupling with the frame, making it easy to transmit vibration directly to the wearer's head.

[0174] During the execution of security actions, dangerous targets and corresponding warning messages can be identified first. These warning messages carry the prompts for the dangerous targets and can consist of text, graphics, or voice information.

[0175] Different dangerous targets and different levels of danger will receive different alert messages.

[0176] When the danger level is the highest level, the vibrator can be controlled to vibrate at the first vibration frequency, while the reminder message is displayed in augmented reality.

[0177] Augmented reality displays are used to overlay warning information onto the wearer's field of vision, allowing the wearer to perceive risks while looking at the environment in front of them. This can be achieved, for example, through a lens display module or a micro-projection module, to overlay warning messages onto the field of vision as semi-transparent subtitles, arrow markers, or highlighted outlines.

[0178] In cases where the danger level is the second danger level, the vibrator is controlled to vibrate at the second vibration frequency to display the reminder message in augmented reality, and the reminder message is played through the bone conduction speaker.

[0179] In the case of the third level of danger, the vibrator is controlled to vibrate at the third vibration frequency, the bone conduction speaker is controlled to play a reminder message, and a danger message is sent to the user terminal corresponding to the smart glasses.

[0180] The first, second, and third vibration frequencies can be set to different pulse repetition frequencies or vibration duty cycles, increasing sequentially, so that the tactile cues increase with the risk level. The third vibration frequency is higher than the second vibration frequency, which is higher than the first vibration frequency.

[0181] Danger messages may include information such as the type, location, distance, and current danger level of the target. The user terminal may be the terminal device of the emergency contact linked to the smart glasses, or the terminal device of other users associated with the smart glasses.

[0182] In this step, a reminder message can be generated first based on the current identification result, and then the corresponding output strategy can be called according to the danger level.

[0183] For lower-level risks, the vibrator provides alerts by vibrating at a low frequency in conjunction with an augmented reality display, allowing the wearer to remain aware of their surroundings.

[0184] For medium-risk scenarios, tactile cues and bone conduction voice cues are output in tandem to improve perceptibility in scenarios with high attention requirements.

[0185] For high-level risks, vibrators, bone conduction speakers, and user terminals work together to trigger alarms, forming a multi-channel early warning mechanism that links local and remote systems.

[0186] This step enables different levels of danger to correspond to different intensities and channels of alert output, thereby reducing the problem of missed alerts in situations with strong noise, distracted vision, or movement when using a single alert method. It also improves the perceptibility and reach of danger information, and enables alarms in high-risk situations to have terminal linkage capabilities, thereby enhancing the real-time warning effect of smart glasses in pedestrian safety protection scenarios.

[0187] The control method for smart glasses provided in this application acquires environmental images of the wearer through a camera device mounted on the smart glasses frame. It determines whether dangerous targets exist in the environmental images, and if so, determines the location information and confidence level of the dangerous targets. Based on the location information and / or confidence level, it determines the corresponding danger level and performs security actions corresponding to the danger level. This method requires no active operation from the wearer. It uses the smart glasses as a moving sensing carrier, combined with the camera device on the frame, to form a continuous image input facing the wearer's actual activity environment. This allows danger identification, level determination, and security response to be continuously carried out around the same usage scenario, providing safety protection services to the wearer before danger occurs, and providing a more timely and targeted safety protection foundation for traffic and personal risks during travel.

[0188] Figure 3 This is a schematic diagram of a control device for smart glasses provided in an embodiment of this application. The executing entity of this embodiment can, for example, be the smart glasses shown in the above embodiment. Figure 3 As shown, the control device 300 for the smart glasses includes:

[0189] The acquisition module 301 is used to acquire an image of the wearer's environment;

[0190] Processing module 302 is used to determine whether there are dangerous targets in the environmental image; if there are dangerous targets in the environmental image, determine the location information and confidence level of the dangerous targets; and determine the corresponding hazard level based on the location information and / or confidence level.

[0191] The control module 303 is used to execute security actions corresponding to the hazard level.

[0192] In one possible implementation, the processing module 302 is used to determine the hazard type of the dangerous target; determine the orientation and distance of the dangerous target relative to the wearer based on location information; and determine the hazard level based on the hazard type, confidence level, orientation and / or distance of the dangerous target relative to the wearer.

[0193] In one possible implementation, the hazard types include: a first type, a second type, and a third type. The processing module 302 is used to determine the hazard level as a first hazard level when the hazard target meets a first condition. The first condition includes at least one of the following: the hazard type is a first type, the distance is within a first distance interval, and the confidence level is less than a preset confidence level.

[0194] If the dangerous target meets the second condition, the danger level is determined to be the second danger level. The second condition includes at least one of the following: the danger type is the second type, the distance is within the second distance interval, the dangerous target is in front of the wearer, and the second distance interval is smaller than the first distance interval.

[0195] If a dangerous target meets the third condition, the danger level is determined to be the third danger level. The third condition includes at least one of the following: the danger type is the third type, the distance is within the third distance interval, the confidence level is greater than the preset confidence level, the third distance interval is smaller than the second distance interval, and the threat level of the first type is less than the threat level of the second type, which is less than the threat level of the third type.

[0196] In one possible implementation, the smart glasses also include: a bone conduction speaker and a vibrator; and a processing module 302, which is also used to determine dangerous targets and warning messages corresponding to the level of danger.

[0197] Control module 303 is used to control the vibrator to vibrate at a first vibration frequency when the hazard level is the first hazard level, and to display the reminder message in augmented reality.

[0198] When the danger level is the second danger level, the vibrator is controlled to vibrate at the second vibration frequency to display the reminder message in augmented reality and to play the reminder message through the bone conduction speaker. The second vibration frequency is higher than the first vibration frequency.

[0199] When the danger level is the third danger level, the vibrator is controlled to vibrate at the third vibration frequency, the bone conduction speaker is controlled to play a reminder message, and a danger message is sent to the user terminal corresponding to the smart glasses. The third vibration frequency is higher than the second vibration frequency.

[0200] In one possible implementation, the environmental image includes: multiple consecutive image frames; a processing module 302 is used to identify multiple candidate objects in the starting image frame of the multiple consecutive image frames; for any one of the multiple candidate objects, based on the multiple consecutive image frames, determine the motion trajectory of the candidate object; based on the motion trajectory, determine the behavioral state of the candidate object; and if the behavioral state indicates that the candidate object is abnormal, determine the candidate object as a dangerous target.

[0201] In one possible implementation, the environmental image includes: multiple consecutive image frames, and the processing module 302 is used to perform recognition processing on the multiple consecutive image frames through an image recognition model to determine whether there are dangerous targets in the environmental image.

[0202] The control device for smart glasses provided in this embodiment can execute the control method for smart glasses provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0203] Figure 4 A schematic diagram of the structure of smart glasses provided in this application embodiment. Figure 2 .like Figure 4 As shown, the smart glasses 400 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the smart glasses 400 also includes a communication interface 403. The processor 401, memory 402, and communication interface 403 are connected via a bus 404.

[0204] In the specific implementation process, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to execute the above-described control method for smart glasses.

[0205] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0206] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0207] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0208] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0209] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0210] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0211] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0212] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0213] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0214] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0215] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0216] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0217] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0218] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0219] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0220] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0221] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0222] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0223] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0224] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0225] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0226] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A control method of smart glasses, characterized by, The method, applicable to smart glasses, includes a camera device mounted on the frame of the smart glasses for capturing environmental images of the wearer. Acquire an environmental image of the wearer and determine whether there are any dangerous targets in the environmental image; If a dangerous target is present in the environmental image, determine the location information and confidence level of the dangerous target; Based on the location information and / or the confidence level, the corresponding hazard level is determined; Perform the security actions corresponding to the stated hazard level.

2. The method of claim 1, wherein, The step of determining the corresponding hazard level based on the location information and / or the confidence level includes: Determine the hazard type of the hazardous target; Based on the location information, the orientation and distance of the dangerous target relative to the wearer are determined; The hazard level is determined based on the hazard type, the confidence level, and the orientation and / or distance of the hazard target relative to the wearer.

3. The method of claim 2, wherein, The hazard types include: type one, type two, and type three. Determining the hazard level based on the hazard type, the confidence level, and the location and / or distance of the hazard target relative to the wearer includes: If the dangerous target meets the first condition, the danger level is determined to be the first danger level. The first condition includes at least one of the following: the danger type is a first type, the distance is within a first distance interval, and the confidence level is less than a preset confidence level. If the dangerous target meets the second condition, the danger level is determined to be a second danger level. The second condition includes at least one of the following: the danger type is a second type, the distance is within a second distance interval, the dangerous target is in front of the wearer, and the second distance interval is smaller than the first distance interval. If the dangerous target meets the third condition, the danger level is determined to be the third danger level. The third condition includes at least one of the following: the danger type is the third type, the distance is within the third distance interval, the confidence level is greater than the preset confidence level, the third distance interval is smaller than the second distance interval, and the threat level of the first type is less than the threat level of the second type is less than the threat level of the third type.

4. The method of claim 3, wherein, The smart glasses also include: a bone conduction speaker and a vibrator, and the execution of the security actions corresponding to the danger level includes: Identify the dangerous target and the corresponding alert message based on the danger level; When the danger level is the first danger level, the vibrator is controlled to vibrate at a first vibration frequency to display the reminder message in augmented reality. When the danger level is the second danger level, the vibrator is controlled to vibrate at a second vibration frequency to display the reminder message in augmented reality, and the bone conduction speaker is controlled to play the reminder message, wherein the second vibration frequency is higher than the first vibration frequency; When the danger level is the third danger level, the vibrator is controlled to vibrate at the third vibration frequency, the bone conduction speaker is controlled to play the warning message, and a danger message is sent to the user terminal corresponding to the smart glasses. The third vibration frequency is higher than the second vibration frequency.

5. The method of claim 1, wherein, The environmental image includes multiple consecutive image frames, and determining whether a dangerous target exists in the environmental image includes: Identify multiple candidate objects in the starting image frame of multiple consecutive image frames; For any one of the multiple candidate objects, the motion trajectory of the candidate object is determined based on multiple consecutive image frames; Based on the motion trajectory, the behavioral state of the candidate object is determined; If the behavioral state indicates that the candidate object is abnormal, the candidate object is determined to be a dangerous target.

6. The method of claim 1, wherein, The environmental image includes multiple consecutive image frames, and determining whether a dangerous target exists in the environmental image includes: An image recognition model is used to identify multiple consecutive image frames to determine whether there are dangerous targets in the environmental image.

7. An intelligent eyewear, characterized in that, include: Eyeglass frames, temples, lenses, camera device, and processor; The camera device is mounted on the frame of the glasses and is used to capture images of the wearer's surroundings. The processor is disposed on the temple and is used to implement the control method of the smart glasses according to any one of claims 1-5.

8. The smart glasses of claim 7, wherein, The number of the camera devices is one or three; When there is only one camera device, the camera device is located on the front of the eyeglass frame and is used to capture the wearer's first-person perspective image; When there are three camera devices, the camera devices are respectively located on the front of the frame, on the side near the left temple, and on the side near the right temple.

9. A control device of smart glasses, characterized by, An application is made to smart glasses, wherein a camera device is installed on the frame of the smart glasses for capturing environmental images of the wearer, the device comprising: The acquisition module is used to acquire an environmental image of the wearer; The processing module is configured to determine whether a dangerous target exists in the environmental image; if a dangerous target exists in the environmental image, determine the location information and confidence level of the dangerous target; and determine the corresponding hazard level based on the location information and / or the confidence level. The control module is used to execute the security actions corresponding to the danger level.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

Images