Wearable Walking Safety Assistance Device Using Gait-Speed Normalization and Control Method Thereof

Abstract

The present invention relates to a wearable walking safety assist device using walking speed normalization and a control method thereof. The wearable walking safety assist device using walking speed normalization and a control method thereof, which is mounted on the head of a wearer, comprises: a distance measuring unit that repeatedly measures the distance to a target object located in front of the wearer; an inertia measuring unit that measures acceleration and angular velocity according to the movement of the wearer; a notification unit that notifies the wearer of danger regarding the target object; and a control unit connected to the distance measuring unit, the inertia measuring unit, and the notification unit. The control unit comprises: a walking speed estimation unit that determines whether the wearer is walking based on inertia information measured by the inertia measuring unit and calculates the walking speed of the wearer if the wearer is walking; and an approach speed calculation unit that calculates the approach speed of the target object based on changes in the distance to the target object measured at different points in time by the distance measuring unit. The system may be characterized by including a risk determination unit that calculates a normalized approach ratio by normalizing the approach speed of the target object based on the walking speed of the wearer, classifies the target object into one of a companion object moving with the wearer, a stationary object stationary relative to the surrounding environment, or an opposing moving object moving toward the wearer according to one of a plurality of standard ranges to which the normalized approach ratio belongs, and determines whether to generate an alarm through the notification unit according to the classification result of the target object.

Description

Technology Field

[0001] The present invention relates to a wearable walking safety assist device using walking speed normalization and a control method thereof. Background Technology

[0003] Visually impaired individuals, the elderly, and pedestrians with reduced mobility find it difficult to quickly perceive obstacles or differences in ground level located ahead while walking. Accordingly, electronic walking safety assistive devices are being developed that detect dangers in the walking environment and notify the wearer using not only physical assistive devices such as white canes, but also ultrasonic sensors, time-of-flight distance sensors, inertial measurement units, and cameras.

[0004] A standard white cane is useful for checking the ground in front of the wearer or the area near their feet via contact, but its detection range is limited by the length of the cane and the height at which contact is possible. Therefore, it is difficult to detect obstacles located at the wearer's chest or head height, such as protruding signs, tree branches, open windows, or loaded objects, in advance. Furthermore, since the presence of an obstacle can only be recognized after actually touching or approaching it, there is a problem in that it is difficult to secure sufficient time and distance to avoid a collision.

[0005] To address these issues, an electronic walking assistance device may be used in which an ultrasonic sensor or a time-of-flight distance sensor is mounted on the wearer's body or an assistive device, and an alarm is output via voice, vibration, or light when the measured distance to a forward object falls below a set reference distance.

[0006] However, conventional distance-based walking assistance devices primarily rely on the estimated collision time calculated from the currently measured distance or changes in distance as the criterion for risk assessment. In this case, as the wearer walks, stationary walls, pillars, or parked vehicles in the surrounding environment are also measured as approaching the wearer; consequently, alarms may occur frequently regardless of the actual likelihood of a collision. Particularly in environments where walls or structures are densely packed, such as indoors, nearby objects may be detected in every direction the wearer looks, leading to continuous alarms.

[0007] Furthermore, companions moving in the same direction and at a similar speed as the wearer, or the white cane used by the wearer, can trigger an alarm simply because they remain within a certain distance, even though the actual risk of collision is low as their relative distance from the wearer hardly changes. Moreover, objects that actually deviate from the collision path—such as pedestrians who cross the wearer's path and then move to the side or brush past—are difficult to clearly distinguish from targets for head-on collisions based solely on simple distance information.

[0008] If such false alarms are repeated, the wearer may experience alarm fatigue or lose trust in the device's judgment results. Consequently, there is a higher likelihood that the alarm will be ignored even when actual danger occurs, which may degrade the practical safety function of the walking safety assist device.

[0009] Meanwhile, a method of applying an object detection neural network to camera images to determine the type and direction of movement of people or obstacles can also be considered. However, since conventional object detection neural networks require a large amount of computation and memory, it is difficult to process them continuously in real time on low-power microcontrollers in small, battery-operated wearable devices. Furthermore, increased heat generation and battery consumption due to the computational load make it unsuitable for long-term wear, and if image processing is delegated to an external server, communication delays or network disconnections may cause safety detection functions to stop or judgments to be delayed.

[0010] In head-mounted walking assistance devices, it is necessary to switch the detection mode by determining whether the wearer is looking straight ahead or has lowered their head toward the ground. However, in methods that directly use pitch angles calculated from specific axes of an inertial measurement unit, the mapping or sign of the measurement axis may vary depending on the sensor's mounting direction, driver, or firmware. Consequently, threshold values ​​must be reset for each device, and the wearer's left-right rotation and head lowering may be confused, leading to an incorrect switching of the detection mode.

[0011] In addition, when the device is mounted horizontally on the wearer's forehead and the measurement beam of the distance sensor is directed forward at head height, protruding obstacles can be effectively detected, whereas forward descending stairs or depressions may be located in a blind spot below the measurement beam. Since it is difficult to constantly detect such ground hazards using only the distance sensor unless the wearer directly lowers their head, a separate judgment structure is required to compensate for the measurement blind spot of the distance sensor while maintaining the processing performance of the low-power device.

[0012] Furthermore, when a wearable walking safety assist device is linked with an external management terminal, such as a smartphone, to perform setting adjustments, status inquiries, and software updates, communication processing and the periodic transmission of status information may compete with sensor detection, risk assessment, and processor resources. Consequently, the detection cycle may become irregular or the response to risks may be delayed; additionally, if data transmission is interrupted during wireless software updates, the existing program may be damaged, potentially preventing the device from operating normally.

[0013] In particular, in structures where an external terminal directly connects to a device via short-range wireless communication such as Bluetooth, there is a possibility that a third party other than the legitimate owner may access the device and execute commands such as factory reset or ownership release. In situations where communication with an external server is impossible, it is difficult to verify the legitimacy of the owner, which can lead to security issues where the device is reset or stolen by a third party.

[0014] Therefore, there is a need for a wearable walking safety assistance technology that can suppress unnecessary warnings by considering the wearer's walking state and the relative motion characteristics of the target object together, stably determine head-height obstacles and ground hazards even in low-power environments, and provide safety functions that are robust to posture determination against axis deviations of inertial sensors and unaffected by communication failures. The problem to be solved

[0016] The present invention aims to provide a wearable walking safety assist device and a control method thereof that can detect head-height obstacles and dangers on the ground ahead that are difficult to detect with a white cane alone while walking, and can selectively provide an alarm when there is a high actual danger, for visually impaired people, the elderly, and walking-vulnerable people with reduced walking ability.

[0017] In addition, the present invention aims to reduce unnecessary alarms by distinguishing between accompanying objects, stationary objects in the surrounding environment, and opposing moving objects moving toward the wearer, by considering both the wearer's walking speed and the target object's approach speed, rather than simply generating an alarm based solely on the current distance to the target object.

[0018] In addition, the present invention aims to identify objects crossing or sideways from the wearer's path of travel and obstacles located at the height of the wearer's head using low-computation image information, while preventing image movement caused by the wearer's head rotation from being incorrectly judged as the actual movement of the target object.

[0019] In addition, the present invention aims to provide a stable risk determination structure capable of suppressing false alarms caused by head vibrations occurring during walking or momentary measurement errors of distance sensors, while generating an alarm without delay when a target object enters within the closest distance.

[0020] In addition, the present invention aims to determine the wearer's head posture using a calibrated gravity vector and a tilting direction vector without directly relying on a specific axis or sign of an inertial sensor, and to prevent unnecessary repeated switching between a forward monitoring mode and a ground monitoring mode.

[0021] In addition, the present invention aims to provide a warning suitable for the type of ground hazard by calculating the vertical drop using the head tilt angle and the measured distance to the ground, and classifying the terrain ahead into ledges, stairs, objects on the ground, or holes based on this.

[0022] In addition, the present invention aims to compensate for blind spots on the ground ahead that are not reached by the measurement beam of the distance sensor using a lightweight image classification neural network, while in the event of the absence of the neural network model or a loading failure, only the image-based compensation function is disabled and the basic safety function based on distance and inertia is maintained.

[0023] In addition, the present invention aims to automatically correct the wearer's reference posture by sequentially detecting the device's preceding movement, continuous stopping, and continuous walking, and to enable re-correction using head gestures without separate visual confirmation or assistance from others.

[0024] In addition, the present invention aims to prevent the initialization or theft of the device by a non-owner by performing risk assessment and alarms within the device itself even when external communication is disconnected, and by verifying the owner internally regarding destructive commands such as factory initialization or release of ownership.

[0025] In addition, the present invention aims to compensate for position errors in satellite navigation and provide guidance for left turns, right turns, or crossing at appropriate times by using the rotation prediction information of an external guidance terminal together with the wearer's head view image to verify whether the actual rotation entry point has been reached.

[0026] However, the problems that the present invention aims to solve are not limited to those explicitly mentioned above, and other technical problems that a person skilled in the art can recognize from the following description may also be included in the problems that the present invention aims to solve. means of solving the problem

[0028] To solve the above problem, a wearable walking safety assist device according to one embodiment of the present invention may include a distance measuring unit that repeatedly measures the distance to a target object located in front of the wearer, an inertia measuring unit that measures the acceleration and angular velocity of the wearer, a notification unit that notifies the wearer of danger, and a control unit that controls the operation of each of the above components.

[0029] The control unit may include a walking speed estimation unit that calculates whether the wearer is walking and the walking speed from inertial information, an approach speed calculation unit that calculates the approach speed of a target object from distance measurements at multiple points in time, and a risk determination unit that calculates a normalized approach ratio by normalizing the approach speed based on the wearer's walking speed.

[0030] The above-mentioned risk determination unit classifies the target object into one of a companion object moving with the wearer, a stationary object stationary relative to the surrounding environment, or an opposing moving object moving toward the wearer, based on the standard range to which the normalized approach ratio belongs, and can determine whether to generate an alarm based on the classification result and the distance to the target object. Accordingly, the motion characteristics of the target object, which are difficult to distinguish based on simple distance information alone, can be determined based on the wearer's own walking speed.

[0031] In addition, the wearable walking safety assist device of the present invention may further include an image acquisition unit that acquires a front image of the wearer. The image fusion determination unit of the control unit can extract a single image blob from a region of interest in the front image and analyze the lateral movement speed, height, and area of ​​the image blob to determine whether it is an object that is passing by, an obstacle at head height, or a valid target object.

[0032] In this case, if the yaw direction angular velocity measured by the inertial measurement unit exceeds a reference value, the lateral movement information of the image blob generated by the wearer's head rotation can be excluded from the basis for alarm suppression. Additionally, if the target object is located within the nearest distance, an alarm is generated immediately regardless of image judgment or consensus judgment, and if the same danger judgment is detected consecutively more than a reference number, a general alarm is generated, and the generated alarm can be maintained for a certain period of time.

[0033] Meanwhile, the attitude determination unit of the control unit can set a tilting direction vector based on the difference between the frontal gravity vector obtained in a frontal attitude and the tilting gravity vector obtained in a head-tilted state, and calculate an attitude projection value by projecting the current acceleration vector onto the tilting direction vector. If the attitude projection value exceeds a first attitude reference value, the system switches from a forward monitoring mode to a ground monitoring mode, and if it decreases to less than a second attitude reference value which is smaller than the first attitude reference value, the system can return to a forward monitoring mode.

[0034] In ground monitoring mode, the ground hazard judgment unit of the control unit calculates a vertical drop from the distance to the ground measured by the distance measuring unit and the head tilt angle, and can classify the front terrain as a ledge, stairs, an object on the floor, or a hole based on the difference between the vertical drop and the ground reference line. Additionally, if a change in the yaw direction angular velocity of left and right head shaking is detected more than a reference number of times within a certain period, calibration can be re-performed.

[0035] To compensate for the risk of stairs or sinkholes on the ground ahead that are not reached by the measurement beam of the distance measuring unit, the control unit may further include a neural network ground risk judgment unit that operates with an image judgment cycle independent of the distance and inertia-based risk judgment. The neural network ground risk judgment unit determines ground risk using the classification results and classification reliability of multiple frames, and if the neural network model does not exist or fails to load, only the neural network judgment function is disabled while the distance and inertia-based risk judgment can continue to be performed.

[0036] In addition, the automatic correction unit of the control unit can temporarily set the frontal gravity vector if a stationary state is maintained for a certain period after the device's preceding movement is detected, and then finalize the correction when continuous walking is detected. Accordingly, the wearer can correct the reference posture solely through the process of wearing the device, without visual confirmation or assistance from others.

[0037] The wearable walking safety assist device may further include a short-range communication unit that communicates with an external management terminal and a non-volatile storage unit that stores owner information. When a destructive command, such as a factory reset or release of ownership, is received, the owner verification unit of the control unit compares the requester identifier with the owner information stored in the device internally and may allow the execution of the command only if the two match.

[0038] In addition, when advance information regarding the direction of rotation is received from an external guidance terminal, the forward image is periodically acquired through an image acquisition unit, and after determining whether the actual rotation entry point has been reached using an internal or external image judgment device, arrival information can be transmitted to the external guidance terminal. Accordingly, left turn, right turn, or crossing guidance can be provided at an appropriate time while compensating for position errors caused by satellite navigation.

[0039] The means for solving the problem described above are merely exemplary and should not be interpreted as intended to limit the present invention. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and the detailed description of the invention. Effects of the invention

[0041] The present invention can distinguish between accompanying objects, stationary objects, and opposing moving objects by normalizing the approach speed of a target object based on the wearer's walking speed, and can selectively provide warnings only for actual collision risks.

[0042] In addition, false alarms can be reduced by using the movement speed, height, and area of ​​a single image blob to identify passing objects and head-height obstacles.

[0043] Based on gravity vector projection, the head posture is stably determined, and jaws, stairs, objects on the floor, and holes can be distinguished using the measurement distance and tilt angle.

[0044] By independently operating distance and inertia-based safety judgments and lightweight image neural networks, the blind spots of the distance sensor on the ground can be compensated for, and basic safety functions can be maintained even in the event of neural network errors.

[0045] Posture correction and re-correction can be performed using wear detection and head gestures without separate visual verification or assistance from others.

[0046] In addition, even in the event of a communication interruption, the device itself performs risk assessment and notification, and can enhance the reliability and security of the device through owner verification and secure software updates.

[0047] However, the effects of the present invention are not limited to those explicitly mentioned above, and other effects that a person skilled in the art can recognize from the composition of the present invention may also be included in the effects of the present invention. Brief explanation of the drawing

[0049] FIG. 1 is a block diagram showing the overall configuration of a wearable walking safety assist device and the functional configuration of a control unit according to one embodiment of the present invention. FIG. 2 is a diagram showing the wearing state of a wearable walking safety assist device according to one embodiment of the present invention and the detection area of ​​the distance measuring unit and the image acquisition unit. FIG. 3 is a flowchart illustrating a process of calculating a normalized approach ratio from the walking speed of a wearer and the approach speed of a target object according to an embodiment of the present invention, classifying the target object, and determining whether to issue an alarm. FIG. 4 is a flowchart illustrating a process of confirming the continuity of the same risk assessment according to one embodiment of the present invention, immediately generating an alarm at the nearest distance, and maintaining the generated alarm for a certain period of time. FIG. 5 is a diagram illustrating a process of performing image fusion determination using the lateral movement speed, screen height, and area of ​​a single image blob extracted from a front image according to an embodiment of the present invention, and compensating for image movement due to head rotation of the wearer. FIG. 6 is a diagram illustrating a process of determining the head posture of a wearer using a frontal gravity vector and a head tilting direction vector according to an embodiment of the present invention, and classifying ground hazards based on the measured distance to the ground and the head tilting angle. FIG. 7 is a flowchart illustrating a process of detecting a wearer's head shaking gesture and re-calibrating according to an embodiment of the present invention, and automatically correcting a reference posture based on the device's preceding movement, continuous stopping, and continuous walking. FIG. 8 is a diagram showing a configuration in which a distance-inertia-based safety judgment channel and an image classification neural network-based ground risk supplement channel run in parallel with independent judgment cycles according to an embodiment of the present invention, and a functional isolation structure in case of neural network judgment failure. FIG. 9 is a flowchart illustrating a process of determining whether to execute a command by comparing owner information stored in a device with a requester identifier when a destructive command is received from an external management terminal according to an embodiment of the present invention. FIG. 10 is a diagram illustrating the process of a wearable walking safety assist device according to an embodiment of the present invention acquiring a front image in conjunction with an external guidance terminal and an external image judgment device, and confirming whether an actual turning entry point has been reached to provide route guidance. Specific details for implementing the invention

[0050] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the invention in relation to one embodiment.

[0051] Furthermore, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, including all equivalents thereof, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0052] Meanwhile, throughout this specification, when a part is described as “comprising” a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, the “part” for a component as used in this specification performs at least one function or operation. And the “part” may perform the function or operation by hardware, software, or a combination of hardware and software.

[0053] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but these components are not limited by the aforementioned terms. The aforementioned terms are used solely for the purpose of distinguishing one component from another.

[0054] In this specification, terms such as “comprising” are intended to indicate the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof. When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected to or coupled with the other component, or that there may be other components in between.

[0055] Furthermore, in describing the present invention, if it is determined that a detailed description of related known functions or configurations may unnecessarily obscure the essence of the invention, such detailed description is abbreviated or omitted.

[0056] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. However, the embodiments described below are merely examples to aid in understanding the present invention, and the scope of the present invention is not limited thereto. Furthermore, the configurations illustrated in each drawing are schematically depicted for convenience of explanation, and some configurations may be combined, separated, omitted, or added depending on the actual implementation.

[0057] FIG. 1 is a block diagram showing the overall configuration of a wearable walking safety assist device and the functional configuration of a control unit according to one embodiment of the present invention.

[0058] FIG. 2 is a diagram showing the wearing state of a wearable walking safety assist device according to one embodiment of the present invention and the detection area of ​​the distance measuring unit and the image acquisition unit.

[0059] FIG. 3 is a flowchart illustrating a process of calculating a normalized approach ratio from the walking speed of a wearer and the approach speed of a target object according to an embodiment of the present invention, classifying the target object, and determining whether to issue an alarm.

[0060] FIG. 4 is a flowchart illustrating a process of confirming the continuity of the same risk assessment according to one embodiment of the present invention, immediately generating an alarm at the nearest distance, and maintaining the generated alarm for a certain period of time.

[0061] FIG. 5 is a diagram illustrating a process of performing image fusion determination using the lateral movement speed, screen height, and area of ​​a single image blob extracted from a front image according to an embodiment of the present invention, and compensating for image movement due to head rotation of the wearer.

[0062] FIG. 6 is a diagram illustrating a process of determining the head posture of a wearer using a frontal gravity vector and a head tilting direction vector according to an embodiment of the present invention, and classifying ground hazards based on the measured distance to the ground and the head tilting angle.

[0063] FIG. 7 is a flowchart illustrating a process of detecting a wearer's head shaking gesture and re-calibrating according to an embodiment of the present invention, and automatically correcting a reference posture based on the device's preceding movement, continuous stopping, and continuous walking.

[0064] FIG. 8 is a diagram showing a configuration in which a distance-inertia-based safety judgment channel and an image classification neural network-based ground risk supplement channel run in parallel with independent judgment cycles according to an embodiment of the present invention, and a functional isolation structure in case of neural network judgment failure.

[0065] FIG. 9 is a flowchart illustrating a process of determining whether to execute a command by comparing owner information stored in a device with a requester identifier when a destructive command is received from an external management terminal according to an embodiment of the present invention.

[0066] FIG. 10 is a diagram illustrating the process of a wearable walking safety assist device according to an embodiment of the present invention acquiring a front image in conjunction with an external guidance terminal and an external image judgment device, and confirming whether an actual turning entry point has been reached to provide route guidance.

[0067] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. The embodiments described below are merely examples to specifically explain the technical concept of the present invention, and the scope of the present invention is not limited to the following embodiments or specific numerical values.

[0068] Additionally, in this specification, the term “connected” of one component to another component may include not only cases where it is directly connected via wired or wireless connection, but also cases where it is indirectly connected through at least one intermediate component.

[0069] In this specification, “objects” may include people, companions, white canes, vehicles, walls, pillars, signs, tree branches, loads, and structures located in front of or in the path of travel of the wearer. “Ground hazards” may include ledges, stairs, objects on the floor, holes, depressions, steep drops, and non-reflective areas.

[0070] Each functional part described in this specification may be implemented as a physically separated circuit or module, or may be functionally implemented by program instructions executed on one or more processors.

[0071] 1. Overall Components of the Wearable Walking Safety Assist Device

[0072] FIG. 1 is a block diagram showing the overall configuration of a wearable walking safety assist device according to one embodiment of the present invention.

[0073] Referring to FIG. 1, the wearable walking safety assist device (100) may include a distance measuring unit (110), an inertial measuring unit (120), an image acquisition unit (130), a control unit (140), a notification unit (160), a short-range communication unit (170), a non-volatile storage unit (180), and an input unit (190).

[0074] The wearable walking safety assist device (100) may be mounted on the wearer's head, forehead, glasses, hat, headband, or a separate head attachment. In one embodiment, the wearable walking safety assist device (100) is mounted horizontally near the wearer's forehead, and the measurement direction of the distance measuring unit (110) may be directed toward the wearer's front and head height.

[0075] The distance measuring unit (110) repeatedly measures the distance to an object located in front of the wearer. The distance measuring unit (110) may include a ToF sensor, which is a time-of-flight distance sensor.

[0076] The distance measuring unit (110) generates a measurement value according to a constant distance measurement cycle. In one embodiment, the distance measurement cycle is about 60 ms, and the forward distance can be measured at a frequency of about 16 Hz. The measurement value of the distance measuring unit (110) can be stored as a distance measurement history along with the measurement time.

[0077] The inertial measurement unit (120) measures acceleration and angular velocity according to the movement of the wearer. The inertial measurement unit (120) may be a 6-axis inertial measurement device including a 3-axis accelerometer and a 3-axis gyroscope.

[0078] The measurement information of the inertial measurement unit (120) can be used to estimate whether the wearer is walking and the walking speed, to determine the wearer's head posture, and to detect recalibration gestures and falls.

[0079] The image acquisition unit (130) acquires an image including the front of the wearer and the front ground. The image acquisition unit (130) can generate a grayscale image of 160×120 pixels with QQVGA resolution, for example, when performing image fusion judgment.

[0080] The image acquisition unit (130) can perform different functions using a single camera. That is, the image acquisition unit (130) can selectively generate a blob image for determining the lateral movement and height of an object, a neural network input image for classifying dangers on the ground ahead, and a front image provided to an external image judgment device (400) in a path guidance mode.

[0081] The control unit (140) is connected to the distance measuring unit (110), the inertial measuring unit (120), the image acquisition unit (130), the notification unit (160), the short-range communication unit (170), the non-volatile storage unit (180), and the input unit (190).

[0082] The control unit (140) receives sensor measurement values, determines the walking state of the wearer and the movement state of the target object, and determines the risk level and whether to issue an alarm. The control unit (140) can control the switching of the device's operation mode, sensor operation, communication processing, and storage of setting values ​​together.

[0083] The control unit (140) may include a walking speed estimation unit (141), an approach speed calculation unit (142), a danger judgment unit (143), an image fusion judgment unit (144), a posture judgment unit (145), a ground danger judgment unit (146), a calibration unit (147), a neural network ground danger judgment unit (148), an automatic correction unit (149), an owner verification unit (150), and a guidance mode control unit (151).

[0084] The notification unit (160) notifies the wearer of danger or the status of the device according to the judgment result of the control unit (140). The notification unit (160) may include an RGB light-emitting element, a vibration motor, a speaker, a bone conduction output device, or a combination thereof.

[0085] The short-range communication unit (170) performs short-range wireless communication with an external management terminal (200). In one embodiment, the short-range communication unit (170) performs BLE communication, and the external management terminal (200) may be a management application installed on a smartphone.

[0086] The non-volatile storage unit (180) can store at least one of calibration information, owner information, factory settings, administrator default values, user settings, program files, image classification neural network models, and operation history.

[0087] The input unit (190) may include a physical button, a head gesture, a virtual button of an external management terminal (200), or a press-and-speak input for initiating voice input.

[0088] 2. Configuration of Operation Modes

[0089] The control unit (140) can operate the device (100) in any one of walking mode, assistant mode, navigation mode, power saving mode, and update mode.

[0090] The walking mode is a basic operation mode and may include a front sub-mode for determining a frontal collision and a ground monitoring sub-mode for determining ground hazards.

[0091] In the front sub-mode, a forward collision judgment is performed based on the distance measuring unit (110) and the inertial measuring unit (120). If necessary, a blob-based judgment of the image fusion judgment unit (144) and a ground risk supplementary judgment of the neural network ground risk judgment unit (148) may be performed in parallel.

[0092] In the ground monitoring sub-mode, the distance measuring unit (110) faces the ground in front while the wearer lowers their head, and the ground hazard judgment unit (146) classifies ledges, stairs, objects on the floor, and holes.

[0093] In assistant mode, voice querying and answering can be performed, and the forward risk detection loop can be selectively stopped to prevent resource conflicts between voice processing and detection processing.

[0094] In the navigation mode, the forward collision detection and image fusion detection may be stopped so that the route guidance of the external guidance terminal (300) and the alarm sound of the device do not overlap. However, the display of walking or stationary state, fall detection, and short-range communication may continue to be maintained.

[0095] In power saving mode, all or part of the detection functions by the distance measuring unit (110), the inertial measuring unit (120), and the image acquisition unit (130) may be stopped. The short-range communication unit (170) can maintain an active state to receive a wake-up command from an external management terminal (200).

[0096] In update mode, the detection function by the sensor can be stopped and software updates can be performed. During the update, the notification unit (160) outputs a separate update display pattern, and after the update is completed, the device can be automatically restarted.

[0097] 3. Wearing Status and Sensor Detection Area

[0098] FIG. 2 shows the wearing state of the device (100) and the detection area of ​​the distance measuring unit (110) and the image acquisition unit (130).

[0099] The measuring beam (111) of the distance measuring unit (110) can be directed forward at the height of the wearer's head. Accordingly, it is possible to detect head-height obstacles (20), such as protruding signs, tree branches, open windows, and stacked objects, which are difficult to detect with a white cane.

[0100] The distance measuring unit (110) can obtain distance measurements for a stationary object (21), an opposing moving object (22) moving toward the wearer, and a companion object (23) moving with the wearer.

[0101] The shooting area (131) of the image acquisition unit (130) may include a wider area than the measurement beam (111) of the distance measuring unit (110). Accordingly, a spatial correspondence can be formed between the target object measured by the distance measuring unit (110) and the blob extracted from the image.

[0102] When the device (100) is mounted horizontally on the forehead, the measuring beam (111) of the distance measuring unit (110) faces the front horizontal direction, so the forward descending step (30) can be located in the ground square area (31) below the measuring beam.

[0103] The neural network ground risk determination unit (148) can use the image of the image acquisition unit (130) to supplementarily determine the risk of stairs, etc. located in the ground blind area (31).

[0104] 4. Estimation of Walking Speed

[0105] Referring to FIG. 3, the walking speed estimation unit (141) receives inertial information including acceleration and angular velocity from the inertial measurement unit (120) (S110).

[0106] The walking speed estimation unit (141) can extract the vibration component caused by walking by removing the gravity component or the long-term average component from the acceleration measurement value.

[0107] If the magnitude of the extracted vibration component exceeds GAIT_STEP_G, which is a preset step impact threshold value, that point in time is detected as a step impact candidate (S120).

[0108] However, since temporary changes in acceleration may occur even when the user turns their head or moves the device with their hand, walking is not confirmed by a single impact alone.

[0109] The walking speed estimation unit (141) calculates the time interval between adjacent step impact candidates and checks whether the time interval is greater than or equal to GAIT_MIN_STEP_MS and less than or equal to GAIT_MAX_STEP_MS.

[0110] It is determined that the wearer is walking only when time intervals falling within the above walking rhythm range are detected consecutively at least once (GAIT_MIN_BEATS times or more) (S130).

[0111] The walking speed estimation unit (141) can calculate the walking speed v using the stride length WALK_STEP_M and the step period Tstep as follows.

[0112] [Mathematical Formula 1]

[0113] v = WALK_STEP_M × 1000 / Tstep

[0114] Here, v is the walking speed in mm / s, and Tstep can be the step cycle in seconds.

[0115] The walking speed estimation unit (141) can determine the walking state according to a fixed period independent of the overall loop speed of the control unit (140). In one embodiment, the walking determination period GAIT_SAMPLE_MS is 20 ms, and accordingly, walking vibration can be analyzed at 50 Hz.

[0116] By applying a fixed decision period, the filter characteristics of the acceleration baseline remain constant even if the processing speed of the entire control loop changes. Accordingly, it is possible to prevent the low-pass baseline from tracking the footstep vibration too quickly and causing the vibration component to be lost.

[0117] If no valid step impact is detected during the recent GAIT_IDLE_MS, the walking speed estimation unit (141) can determine that the wearer is in a stationary state and set the walking speed v to 0.

[0118] 5. Calculation of Access Speed ​​and Normalized Access Ratio

[0119] The distance measuring unit (110) repeatedly measures the distance to the target object at a measurement cycle corresponding to TOF_POLL_MS (S150). In one embodiment, TOF_POLL_MS is 60 ms.

[0120] The approach speed calculation unit (142) generates a distance measurement history by storing the distance measurement value and measurement time of the observation section corresponding to the recent REG_WINDOW_MS (S160). In one embodiment, REG_WINDOW_MS is 300 ms.

[0121] The approach speed calculation unit (142) can calculate the approach speed r of the target object by calculating the slope of the distance value decreasing over time in the observation section (S170).

[0122] The approach speed r may be calculated by the difference between two measurements or by the linear regression slope of multiple distance measurements. When using multiple measurements, abrupt changes in the approach speed caused by instantaneous noise from the sensor can be mitigated.

[0123] The risk determination unit (143) calculates the normalized approach rate R by normalizing the approach speed r of the target object to the walking speed v of the wearer (S180).

[0124] [Mathematical Formula 2]

[0125] R = r / v

[0126] If the target object is a companion or a white cane moving in the same direction and at a similar speed as the wearer, the relative distance between the wearer and the target object does not change much, so r and R can approach 0.

[0127] When the target object is a stationary object relative to the surrounding environment, such as a wall, a pillar, or a parked vehicle, R can approach 1 because the target object is measured to be approaching at a speed similar to the wearer's walking speed due to the wearer's movement.

[0128] In the case where the target object is an oncoming pedestrian moving toward the wearer, the wearer's walking speed and the target object's moving speed are both reflected in the approach speed, so R can be calculated to be greater than 1.

[0129] In one embodiment, the risk assessment unit (143) may apply the following criteria.

[0130] R ≤ 0.25: Determines as a companion object or white cane and suppresses the alarm

[0131] R ≥ 0.6: Determined as a stationary object relative to the surrounding environment

[0132] R ≥ 1.55: Determined as an opposing moving object moving toward the wearer

[0133] The risk determination unit (143) can determine whether to issue a final warning by using the above criteria together with the distance to the target object, the actual approach amount of the observation section, the image fusion result, and the number of consecutive judgments, rather than mechanically applying only one of the above criteria.

[0134] 6. Ensuring Reliability of Distance Measurements and Handling Exceptions

[0135] The three most recent measurements of the distance measuring unit (110) can be smoothed using the median method.

[0136] However, if the current measurement value is closer than TOF_SUDDEN_MM than the median of recent measurements, smoothing may be omitted and the current measurement value may be adopted immediately. In one embodiment, TOF_SUDDEN_MM is 150 mm.

[0137] This is intended to prevent a delayed alarm caused by smoothing when a pillar or wall that was hidden behind a preceding pedestrian is suddenly detected at close range immediately after the pedestrian has stepped aside.

[0138] The approach speed calculation unit (142) can determine that the approach is valid only when the actual distance reduction within the observation section is greater than or equal to MIN_CLOSURE_MM. In one embodiment, MIN_CLOSURE_MM is 80 mm.

[0139] Requiring a minimum net approach amount in this way can reduce the misidentification of the slope calculated from minute noise of the distance sensor as the actual approach speed.

[0140] If the change in distance between samples exceeds TRACK_JUMP_MM, the approach speed calculation unit (142) can determine that the existing target object has been lost and a new target object has appeared. In one embodiment, TRACK_JUMP_MM is 600 mm.

[0141] If a new target object is identified, the existing distance measurement history and approach speed are reset, and a new track is created. Conversely, temporary sensor misses are allowed up to the number corresponding to TRACK_MISS_MAX to maintain the measurement history of the existing target object.

[0142] 7. Judgment while the wearer is stationary

[0143] When the wearer's walking speed v is less than STANDING_MMS, it is difficult to directly apply the normalization approach ratio R. Accordingly, the risk determination unit (143) can perform a separate determination branch corresponding to the stationary state.

[0144] If a target object approaches the wearer at a speed greater than STANDING_R_MMS while the wearer is stationary, it can be determined as a danger caused by a moving object. In one embodiment, STANDING_R_MMS is 350 mm / s.

[0145] If the target object is stationary but located within STANDING_OBSTACLE_MM of the wearer, it may be notified of a “front block” state. In one embodiment, STANDING_OBSTACLE_MM is 700 mm.

[0146] Therefore, the device (100) can apply different risk assessment criteria when the wearer is walking and when the wearer is stopped.

[0147] 8. Consensus Decision and Last Stand

[0148] Referring to FIG. 4, the risk determination unit (143) can check whether the same risk determination is detected consecutively without immediately generating a general alarm when one risk candidate is detected.

[0149] If the same judgment is detected consecutively N or more times, the risk is confirmed. In one embodiment, CONSENSUS_N is 2 times.

[0150] During walking, the wearer's head vibrates up and down, causing the measurement direction of the distance measuring unit (110) to change repeatedly. In this case, in one measurement, the object may appear to be approaching, and in the next measurement, it may appear to be moving away.

[0151] Requiring continuity of the same risk assessment can suppress false alarms caused by walking vibrations in which the direction is repeatedly reversed.

[0152] However, if the target object is located within HARD_STOP_MM, an alarm is generated immediately regardless of the consensus judgment or image fusion result. In one embodiment, HARD_STOP_MM is 600 mm.

[0153] That is, even if the image fusion judgment unit (144) determines that the object is to be bypassed, if the target object enters within the final defense line, the image-based alarm suppression can be released.

[0154] In addition, the danger determination unit (143) can generate a warning at WARN_DIST_MM regardless of speed if it determines that the target object is located in the collision path. In one embodiment, WARN_DIST_MM is 1,200 mm.

[0155] After the alarm occurs, the alarm state is maintained for HAZARD_HOLD_MS. In one embodiment, HAZARD_HOLD_MS is 800 ms.

[0156] Accordingly, even if the target object stops for a moment and the approach speed r becomes instantaneously zero, the alarm is immediately released, preventing intermittent flashing.

[0157] 9. Image Fusion Determination Using Single Image Blobs

[0158] Referring to FIG. 5, the image fusion judgment unit (144) sets a central region of interest (510) among the 160×120 pixel grayscale images acquired by the image acquisition unit (130).

[0159] The region of interest (510) can be set to about 60% of the image width to correspond to the measurement cone of the distance measuring unit (110).

[0160] The image fusion judgment unit (144) can extract a region with a large contrast with the background in the region of interest (510) as a blob candidate and select a single image blob (520) with the largest area among multiple blob candidates.

[0161] A single image blob (520) can be used as a valid image basis only when its area is greater than or equal to FUSION_MIN_AREA. Accordingly, small noise or lighting changes can be prevented from being identified as the target object.

[0162] The image fusion judgment unit (144) tracks the change in center coordinates of a single image blob (520) in consecutive frames and calculates the lateral movement speed.

[0163] If the lateral movement speed exceeds CAM_CROSS_PX_S, it can be determined that the target object crosses the wearer's path of movement or deviates from the frontal collision path. In one embodiment, CAM_CROSS_PX_S is 15 pixels / s.

[0164] If the target object is determined to be a bypassed object, the danger judgment unit (143) may suppress the alarm. However, if the target object is located within the final defense line, the suppression of the alarm may not be allowed.

[0165] The image fusion judgment unit (144) also calculates the center y-coordinate of a single image blob (520). The closer the center of the blob is to the top of the image, the more the target object can be determined to be located at the height of the wearer's head.

[0166] In one embodiment, if the center y-coordinate is smaller than 40 pixels out of a total of 120 pixels, it can be determined as a head height obstacle.

[0167] 10. Compensation for the wearer's own head rotation

[0168] The inertial measuring unit (120) measures the angular velocity in the yaw direction relative to the wearer's head.

[0169] When the wearer turns their head left or right, even a stationary object appears to be moving laterally within the video. In this case, if only simple blob tracking is applied, a frontal obstacle may be misidentified as an object moving away, potentially suppressing the alarm.

[0170] The image fusion determination unit (144) determines whether the yaw angular velocity exceeds CAM_YAW_MAX_DPS. In one embodiment, CAM_YAW_MAX_DPS is 15° / s.

[0171] Lateral movement information of the blob acquired while the yaw angular velocity exceeds the above reference value is excluded from the basis for alarm suppression.

[0172] However, information that is relatively less affected by head rotation, such as the presence, area, or height of the blob, may continue to be used. Accordingly, false alarms caused by the wearer's self-rotation can be prevented.

[0173] 11. Head posture determination by gravity vector projection

[0174] Referring to FIG. 6(a), the posture determination unit (145) acquires the front gravity vector vf and the downward gravity vector vdown during the calibration process.

[0175] The frontal gravity vector vf may be the average value of multiple acceleration vectors measured by the inertial measurement unit (120) while the wearer is standing still and looking straight ahead.

[0176] The tilting gravity vector vdown may be the average value of multiple acceleration vectors measured while the wearer tilts their head down to the point where they are looking at the ground.

[0177] The posture determination unit (145) calculates the bending direction vector d as follows.

[0178] [Mathematical Formula 3]

[0179] d = vdown - vf

[0180] Then, the difference between the current acceleration vector a and the frontal gravity vector vf is projected onto the bending direction vector d to calculate the attitude projection value P.

[0181] [Mathematical Formula 4]

[0182] P = {(a - vf)·d} / |d|²

[0183] The posture projection value P approaches 0 in the frontal posture and can approach 1 in the calibrated bent-over posture.

[0184] The attitude determination unit (145) switches from the front sub-mode to the ground monitoring sub-mode when P exceeds the first attitude reference value, and returns to the front sub-mode when P decreases to less than the second attitude reference value.

[0185] The second posture reference value is set to be smaller than the first posture reference value. In one embodiment, the first posture reference value may be 0.6 and the second posture reference value may be 0.35.

[0186] Applying different entry and return criteria can prevent the two modes from switching repeatedly at short intervals in the alert stance.

[0187] Since the above projection method does not directly designate a specific sensor axis as the pitch axis, the attitude can be determined based on the calibrated actual tilting direction even if the axial direction or sign of the inertial sensor changes.

[0188] 12. Geometric Classification of Ground Hazards

[0189] Referring to FIG. 6(b), in the ground monitoring submode, the measurement direction of the distance measuring unit (110) is directed toward the front ground.

[0190] The ground hazard assessment unit (146) can calculate the head tilt angle θ from the angle between the front gravity vector vf and the current acceleration vector a.

[0191] [Mathematical Formula 5]

[0192] θ = arccos{(a·vf) / (|a||vf|)}

[0193] The vertical drop h is calculated using the distance d to the ground measured by the distance measuring unit (110) and the head tilt angle θ.

[0194] [Mathematical Formula 6]

[0195] h = d·sinθ

[0196] The ground hazard determination unit (146) can set a ground baseline using the first FLOOR_BASE_N normal ground samples after entering the ground monitoring submode. In one embodiment, FLOOR_BASE_N is 8.

[0197] If the current vertical drop is closer than the baseline, it can be determined that a ledge, ascending stairs, or an object on the floor is present.

[0198] If the deviation from the baseline exceeds FLOOR_OBJ_MM, it is determined to be an object on the floor, and if it is less than or equal to it, it can be determined to be a ledge or a step. In one embodiment, FLOOR_OBJ_MM is 210 mm.

[0199] Currently, if the vertical drop is more than FLOOR_HOLE_MM away from the baseline, it can be determined as a hole or a steep drop. In one embodiment, FLOOR_HOLE_MM is 550 mm.

[0200] Even if the distance measuring unit (110) fails to acquire valid reflection values ​​FLOOR_NONE_N times consecutively, it can be determined that there is empty space or a hole in front. In one embodiment, FLOOR_NONE_N is 4 times.

[0201] If the change in vertical drop between consecutive samples exceeds FLOOR_JUMP_MM and changes downwards, it can be determined as the beginning of a wide pit. In one embodiment, FLOOR_JUMP_MM is 250 mm.

[0202] The ground risk determination unit (146) can determine the ground risk only when the same risk is determined in K or more of the recent FLOOR_VOTE_N samples. In one embodiment, it may require three or more matches among the recent five samples.

[0203] 13. Recalibration by head shaking

[0204] Referring to FIG. 7(a), the calibration unit (147) projects the angular velocity of the inertial measurement unit (120) onto the current gravity axis to calculate the yaw direction angular velocity component of the head.

[0205] It is determined whether the magnitude of the above yaw direction angular velocity component is greater than or equal to SHAKE_YAW_DPS. In one embodiment, SHAKE_YAW_DPS is 100° / s.

[0206] The calibration unit (147) calculates the number of times the sign of the yaw direction angular velocity is alternately switched within the time window corresponding to SHAKE_WINDOW_MS. In one embodiment, the time window is 1,600 ms, and if the number of direction changes is 5 or more, it can be determined as a recalibration gesture.

[0207] Since it utilizes the yaw component projected onto the axis of gravity, it is possible to distinguish between head-nodding motions, walking vibrations, and left-right shaking motions.

[0208] When a recalibration gesture is detected, the calibration unit (147) may initiate a procedure to reacquire the front gravity vector or the tilting gravity vector. The start and completion of the calibration may be notified by a separate flashing or voice signal from the notification unit (160).

[0209] 14. Wear detection-based automatic calibration

[0210] Referring to FIG. 7(b), the automatic correction unit (149) can automatically set the front gravity vector based on the wearing act itself without visual confirmation or assistance from others.

[0211] The automatic correction unit (149) first determines whether a preceding movement has occurred in which the amount of change in acceleration exceeds a first movement reference value. In one embodiment, the first movement reference value is 0.06 g.

[0212] Preceding movement may occur during the process of the device being lifted from a desk or mounted on the wearer's head.

[0213] After a preceding movement is detected, it is determined whether a state in which the amount of change in acceleration is less than a second movement threshold value is maintained continuously for a certain period of time. In one embodiment, the second movement threshold value is 0.03 g, and the continuous stopping time is about 3 seconds.

[0214] If a continuous stationary state is confirmed, the average value of the acceleration vector measured in that section is temporarily set as the frontal gravity vector.

[0215] If only a stop is detected without any preceding movement, the front gravity vector may not be set because there is a possibility that the device is not worn and is placed on the floor or a table.

[0216] After the frontal gravity vector is temporarily set, the walking speed estimation unit (141) determines whether continuous walking is detected. In one embodiment, if continuous walking is confirmed for about 1.8 seconds, the correction for the frontal gravity vector is finalized.

[0217] If walking is interrupted midway, the walking confirmation count can be decreased or reset instead of being immediately confirmed. This prevents movements while holding the device in the hand from being mistaken for actual walking after wearing it.

[0218] 15. Neural Network-Based Supplemental Ground Risk Assessment

[0219] Referring to Fig. 8, the distance and inertia-based safety judgment channel and the neural network-based ground risk supplement channel can operate with independent judgment cycles.

[0220] The distance and inertia-based safety determination channel determines the risk of collision with a forward object in real time using a distance measuring unit (110) and an inertia measuring unit (120).

[0221] The neural network ground hazard determination unit (148) inputs the forward ground image acquired from the image acquisition unit (130) into the lightweight image classification neural network.

[0222] In one embodiment, the image classification neural network is an int8 quantization model based on MobileNetV1 0.25 and can classify four classes including normal walkways, stairs, sinkholes, and other conditions.

[0223] The neural network inference cycle can be set to CNN_POLL_MS. In one embodiment, CNN_POLL_MS is 250 ms, the inference time per iteration is about 44 ms, and the operation duty may be about 18%.

[0224] Risk can be confirmed only when the same risk is detected in two or more frames out of the last three frames and the classification confidence is greater than or equal to CNN_MIN_CONF. In one embodiment, CNN_MIN_CONF is 0.50.

[0225] According to the embodiment, actual notification is limited to the stair class, and the sinkhole class is available only for observation or recording. This is intended to reduce indoor false alarms of the sinkhole class.

[0226] If the model of the image classification neural network does not exist, or if model loading fails or an exception occurs during inference, only the neural network ground risk judgment unit (148) is deactivated.

[0227] Even in this case, the distance- and inertia-based safety judgment channel continues to operate, so the failure of the supplementary channel does not stop the basic forward collision detection function.

[0228] The image classification neural network can remain loaded in memory even when inactive, and if activated via remote settings, it can immediately resume inference without rebooting.

[0229] Immediately after neural network inference, temporary memory can be reclaimed to suppress fragmentation of limited heap memory.

[0230] 16. Sense-first short-range communication

[0231] The short-range communication unit (170) can communicate directly with an external management terminal (200) using BLE.

[0232] In the reception interrupt of short-range communication, only the received bytes are accumulated in the buffer, and the interpretation and application of the command can be performed in the main detection loop of the control unit (140).

[0233] The control unit (140) can process only one command line or a limited amount of data per detection loop so that communication processing does not excessively occupy the sensor judgment cycle.

[0234] The status information may be transmitted in a request-type manner, responding once when there is a request from an external management terminal (200), rather than being transmitted continuously at regular intervals. In one embodiment, the periodic status transmission interval, BLE_STATE_MS, is set to 0.

[0235] Large amounts of data can be transmitted in multiple small chunks in accordance with the BLE connection cycle. In one embodiment, one chunk is 20 bytes and two chunks can be transmitted per connection cycle.

[0236] If transmission fails or throughput is exceeded, the corresponding data can be discarded without performing infinite retries to prevent delays in the detection loop.

[0237] When a remote configuration value is received, the existence of the configuration key, data format, and safe lower limit are verified. For example, a command to set the CNN decision cycle to less than 250 ms can be rejected.

[0238] Settings can be stored in a hierarchy of factory settings, administrator defaults, and user adjustments. Accordingly, when restoring defaults, the settings may be restored to the reference values ​​set by the administrator rather than the absolute values ​​from the manufacturing stage.

[0239] 17. Verified Wireless Software Update

[0240] The external management terminal (200) can query the file-specific checksum and file size stored in the device (100).

[0241] The external management terminal (200) can compare the file stored in the device with the file to be updated and selectively transmit only the changed file.

[0242] The transmitted file data may consist of multiple fragments in base64 format and is written to a temporary file in a non-volatile storage unit (180).

[0243] The control unit (140) does not directly modify the source file to be updated, but verifies the size and checksum of the temporary file after all file fragments have been received.

[0244] Atomic replacement is performed to change the name of the temporary file to the name of the original file only when verification is complete.

[0245] In the event of a verification failure or interruption of communication, temporary files are deleted and the original files are preserved. This prevents the device from becoming inoperable due to a software update failure.

[0246] You can check if the filename contains path escape characters or filenames with restricted access.

[0247] During the update, the notification unit (160) flashes blue at a constant rate, and after the update is completed, the device (100) can be automatically restarted.

[0248] 18. Verification of owner within the device

[0249] Referring to FIG. 9, when an external management terminal (200) requests initial registration for the device (100), the owner verification unit (150) receives the account identifier of the external management terminal (200).

[0250] The owner verification unit (150) stores the account identifier as owner information in the non-volatile storage unit (180). The owner information may be stored in the form of an original text, an encrypted value, a hash value, or an authentication token.

[0251] Subsequently, when a destructive command such as factory initialization or release of ownership is received, the owner verification unit (150) extracts the requester identifier included in the command.

[0252] The execution of a destructive command is allowed only when the requester identifier matches the owner information stored in the non-volatile storage unit (180).

[0253] If the requester identifier does not match or does not include identification information, the command is blocked and the existing owner information is retained.

[0254] The above owner verification can be performed within the device (100) without connecting to an external server. Therefore, even when the network is disconnected, it is possible to prevent hijacking in which a third party initializes the device and registers it to their own account.

[0255] 19. Check navigation cooperation mode and turning entry points

[0256] Referring to FIG. 10, the external guidance terminal (300) may be a walking path guidance application installed on a smartphone.

[0257] When an external guidance terminal (300) requests the execution of a route guidance mode, the guidance mode control unit (151) stops at least one of a forward collision judgment and a blob-based image fusion judgment.

[0258] This is to prevent the alarm sound of the notification unit (160) and the rotation guidance voice of the external guidance terminal (300) from being output simultaneously, and to allocate the computational resources of the image acquisition unit (130) to the path guidance.

[0259] However, walking state, stationary state, fall detection, and short-range communication can be maintained even in navigation mode.

[0260] The external guidance terminal (300) transmits advance information including the direction of rotation and the expected location to the device (100) when the wearer approaches a point where the wearer is scheduled to turn left, turn right, or cross.

[0261] When the guidance mode control unit (151) receives advance information, it acquires a forward image at a constant interval using the image acquisition unit (130). In one embodiment, the image acquisition interval is 5 seconds.

[0262] The front image may be processed directly in the device (100) or transmitted to an external image judgment device (400).

[0263] The external image judgment device (400) can determine whether an intersection, corner, crosswalk, or entry point in the corresponding direction is located within a few steps from the forward image, as yes or no.

[0264] If the rotation entry point is not reached, periodic image acquisition continues, and if the rotation entry point is reached, arrival information is transmitted to an external guidance terminal (300).

[0265] The external guidance terminal (300) can output specific guidance for turning left, turning right, or crossing at the time it receives arrival information.

[0266] Accordingly, the longitudinal position error between the expected rotation position calculated by GPS and the actual rotation entry point can be corrected using the wearer's head view image.

[0267] When the rotation entry point is reached, the notification unit (160) may flash blue twice to provide non-visual or auxiliary arrival feedback.

[0268] In addition, upon request from the external guidance terminal (300), the image acquisition unit (130) can take a picture of the front image, and the external image judgment device (400) can determine and provide a crosswalk, stairs, construction zone, or barrier a few steps ahead in one sentence.

[0269] 20. Fall Detection and Priority Processing

[0270] The control unit (140) can detect a free fall section in which the magnitude of the acceleration of the inertia measuring unit (120) decreases to less than FALL_FREEFALL_G. In one embodiment, FALL_FREEFALL_G is 0.4 g.

[0271] If an impact with an acceleration magnitude exceeding FALL_IMPACT_G is detected within approximately 500 ms after the free fall section is detected, it can be determined as a fall. In one embodiment, FALL_IMPACT_G is 2.5 g.

[0272] Fall detection can be processed with priority over at least some of walking mode, ground monitoring mode, navigation mode, assistant mode, and power saving mode.

[0273] When a fall is detected, the notification unit (160) may alternately display red and blue during FALL_LED_MS and output voice or vibration. In one embodiment, FALL_LED_MS is 5,000 ms.

[0274] The fall detection function can be disabled depending on the user's choice.

[0275] 21. Resource Management and Detection Priority Log

[0276] The control unit (140) can perform garbage collection when the increase in available heaps exceeds a preset rate in order to perform image and neural network processing in limited memory.

[0277] In one embodiment, garbage collection is performed based on an increase amount corresponding to about 25% of the available heap, and a separate immediate memory reclamation can be performed immediately after neural network inference.

[0278] Modules that are not directly required for pedestrian safety judgment, such as assistant functions, may be delayed-loaded at the time of initial entry into the mode.

[0279] Since the task of recording walking history or state history in the non-volatile storage unit (180) can cause a delay of tens of milliseconds, it can be performed first during the period when the wearer is stopped.

[0280] In cases where continuous walking persists for a long time, recording can be performed only when the maximum delay time is reached to prevent recording omissions. By prioritizing the detection loop over flash recording in this way, delays in responding to danger during walking can be prevented.

[0281] 22. Output pattern of the notification unit

[0282] The notification unit (160) can use different output patterns to distinguish between the danger state and the device state.

[0283] In a safe state while walking, a faint green indicator can be displayed, and in a stopped state, a faint white indicator can be displayed.

[0284] In the early warning state, the red light flashes at a cycle of about 400 ms, and in the final warning state, the red light flashes rapidly at a cycle of about 70 ms.

[0285] Among the hazards on the ground, ledges or stairs can be notified by red, objects on the floor by yellow, and holes or steep drops by flashing white.

[0286] During software updates, the blue light may blink at a cycle of approximately 300 ms. When the remote settings are successfully applied, the blue light may blink 5 times to notify the wearer, who may have difficulty seeing the screen, that the application is complete.

[0287] The same voice guidance can limit repeated output during CUE_COOLDOWN_MS. In one embodiment, CUE_COOLDOWN_MS is 3,000 ms.

[0288] The above colors and flashing cycles are examples only and can be changed to vibration patterns, voice, bone conduction output, or a combination thereof.

[0289] 23. Control Method

[0290] A control method for a wearable walking safety assist device according to one embodiment of the present invention may include the steps of measuring inertia information, detecting footstep impact candidates, determining the continuity of a walking rhythm, and calculating the walking speed of the wearer.

[0291] In parallel with this, the distance to the target object is repeatedly measured, and the approach speed of the target object can be calculated from the change in distance over a certain observation interval.

[0292] A normalized approach ratio is calculated by normalizing the approach speed of the target object by the wearer's walking speed, and the target object can be classified as a companion object, a stationary object, or an opposing moving object based on the normalized approach ratio.

[0293] Subsequently, whether to issue an alarm can be determined based on at least one of the classification result of the target object, the distance to the target object, the net approach amount, the characteristics of the image blob, and the continuity of the same judgment.

[0294] When the wearer lowers their head, the system switches to ground monitoring mode based on gravity vector projection and can classify ground hazards based on the measured distance and the angle of inclination.

[0295] Ground hazards located in the blind spot of the distance sensor can be determined by supplementing with an independent periodic image classification neural network.

[0296] Each of the above steps does not necessarily need to be performed in the order described, and multiple steps may be performed in parallel or some steps may be omitted. Additionally, each step may be implemented in hardware, software, or a combination thereof. Explanation of the symbols

[0298] 100. Wearable walking safety assist device

Claims

Claim 1 A wearable walking safety assist device using walking speed normalization mounted on the head of a wearer and a control method thereof, comprising: a distance measuring unit that repeatedly measures the distance to a target object located in front of the wearer; an inertial measuring unit that measures acceleration and angular velocity according to the movement of the wearer; a notification unit that notifies the wearer of danger regarding the target object; and a control unit connected to the distance measuring unit, the inertial measuring unit, and the notification unit; wherein the control unit comprises: a walking speed estimation unit that determines whether the wearer is walking based on inertial information measured by the inertial measuring unit and calculates the walking speed of the wearer if the wearer is walking; and an approach speed calculation unit that calculates the approach speed of the target object based on changes in the distance to the target object measured at different points in time by the distance measuring unit. A wearable walking safety assist device utilizing walking speed normalization, characterized by comprising: a risk determination unit that calculates a normalized approach ratio by normalizing the approach speed of the target object based on the walking speed of the wearer, classifies the target object into one of a companion object moving with the wearer, a stationary object stationary relative to the surrounding environment, and an opposing moving object moving toward the wearer according to any one of a plurality of standard ranges to which the normalized approach ratio belongs, and determines whether to generate an alarm through the notification unit according to the classification result of the target object. Claim 2 A wearable walking safety assist device using walking speed normalization, characterized in that, in claim 1, the risk determination unit determines to generate an alarm through the notification unit when the same risk determination result for the target object is generated consecutively more than a preset number of times, and when the target object is located within a preset nearest distance, it determines to generate the alarm regardless of whether the same risk determination result is generated consecutively, and after the alarm is generated, controls the notification unit to maintain the alarm for a preset maintenance time. Claim 3 A wearable walking safety assist device using walking speed normalization, characterized in that, in claim 1, it further includes an image acquisition unit for acquiring a front image of the wearer, and the control unit further includes an image fusion determination unit, wherein the image fusion determination unit extracts a single image blob that is distinguished from the background in a region of interest corresponding to the measurement area of ​​the distance measuring unit among the front images acquired by the image acquisition unit, calculates the lateral movement speed relative to the center of the single image blob, the height of the center on the screen, and the area of ​​the single image blob, and if the area of ​​the single image blob is greater than or equal to a preset area reference value, the single image blob is set as a valid image basis, and if the lateral movement speed exceeds a preset movement speed reference value, the target object is determined to be an object that crosses the wearer's path of travel or deviates from the path of travel, thereby suppressing an alarm by the danger determination unit, and if the center is located above a preset height on the screen, the target object is determined to be an obstacle located at the height of the wearer's head. Claim 4 A wearable walking safety assist device using walking speed normalization according to claim 3, wherein the image fusion judgment unit calculates the yaw direction angular velocity for the wearer's head based on the angular velocity measured by the inertial measurement unit, and in the section where the yaw direction angular velocity exceeds a preset angular velocity reference value, the lateral movement speed calculated in that section is excluded from the image basis for determining whether to suppress the alarm in order to prevent the lateral movement of the single image blob caused by the rotation of the wearer's head from being judged as the lateral movement of the target object itself. Claim 5 A wearable walking safety assist device using walking speed normalization, wherein, in claim 1, the control unit further includes a posture determination unit for determining the head posture of the wearer, and the posture determination unit acquires a frontal gravity vector measured by the inertial measurement unit while the wearer is facing forward during the calibration process and a tilting gravity vector measured while the wearer is tilting their head, sets a tilting direction vector based on the difference between the tilting gravity vector and the frontal gravity vector, calculates a posture projection value by projecting the difference between the current acceleration vector measured by the inertial measurement unit and the frontal gravity vector onto the tilting direction vector, switches from a frontal monitoring mode that monitors the front of the wearer to a ground monitoring mode that monitors the ground when the posture projection value exceeds a first posture reference value, and switches from the ground monitoring mode to the frontal monitoring mode when the posture projection value decreases to less than a second posture reference value that is smaller than the first posture reference value. Claim 6 A wearable walking safety assist device using walking speed normalization, wherein, in claim 5, the control unit further includes a ground hazard determination unit, and the ground hazard determination unit calculates a vertical drop based on a head tilt angle corresponding to the angle between the frontal gravity vector and the current acceleration vector and the measured distance to the ground measured by the distance measuring unit in the ground monitoring mode, classifies the terrain in front of the wearer into one of a chin, stairs, an object on the floor, or a hole according to the direction and magnitude of deviation between the vertical drop and a preset ground reference line, and controls the notification unit so that an alarm corresponding to the classification result of the terrain in front is output. Claim 7 In claim 5, the control unit further includes a calibration unit that controls the calibration, wherein the calibration unit calculates the yaw direction angular velocity component for the wearer's head by projecting the angular velocity measured by the inertial measurement unit onto the current gravity axis, and when the yaw direction angular velocity component exceeds a preset gesture reference value and the direction of the yaw direction angular velocity component alternates more than a preset number of times within a preset time interval, it is determined that the wearer has performed a recalibration gesture of shaking the head left and right, and when it is determined that the recalibration gesture has been performed, the calibration is performed to reset at least one of the front gravity vector and the tilting direction vector. Claim 8 A wearable walking safety assist device using walking speed normalization mounted on the head of a wearer and a control method thereof, comprising: a step of measuring the acceleration and angular velocity of the wearer through an inertial measurement unit; a step of detecting a point in time when the vibration component, from which the gravitational component has been removed from the acceleration, exceeds a preset impact threshold value as a step impact candidate; a step of determining that the wearer is walking if a state in which the time interval between the step impact candidates falls within a preset walking rhythm range is detected continuously for more than a preset number of times, and calculating the walking speed of the wearer based on the time interval between the step impact candidates; a step of repeatedly measuring the distance to a target object located in front of the wearer through a distance measurement unit; a step of calculating the approach speed of the target object based on the change in the distance to the target object measured at different points in time; a step of, when the wearer is walking, calculating a normalized approach ratio by normalizing the approach speed of the target object based on the wearer's walking speed; and, according to one of a plurality of reference ranges to which the normalized approach ratio belongs, classifying the target object into one of a companion object moving with the wearer, a stationary object stationary relative to the surrounding environment, and an opposing moving object moving toward the wearer. A method for controlling a wearable walking safety aid using walking speed normalization, characterized by comprising: a step of classifying; and a step of notifying the wearer of a danger to the target object according to the classification result of the target object. Claim 9 A wearable walking safety assist device using walking speed normalization, characterized in that, in claim 1, it further includes an image acquisition unit that acquires a front image including the ground in front of the wearer, and the control unit further includes a neural network ground hazard determination unit, wherein the neural network ground hazard determination unit classifies the dangerous terrain of the front ground by inputting the front image into an image classification neural network according to an image determination cycle set independently of the distance measurement cycle by the distance measurement unit, and controls the notification unit to output an alarm for the dangerous terrain only when a number of image frames or more among a recently preset number of image frames are classified as the same dangerous terrain and the classification reliability for the same dangerous terrain is greater than or equal to a preset reliability threshold value, and if the model of the image classification neural network does not exist or the loading of the model fails, it deactivates the dangerous terrain classification function using the image classification neural network and continues to perform distance and inertia-based danger determination by the walking speed estimation unit, the approach speed calculation unit, and the danger determination unit. Claim 10 In claim 5, the control unit further includes an automatic correction unit that performs the calibration based on wear detection, and the automatic correction unit is characterized by: after a preceding movement in which the amount of change in acceleration measured by the inertial measurement unit exceeds a first movement reference value is detected, if a stationary state in which the amount of change in acceleration is less than a second movement reference value that is smaller than the first movement reference value is continuously maintained for a preset first time period, the automatic correction unit temporarily sets the frontal gravity vector based on the acceleration measured in the stationary state, and after the frontal gravity vector is temporarily set, if a walking state is continuously detected by the walking speed estimation unit for a preset second time period, the automatic correction unit confirms the correction for the frontal gravity vector.

Images