Walking assistance guidance output method and obstacle detection glasses using same

Abstract

Obstacle detection glasses according to the present invention comprise: a distance sensor for acquiring distance information of an obstacle located in front of the user; a vision sensor for acquiring image information of the forward view; a processor for analyzing the forward situation on the basis of at least one of the distance information and the image information to generate walking guidance information; and a speaker for outputting the walking guidance information, wherein the processor detects an obstacle located within a preset distance on the basis of at least one of the distance information and the image information, sets a corresponding tracking mode on the basis of information on the detected obstacle, and outputs the walking guidance information to a pedestrian on the basis of the set tracking mode.

Description

Method for outputting walking assistance guidance and obstacle detection glasses using the same

[0001] The present invention relates to smart glasses, and more specifically, to obstacle detection glasses that provide a method for outputting walking assistance guidance.

[0002] When individuals with low vision or the visual impairment, whose visual function is significantly lower or completely lost compared to normal people, walk outdoors, there is a high risk of injury from falls due to their inability to identify objects in their path, leading to collisions, or when the terrain of the walking path changes, such as on uphill or downhill slopes.

[0003] Consequently, while it is difficult for visually impaired individuals to walk outdoors without the assistance of a guardian or a guide dog, it is practically impossible for every visually impaired person to be accompanied by a guardian or guide dog whenever they go out. To address these issues, devices have been developed to assist the walking of the visually impaired. Specifically, such devices are generally equipped with a camera that identifies objects from image data captured by the camera and provides information about the identified objects to the visually impaired person via voice output.

[0004] However, conventional methods simply identify objects in the direction of gaze of the visually impaired person through image data, and cannot determine whether the object is in the direction of walking of the visually impaired person or identify the distance between the object and the visually impaired person.

[0005] Therefore, there is a growing need for a device that provides walking assistance information to visually impaired and low-vision individuals, identifies objects in the pedestrian's walking direction, identifies the distance to objects to provide information on the possibility of collision, and identifies the shape of the pedestrian walkway to provide this information to the visually impaired.

[0006] The present invention aims to provide walking assistance guidance based on a distance sensor and a vision sensor.

[0007] In addition, the present invention aims to provide walking assistance guidance suitable for pedestrians by switching to a step-by-step driving mode suitable for the walking situation ahead.

[0008] The problems to be solved by the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below.

[0009] For the above purpose, the present invention includes a distance sensor for acquiring distance information of an obstacle located in front of obstacle detection glasses according to one embodiment of the present invention; a vision sensor for acquiring image information of the front; a processor for analyzing the situation in front and generating walking guidance information based on at least one of the distance information and the image information; and a speaker for outputting the walking guidance information, wherein the processor detects an obstacle located within a preset distance based on at least one of the distance information and the image information, sets a corresponding tracking mode based on the detected obstacle information, and outputs the walking guidance information to a pedestrian based on the set tracking mode.

[0010] According to one embodiment, the tracking mode includes a tracking off mode that disables the distance sensor and the vision sensor; a first tracking mode that enables the distance sensor; a second tracking mode that enables the vision sensor; and a third tracking mode that enables the distance sensor and the vision sensor.

[0011] According to one embodiment, when the first tracking mode is active, the processor detects an obstacle within a guidance distance from the pedestrian based on the distance sensor information and determines whether there is an obstacle.

[0012] According to one embodiment, when the second tracking mode is active, the processor determines the direction of the obstacle based on the vision sensor information.

[0013] According to one embodiment, when the third tracking mode is active, the processor determines the risk to the pedestrian based on the location and degree of approach of the obstacle.

[0014] According to one embodiment, the processor controls the vision sensor based on the distance sensor information.

[0015] According to one embodiment, if the processor detects the obstacle at a distance greater than the guidance distance in the second tracking mode, the vision sensor is disabled or controlled to a low frame state.

[0016] According to one embodiment, when the processor detects the obstacle at a distance closer than the guidance distance in the second tracking mode, the vision sensor is controlled to a high frame state.

[0017] According to one embodiment, the system further includes an inertial measurement unit for determining the walking direction, walking speed, and head direction of the pedestrian, and the processor analyzes the forward situation based on the distance sensor information and the walking direction, walking speed, and head direction of the pedestrian, determines whether there is a walking risk factor based on the analyzed forward situation, and if the walking risk factor is present, activates the vision sensor.

[0018] According to one embodiment of the present invention, a driving mode suitable for pedestrians can be switched according to the condition and congestion level of the walking section to provide a guidance level suitable for pedestrians.

[0019] In addition, depending on the driving mode, the sensor can be operated in a low-power mode to increase the usage time of the device.

[0020] Further scopes of the applicability of the present invention will become apparent from the following detailed description. However, since various changes and modifications within the spirit and scope of the present invention are clearly understood by those skilled in the art, specific embodiments, such as the detailed description and preferred embodiments of the present invention, should be understood as being given merely as examples.

[0021] FIG. 1 is a perspective view schematically showing obstacle detection glasses, which is an embodiment of the present invention.

[0022] FIG. 2 is a plan view showing a system of obstacle detection glasses, which is an embodiment of the invention.

[0023] FIGS. 3 and 4 are drawings illustrating a change in sensor driving according to the walking assistance driving step of obstacle detection glasses according to an embodiment of the present invention.

[0024] FIGS. 5 to 7 are drawings illustrating an obstacle distance-based tracking mode of obstacle detection glasses according to an embodiment of the present invention.

[0025] FIG. 8 is a flowchart illustrating a mode switching for sensor control based on distance from an obstacle according to an embodiment of the present invention.

[0026] FIG. 9 is a flowchart illustrating mode switching based on image analysis according to one embodiment of the present invention.

[0027] FIG. 10 is a diagram illustrating mode switching based on detection of pedestrian posture according to one embodiment of the present invention.

[0028] FIG. 11 is a flowchart illustrating mode switching based on detection of pedestrian posture according to one embodiment of the present invention.

[0029] FIG. 12 is a drawing for explaining a walking assistance guidance analysis module according to one embodiment of the present invention.

[0030] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols are given the same reference number, and redundant descriptions thereof will be omitted.

[0031] The suffixes "module" and "part" for components used in the following description are assigned or used interchangeably solely for the sake of ease of drafting the specification, and do not inherently possess distinct meanings or roles. Furthermore, in describing the embodiments disclosed in this specification, detailed descriptions of related prior art are omitted if it is determined that such detailed descriptions could obscure the essence of the embodiments disclosed in this specification.

[0032] In addition, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings; it should be understood that all modifications, equivalents, and substitutions included within the concept and technical scope of the present invention are included.

[0033] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0034] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0035] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0036] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0037] FIG. 1 is a perspective view schematically showing obstacle detection glasses according to one embodiment of the present invention, and FIG. 2 is a plan view showing a system of obstacle detection glasses according to one embodiment of the invention.

[0038] As shown in FIGS. 1 and 2, obstacle detection glasses of one embodiment of the present invention are equipped with a distance sensor (10), a vision sensor (20), a speaker (30), an inertial measurement unit (40), and a processor (60).

[0039] First, the distance sensor (10) may be positioned on one side of the front, which is spaced apart from the vision sensor (20). The distance sensor (10) may be a 1D sensor that acquires information on the presence or absence of obstacles in front and one-dimensional (1D) distance information.

[0040] For example, the distance sensor (10) can be made of various types, such as an ultrasonic sensor that detects obstacles by outputting ultrasonic waves, and in particular, it can include a LiDAR sensor that can predict the direction by observing the movement of people or animals, and can calculate and predict the location of static obstacles, dynamic obstacles, etc. at each point.

[0041] A vision sensor (20) may be placed on the other side of the front of the obstacle detection glasses. The vision sensor (20) may capture an image of the direction or object that the pedestrian is looking at. The vision sensor (20) may be a 2D sensor that acquires two-dimensional (2D) image information.

[0042] The vision sensor (20) can generate image data to identify objects located in the direction of a visually impaired person's head.

[0043] For example, the vision sensor (20) is based on a 2D RGB camera and has 3D obstacle detection and recognition capabilities, and can receive video input to guide road surface conditions, the direction of obstacles, and the direction of movement of pedestrians.

[0044] A speaker (30) is provided on the temple (13) of the obstacle detection glasses and can notify a visually impaired person of the presence or absence of an obstacle based on detection data from at least one of the vision sensor (20) and the distance sensor (10).

[0045] For example, the speaker (30) can be made of various types, and in particular, it can be made of bone conduction earphones such as bone conduction Bluetooth earphones that transmit vibrations corresponding to voice signals to the skull of a visually impaired person so that the pedestrian can recognize the sound.

[0046] The inertial measurement unit (40) may be provided on the temple of the obstacle detection glasses. The inertial measurement unit (40) may include an IMU (Inertial Measurement Unit) sensor.

[0047] The inertial measurement unit (40) can detect the movement of a pedestrian, such as walking state / speed. The inertial measurement unit (40) may be composed of a gyroscope sensor, a 9-axis accelerometer, etc., that detects the movement of obstacle detection glasses when a visually impaired person wearing obstacle detection glasses walks.

[0048] For example, the inertial measurement unit (40) measures movement in real time using a 9-axis sensor (Accel, Gyro, Magnetic) built into the obstacle detection glasses.

[0049] The first to fourth buttons (51, 52, 53, 54) may be provided on the temples of the obstacle detection glasses. The first button (51) may be placed on one temple of the obstacle detection glasses, and the second to fourth buttons (52, 53, 54) may be placed on the other temple of the obstacle detection glasses.

[0050] For example, the first button (51) is a button for controlling a device that is pre-set by turning the walking assistance guidance by the vision sensor (20) and the distance sensor (20) On / Off.

[0051] For example, the second to fourth buttons (52, 53, 54) may be buttons for pre-configured device control purposes, such as obstacle detection glasses full Poser buttons and volume control buttons.

[0052] The processor (60) can process image information through the vision sensor (20) and depth information (Depth) through the distance sensor (10) as inputs and control the speaker (30) to output walking guidance information to the pedestrian.

[0053] The processor (60) can control the obstacle detection glasses based on the operation of the first to fourth buttons (51, 52, 53, 54).

[0054] The processor (60) can determine the degree of proximity from the current pedestrian location by processing AI Depth estimation technology and distance sensor (10) information together based on the image received through the vision sensor (20).

[0055] The processor (60) is involved in the distance information of the distance sensor (10), the On / Off state of the vision sensor (20), and operating states such as the frame rate, and can manage the device power consumption.

[0056] The processor (60) can detect an obstacle located within a preset distance based on at least one of the distance information and the image information.

[0057] The processor (60) can set a corresponding tracking mode based on the detected obstacle information.

[0058] The tracking mode may include a tracking off mode that disables the distance sensor and the vision sensor; a first tracking mode that enables the distance sensor; a second tracking mode that enables the vision sensor; and a third tracking mode that enables the distance sensor and the vision sensor.

[0059] In the case of the first tracking mode, the processor (60) can detect obstacles within a guidance distance from the pedestrian based on the distance sensor information and determine whether there are obstacles.

[0060] The processor (60) can determine the direction of the obstacle based on the vision sensor information when the second tracking mode is active.

[0061] In the case of the third tracking mode, the processor (60) can determine the risk to the pedestrian based on the location and proximity of the obstacle.

[0062] Meanwhile, the processor (60) can control the vision sensor based on the distance sensor information.

[0063] If the processor (60) detects the obstacle at a distance greater than the guidance distance in the second tracking mode, it may disable the vision sensor or control it to a low frame state.

[0064] The processor (60) can control the vision sensor to a high frame state when it detects the obstacle at a distance closer than the guidance distance in the second tracking mode.

[0065] The processor (60) can output the walking guidance information to the pedestrian based on the set tracking mode.

[0066] FIGS. 3 and 4 are drawings illustrating a change in sensor driving according to the walking assistance driving step of obstacle detection glasses according to an embodiment of the present invention.

[0067] Referring to FIG. 3, the processor (60) switches the driving mode for walking assistance by internally determining the pedestrian's settings or the situation ahead. Specifically, the driving mode may include Tracking Off, Tracking Mode 1, Tracking Mode 2, and Tracking Mode N.

[0068] The processor (60) can selectively use sensors and the level of obstacle guidance varies depending on the driving mode. At this time, multiple levels of settings can be configured for the sensors, and the frame rate, etc., can be changed as needed.

[0069] The processor (60) can change the sensor drive based on the tracking stage classification.

[0070] For example, the processor (60) can disable all sensors with the walking assistance function turned off when in Tracking Off mode.

[0071] When the processor (60) is in Tracking Mode 1, only the distance sensor (10) is activated, and the vision sensor (20) is disabled (Off) or maintained at a low frame rate. When the processor (60) is in Tracking Mode 1, it determines only the presence or absence of an obstacle and can detect it at a certain distance.

[0072] When the processor (60) is in Tracking Mode 2, it uses only the vision sensor (20) and can optionally use a Vision AI model. When the processor (60) is in Tracking Off mode, it cannot distinguish obstacles but can provide direction guidance.

[0073] When the processor (60) is in Tracking Mode N, it can use both the 1D sensor and the vision sensor (20) to provide detailed walking guidance to the pedestrian.

[0074] Accordingly, the processor (60) can combine, in tracking mode, the distance sensor (10), the optional use of a Vision AI model, the optional notification of obstacle elements, etc.

[0075] Referring to FIG. 4, the processor (60) can switch the driving mode by internally determining the pedestrian's settings or the situation ahead. Depending on the mode, the processor (60) can change the level of guidance possible, such as determining the presence or absence of obstacles, the directionality of obstacles, the type of obstacle, and the distance of obstacles. Here, the driving mode distinction involves different processing steps, such as sensor control and the selection of an AI model suitable for the situation, and differences in system load, such as processing time, may occur.

[0076] The processor (60) can change the tracking mode based on the distance between the pedestrian and the obstacle. That is, the processor (60) can change the mode according to the preset distance between the pedestrian and the obstacle.

[0077] When the processor (60) detects an obstacle within a certain distance from the pedestrian, it can switch from Tracking Off mode to Tracking Mode 1. Tracking Mode 1 determines only the presence or absence of an obstacle and can determine only the approximate proximity of an obstacle, but cannot determine the direction.

[0078] The processor (60) can switch to the second tracking mode (Tracking Mode 2) when an obstacle is detected at a certain distance in the first tracking mode (Tracking Mode 1). The second tracking mode (Tracking Mode 2) cannot distinguish between obstacles but can recognize the directionality regarding the location of the obstacles.

[0079] The processor (60) may switch to the Nth Tracking Mode (Tracking Mode N) when it is closer to an obstacle than in the previous mode (e.g., the 2nd Tracking Mode (Tracking Mode 2). The Nth Tracking Mode (Tracking Mode N) can track the type of obstacle, the direction of proximity, and the distance.

[0080] FIGS. 5 to 7 are drawings illustrating an obstacle distance-based tracking mode of obstacle detection glasses according to an embodiment of the present invention.

[0081] Referring to FIG. 5, the processor (60) can operate multiple sensors by classifying them into a distance sensor low-power mode, a vision sensor low-spec mode, and a vision sensor (20) high-spec mode.

[0082] The distance sensor (10) uses a sensor capable of low-power operation such as ultrasound, and the vision sensor (20) uses an RGB camera, etc., and can distinguish obstacles through Vision AI technology and synthesize Depth estimation results to determine the location / distance of the obstacles.

[0083] The distance sensor's low-power mode can only determine the approximate proximity of an obstacle and may be a mode where the direction cannot be determined.

[0084] The low-spec mode of the vision sensor cannot distinguish obstacles but can provide directional guidance.

[0085] The high-performance mode of the vision sensor (20) can be a mode that can track the type of obstacle, proximity direction, and distance.

[0086] The processor (60) can switch modes depending on whether there is an obstacle based on the pedestrian reference distance.

[0087] The processor (60) can switch to a vision sensor low-spec mode if, as a result of algorithm recognition in the distance sensor low-power mode, the condition of maintaining an object within the guidance area (A) is satisfied.

[0088] The processor (60) can switch to a distance sensor low-power mode if there are no obstacles within the guidance area (A) for a certain period of time in the vision sensor low-spec mode.

[0089] The processor (60) can switch to the vision sensor (20) high-spec mode when there is an obstacle in the near-field guidance area (B) in the vision sensor low-spec mode.

[0090] The processor (60) can switch from the vision sensor (20) high-spec mode to the vision sensor low-spec mode when there are no obstacles in the near-field guidance area (B).

[0091] Referring to FIG. 6, the processor (60) can switch the tracking mode by internally determining the pedestrian's settings or the situation ahead.

[0092] For example, the processor (60) can switch the tracking mode by determining the obstacle based on the degree of proximity of the obstacle within close range, the direction, the degree of movement of the object requiring guidance, etc.

[0093] For example, the processor (60) can switch tracking modes by having the distance sensor or vision sensor operate independently or by synthesizing each sensor information to generate useful information.

[0094] Referring to FIG. 7, in Tracking Mode 1, when walking, only the distance sensor is activated to determine the presence or absence of an obstacle while maintaining a situation where there are no feature points in front or no obstacles within a certain distance. In Tracking Mode 1, the vision sensor may be deactivated or a low framerate may be maintained.

[0095] Tracking Mode 2 can analyze more in detail than Tracking Mode 1 by activating the Vision sensor and adjusting the frame rate when an obstacle is detected within a certain distance in Tracking Mode 1. In Tracking Mode 2, the AI ​​model level does not distinguish specific obstacles, but can determine the general direction in which the obstacle is located.

[0096] Tracking mode 3 can be switched when obstacle elements are closer than in Tracking mode 2.

[0097] Tracking mode 3 can run a vision-based AI obstacle classification and depth estimation model and can guide pedestrians to the specific location and proximity of obstacles.

[0098] Tracking mode 3 is used when there are many elements such as people and obstacles in front and the degree of proximity needs to be known in detail. It operates a 3D estimation model to determine the type and distance of obstacles in detail.

[0099] The processor (60) can control heterogeneous sensors and switch driving modes. The processor (60) can control the vision sensor (20) through distance information for low power management. For example, the processor (60) can perform controls such as disabling the vision sensor (20) and adjusting the frame rate.

[0100] FIG. 8 is a flowchart illustrating a mode switching for sensor control based on distance from an obstacle according to an embodiment of the present invention.

[0101] Referring to FIG. 8, the processor (60) can perform a tracking mode through the distance sensor (10) to switch driving modes between heterogeneous sensors.

[0102] When the processor (60) performs a tracking mode using a distance sensor as the initial operation, it can determine only whether there is an obstacle to guide the pedestrian in front of the distance sensor as the main (S101). And, the vision sensor (20) can be kept in an off or low frame state.

[0103] After step S101, the processor (60) can detect an obstacle within a certain area (S102).

[0104] After step S102, the processor (60) can determine whether it is in close proximity to an obstacle based on the detection result (S103).

[0105] After step S103, the processor (60) can receive the distance sensor (10) information again without switching modes if an obstacle is detected but is maintaining a certain distance from the pedestrian or is moving away, depending on the detection result.

[0106] The processor (60) can activate (turn on) the vision sensor (20) by switching from the distance sensor (10) usage mode to the vision sensor (20) tracking mode when it is in close proximity to an obstacle based on the detection result (S104). Additionally, the processor (60) can switch the vision sensor (20) to the High Frame tracking mode.

[0107] After step S104, the processor (60) can receive information from the vision sensor (20) (S105).

[0108] After step S105, the processor (60) can perform 3D image analysis and risk analysis based on the vision sensor (20) information (S106). To do this, the processor (60) can configure the forward information into 3D information through road surface recognition, obstacle recognition, and depth estimation engines within the vision sensor (20) information.

[0109] After step S106, the processor (60) can detect risk factors based on the results of 3D image analysis and risk analysis (S107).

[0110] After step S107, the processor (60) can disable (turn off) the vision sensor (20) if no risk factor is detected (S108). Specifically, if there is no risk factor for a certain period of time or longer, the processor (60) can disable the vision sensor (20) again or change it to a low-frame driving state and switch to a distance sensing mode.

[0111] Meanwhile, after step S107, if a risk factor is detected, the processor (60) can provide feedback on the analyzed situation (guide block, obstacle, deviation from direction, etc.) and output step-by-step feedback according to the risk level such as proximity (S109).

[0112] FIG. 9 is a flowchart illustrating mode switching based on image analysis according to one embodiment of the present invention.

[0113] Referring to FIG. 9, the processor (60) may have several analysis steps based on the Vision AI model type and may switch modes based on conditions such as whether there is an obstacle within a certain area based on pedestrians.

[0114] When the vision sensor (20) is activated (on) (S201), the processor (60) can operate in a vision sensor low-spec mode (S202).

[0115] After step S202, the processor (60) can analyze the 2D image (S203).

[0116] After step S203, the processor (60) can detect obstacles based on the 2D image analysis results (S204). Through this, the processor (60) cannot distinguish between obstacles, but can analyze key elements such as the direction of the obstacle and guide blocks.

[0117] After step S204, if an obstacle is detected based on the 2D image analysis result, the processor (60) can output guidance regarding the detected obstacle (S205).

[0118] After step S205, the processor (60) can determine whether an obstacle is located in close proximity after the guidance output (S206).

[0119] After step S206, the processor (60) can switch from 2D low-spec mode to 2D high-spec mode when an obstacle is located in close proximity (S207). When the processor is in 2D high-spec mode, it can distinguish the type of obstacle and track the proximity direction and distance.

[0120] After step s207, the processor (60) can analyze the 3D image (S208).

[0121] After step S208, the processor (60) can determine obstacles based on the 3D image analysis results (S209).

[0122] After step S209, if an obstacle is detected based on the detection result, the processor (60) can output guidance regarding the detected obstacle (S205).

[0123] After step S209, if no obstacle is detected based on the detection result, the processor (60) can switch from 2D high-spec mode to 2D low-spec mode (S202).

[0124] Meanwhile, after step S204, if no obstacle is detected based on the 2D image analysis result, the processor (60) can disable (turn off) the vision sensor (20) (S210).

[0125] FIG. 10 is a diagram illustrating mode switching based on detection of pedestrian posture according to one embodiment of the present invention.

[0126] When a pedestrian wearing obstacle detection glasses lowers their head on a flat surface, the distance sensor can perceive that the obstacle is getting closer. Additionally, an inertial measurement sensor can be utilized to determine the state of wearing, moving, or lowering the obstacle detection glasses.

[0127] Afterwards, the obstacle detection glasses receive information from the distance sensor (10) and inertial measurement sensor information, and can determine a dangerous situation by comprehensively judging the walking progress, the degree of bending, and the degree of proximity of the obstacle.

[0128] As shown in FIG. 10(a), when you lower your head on a flat surface, the distance sensor perceives the obstacle as getting closer.

[0129] As illustrated in FIG. 10(b), in the case of a downhill staircase or similar object in front, it can be determined that the head is lowered beyond a certain angle but the distance is not information or does not decrease.

[0130] The obstacle detection glasses can change the driving mode, such as activating the vision sensor (20) (on) through danger detection.

[0131] FIG. 11 is a flowchart illustrating mode switching based on detection of pedestrian posture according to one embodiment of the present invention.

[0132] Referring to FIG. 11, the distance sensor (10) can detect an obstacle in front of the pedestrian (S301).

[0133] The inertial measurement unit (40) can detect the state of wearing, moving, or bending of the obstacle detection glasses through the IMU sensor (S302).

[0134] The processor (60) can receive IMU detection information and distance sensor information to analyze walking progress / bending degree / closeness of objects (S303).

[0135] After step S303, the processor (60) can determine whether a risk factor is detected in front of the pedestrian (S304).

[0136] After step S303, the processor (60) can activate (On) the vision sensor (20) if a risk factor is detected (305).

[0137] Meanwhile, after step S303, if no risk factor is detected, the processor (60) can re-perform whether to detect a risk factor.

[0138] FIG. 12 is a drawing for explaining a walking assistance guidance analysis module according to one embodiment of the present invention.

[0139] Referring to FIG. 12, the signal recognition unit can acquire a signal from a one-dimensional distance sensor facing forward.

[0140] The video input unit can acquire images by using a wide-angle camera facing forward.

[0141] The inertial sensor can measure movement in real time using a 9-axis sensor (Accel, Gyro, Magnetic) embedded in the smart glasses.

[0142] The walking direction (angle) recognition unit can measure the user's walking direction and head direction (direction of smart glasses) in real time using an image or inertial sensor and calculate the difference between the two.

[0143] The walking speed extraction unit can estimate the user's walking speed using an inertial sensor. The walking speed extraction unit can vary the range of the guidance area using the estimated speed.

[0144] The guidance area calculation unit can set the guidance radius and guidance area using the information generated by the walking direction recognition and walking speed extraction units described earlier.

[0145] The road surface analysis engine can estimate the type of road surface using information received from the image.

[0146] The object analysis engine can estimate the type of object based on information received from the image. The object analysis engine can estimate whether an object is moving, and if so, in what direction and at what speed.

[0147] The depth estimation engine can analyze forward distance information using information obtained from 1D distance information and 2D images.

[0148] The sensor control unit can control the frame rate, sensor activation, and other sensor driving switching through the acquired signal and analysis results.

[0149] The risk analysis engine can determine the level of risk within the guidance area by comprehensively assessing whether the current road surface is walkable and whether there is or is approaching an object ahead.

[0150] The output interface can provide notifications to the user based on the determined risk level. The output interface can output notifications through various methods, such as sound (speaker) or vibration.

[0151] Meanwhile, the block diagrams disclosed in this disclosure may be interpreted by those skilled in the art as conceptual representations of circuits for implementing the principles of this disclosure. Similarly, it will be recognized by those skilled in the art that any flow chart, flow diagram, state transition diagram, pseudocode, etc., substantially represented on a computer-readable medium, represents various processes that can be executed by such computer or control unit, whether or not such computer or control unit is explicitly depicted.

[0152] Accordingly, the embodiments of the present disclosure described above can be written as a program that can be executed on a computer and can be implemented in a general-purpose digital computer that operates said program using a computer-readable recording medium. The computer-readable recording medium may include storage media such as magnetic storage media (e.g., ROM, floppy disk, hard disk, etc.) and optical reading media (e.g., CD-ROM, DVD, etc.).

[0153] The functions of the various elements illustrated in the drawings may be provided through the use of dedicated hardware as well as hardware capable of executing software in connection with appropriate software. When provided by a control unit, such functions may be provided by a single dedicated control unit, a single shared control unit, or multiple individual control units in which parts may be shared.

[0154] Additionally, the term “control unit” or any explicit use of “control unit” shall not be interpreted as exclusively referring to hardware capable of executing software, and may implicitly include, without limitation, digital signal control unit (DSP) hardware, read-only memory (ROM), random access memory (RAM) for storing software, and non-volatile storage devices.

[0155] It is obvious to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit and essential features of the invention.

[0156] The foregoing detailed description should not be interpreted restrictively in all respects and should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

[0157] Various embodiments for implementing the present invention have been described in detail in the previous table of contents.

[0158] Since the present invention is applicable to technology related to smart glasses, its industrial applicability is recognized.

Claims

1. A distance sensor that acquires distance information of an obstacle located in front; Vision sensor that acquires forward image information; A processor that analyzes the forward situation and generates walking guidance information based on at least one of the distance information and the image information; and It includes a speaker that outputs the above walking guidance information, and The above processor Detecting obstacles located within a preset distance based on at least one of the distance information and the image information, and Set a corresponding tracking mode based on detected obstacle information, and Characterized by outputting the above-mentioned walking guidance information to a pedestrian based on a set tracking mode. Obstacle detection glasses.

2. In Paragraph 1, The above tracking mode is Tracking off mode that disables the distance sensor and the vision sensor; A first tracking mode that activates the distance sensor; Activating the above vision sensor and the second tracking mode; and Characterized by including a third tracking mode that activates the distance sensor and vision sensor. Obstacle detection glasses.

3. In Paragraph 2, The above processor In the case of the above-mentioned first tracking mode, Characterized by detecting obstacles within a guidance distance from the pedestrian based on the distance sensor information and determining the presence or absence of obstacles. Obstacle detection glasses.

4. In Paragraph 3, The above processor In the case of the above second tracking mode, Characterized by determining the direction of the obstacle based on the vision sensor information. Obstacle detection glasses.

5. In Paragraph 4, The above processor In the case of the above third tracking mode, Characterized by determining the level of risk to the pedestrian based on the location and degree of approach of the obstacle. Obstacle detection glasses.

6. In Paragraph 5, The above processor Characterized by controlling the vision sensor based on the distance sensor information. Obstacle detection glasses.

7. In Paragraph 6, The above processor In the above second tracking mode, if the obstacle is detected at a distance greater than the guidance distance, Characterized by disabling the vision sensor or controlling it to a low frame rate state. Obstacle detection glasses.

8. In Paragraph 6, The above processor When the obstacle is detected at a distance closer than the guidance distance in the second tracking mode above, Characterized by controlling the above vision sensor in a high frame rate state Obstacle detection glasses.

9. In Paragraph 1, It further includes an inertial measurement unit that determines the walking direction, walking speed, and head direction of the pedestrian. The above processor Analyze the situation ahead based on the distance sensor information and the walking direction, walking speed, and the pedestrian's head direction, and Based on the analyzed forward situation above, determine whether there are pedestrian risk factors, and Characterized by activating the vision sensor when the above walking risk factor exists. Obstacle detection glasses.

10. A step of detecting an obstacle located within a preset distance based on at least one of distance information of an obstacle located in front and image information of the front; A step of setting a corresponding tracking mode based on detected obstacle information; A step of outputting walking guidance information to a pedestrian based on a set tracking mode Method for printing walking assistance guidance.

11. In Paragraph 10, The above tracking mode is Tracking off mode that disables the distance sensor and the vision sensor; A first tracking mode that activates the distance sensor; Activating the above vision sensor and the second tracking mode; and A third tracking mode that activates the distance sensor and vision sensor Method for printing walking assistance guidance.

12. In Paragraph 11, The above first tracking mode is Characterized by detecting obstacles within a guidance distance from the pedestrian based on the distance sensor information and determining the presence or absence of obstacles. Method for printing walking assistance guidance.

13. In Paragraph 12, The above second tracking mode is Characterized by determining the direction of the obstacle based on the vision sensor information. Method for printing walking assistance guidance.

14. In Paragraph 13, The above third tracking mode is Characterized by determining the level of risk to the pedestrian based on the location and degree of approach of the obstacle. Method for printing walking assistance guidance.

15. In Paragraph 10, The method further includes the step of controlling the vision sensor based on the distance sensor information. Method for printing walking assistance guidance.

16. In Paragraph 15, The step of controlling the vision sensor based on the distance sensor information above If the obstacle is detected at a distance greater than the guidance distance in the second tracking mode, the method further includes the step of disabling the vision sensor or controlling it to a low frame rate state. Method for printing walking assistance guidance.

17. In Paragraph 15, The step of controlling the vision sensor based on the distance sensor information above When the obstacle is detected at a distance closer than the guidance distance in the second tracking mode, the method further includes the step of controlling the vision sensor to a high frame rate. Method for printing walking assistance guidance.

18. In Paragraph 10, A step of analyzing the situation ahead based on the distance sensor information and the walking direction, walking speed, and the pedestrian's head direction; A step of determining the presence or absence of pedestrian risk factors based on the analyzed forward situation above; and If the above-mentioned walking risk factor exists, the step of activating the vision sensor is further included. Method for printing walking assistance guidance.

Images