Walking safety service application stored in terminal and terminal

Abstract

The present invention relates to a walking safety application stored in a terminal, the walking safety application including: a sensor information acquisition unit for acquiring sensor information from a plurality of sensors provided in the terminal; a user state determination unit for analyzing a state of a user gazing upon the terminal and a movement state of the terminal on the basis of the sensor information and setting the terminal state to any one of a normal state or a warning activation state; a collision risk determination unit for recognizing the shape of an obstacle in a walking direction based on the sensor information, determining whether the obstacle is a risk factor, and predicting a risk factor arrival time to generate risk alarm information; and a warning output unit for generating an alarm warning via a warning means provided in the terminal on the basis of the risk alarm information.

Description

Pedestrian safety service application and terminal stored on the terminal

[0001] The present invention relates to a pedestrian safety service application stored in a terminal, and more specifically, to a pedestrian safety service application that determines risk factors while walking and transmits an alarm to a pedestrian.

[0002] As mobile devices such as cell phones become more widespread, the use of mobile devices while walking is becoming more frequent.

[0003] In particular, accidents frequently occur while walking because users are not aware of their surroundings while watching videos, playing games, or browsing web pages on their devices.

[0004] In this regard, Korean Patent Registration No. 10-2232045 discloses a technology for preventing the risk of collision for pedestrians using a terminal by determining the direction in which an external object is located based on an audio signal and displaying the external object on the terminal.

[0005] The above patent discloses a technology for recognizing and judging external objects by sound, but there are many external objects in the external environment, a lot of noise unrelated to external objects, and the presence of external objects unrelated to noise, such as trees and pillars, so there are limitations as a technology for preventing collision risks with pedestrians.

[0006] The purpose of the present invention is to provide a pedestrian safety service application that can substantially prevent the risk of collision for pedestrians by using sensor information received from a plurality of sensors equipped in a user terminal.

[0007] A pedestrian safety application stored in a terminal according to an embodiment of the present invention includes: an information acquisition unit that acquires sensor information and driving information from a terminal; a user status determination unit that determines whether a user is using the terminal while walking based on the sensor information and the driving information and sets the terminal status to either a normal status or a warning activation status; a collision risk determination unit that, when the terminal status is a warning activation status, recognizes the shape of an obstacle in the walking direction based on the sensor information, determines whether the obstacle is a hazard, and generates hazard alarm information; and a warning output unit that generates an alarm warning through a warning means provided in the terminal based on the hazard alarm information.

[0008] In addition, in the embodiment, when the sensor information acquisition unit fails to acquire sensor information from the plurality of sensors, it generates sensor information reception error alarm information and provides it to the warning output unit.

[0009] In addition, in the embodiment, the user status determination unit includes a gaze status determination unit that determines the user's terminal gaze status based on a first image in the direction of the user's face and measures the duration of the terminal gaze status; a terminal movement status determination unit that determines the terminal's movement status based on acceleration sensor information and GPS sensor information of the terminal; and a terminal status setting unit that sets the terminal status to one of a normal status and a warning activation status based on the terminal gaze status, the gaze status duration, and the terminal movement status.

[0010] In addition, in the embodiment, the terminal state setting unit adjusts the duration judgment standard time, which is the judgment standard for the gaze state duration, based on the terminal movement speed derived from the acceleration sensor information and GPS sensor information of the terminal.

[0011] In addition, in the embodiment, the terminal status determination unit sets the terminal status based on the terminal gaze status, gaze status duration, and terminal movement status for each preset status inspection unit time.

[0012] In addition, in the embodiment, the user status determination unit further includes a first sensor information determination unit that generates first error alarm information when the user holds the front camera.

[0013] In addition, in the embodiment, the collision risk determination unit includes an obstacle determination unit that determines whether there is an obstacle and the lower boundary of the obstacle in the second image in the walking direction based on object recognition technology by deep learning; a risk factor determination unit that applies a risk area filter to the second image to determine whether the obstacle determined by the obstacle determination unit is a risk factor; and when the lower boundary of the obstacle reaches the risk reference line, which is the lower boundary of the risk area filter, risk factor alarm information is generated and provided to the warning output unit.

[0014] In addition, in the embodiment, the collision risk determination unit further includes a second sensor information determination unit that generates second error alarm information when the user's rear camera is caught.

[0015]

[0016] *In addition, in the embodiment, the warning output unit generates a warning alarm using a warning means provided in an electronic device connected to the terminal.

[0017] A terminal stored in a pedestrian safety application according to an embodiment of the present invention includes: an information acquisition unit that acquires sensor information and driving information from a terminal; a user status determination unit that determines whether a user is using the terminal while walking based on the sensor information and the driving information and sets the terminal status to either a normal status or a warning activation status; a collision risk determination unit that, when the terminal status is a warning activation status, recognizes the shape of an obstacle in the walking direction based on the sensor information, determines whether the obstacle is a hazard, and generates hazard alarm information; and a warning output unit that generates an alarm warning through a warning means provided in the terminal based on the hazard alarm information.

[0018] According to the present invention, a pedestrian safety service application stored in a terminal can prevent safety accidents of terminal users while walking by recognizing and warning of risk factors in advance even while the pedestrian is using the terminal.

[0019] FIG. 1 is a drawing for explaining a pedestrian safety application stored in a terminal according to an embodiment of the present invention.

[0020] Figure 2 is a drawing for explaining the user status judgment unit illustrated in Figure 1.

[0021] FIG. 3 is a drawing for explaining another type of user status judgment unit illustrated in FIG. 1.

[0022] Figure 4 is a drawing for explaining the collision risk judgment unit illustrated in Figure 1.

[0023] Figure 5 is an exemplary diagram for explaining the judgment method of the collision risk judgment unit illustrated in Figure 4.

[0024] Figure 6 is a drawing for explaining how the risk area filter setting unit adjusts the risk boundary line.

[0025] Figure 7 is a drawing for explaining how the risk area filter setting unit adjusts the risk baseline.

[0026] First, the terms used in this specification and claims are general terms selected in consideration of their functions in various embodiments of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, legal or technical interpretations, and the emergence of new technologies. Furthermore, some terms may have been arbitrarily selected by the applicant. These terms may be interpreted according to the meanings defined in this specification, and in the absence of a specific definition, they may be interpreted based on the overall content of this specification and common technical knowledge in the relevant technical field.

[0027] Additionally, the same reference numbers or symbols in each drawing attached to this specification represent parts or components that perform substantially the same functions. For convenience of explanation and understanding, the same reference numbers or symbols are used in different embodiments. In other words, even if components with the same reference numbers are all depicted in multiple drawings, the multiple drawings do not necessarily represent a single embodiment.

[0028] Additionally, terms including ordinal numbers, such as "first," "second," etc., may be used in this specification and claims to distinguish between components. These ordinal numbers are used to distinguish identical or similar components from each other, and the use of these ordinal numbers should not be interpreted in a limited manner. For example, components associated with these ordinals should not be restricted in their order of use or arrangement by their numbers. If necessary, each ordinal number may be used interchangeably.

[0029] In this specification, singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0030] In the embodiments of the present disclosure, terms such as "module," "unit," "part," etc. are terms used to refer to components that perform at least one function or operation, and such components may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of "modules," "units," "parts," etc. may be integrated into at least one module or chip and implemented as at least one processor, except in cases where each needs to be implemented as a separate, specific hardware.

[0031] Additionally, in the embodiments of the present disclosure, when a part is said to be connected to another part, this includes not only a direct connection but also an indirect connection through another medium. Furthermore, unless specifically stated otherwise, the statement that a part includes a certain component does not exclude other components, but rather implies that other components may be included.

[0032] In embodiments of the present disclosure, the meaning of “at least one of configuration 1, configuration 2 or configuration 3” may include all of the cases of “only the first configuration”, “only the second configuration”, “only the third configuration”, “both the first configuration and the second configuration”, “both the second configuration and the third configuration”, “both the first configuration and the third configuration”, and “both the first configuration, the second configuration and the third configuration”.

[0033] FIG. 1 is a drawing for explaining a pedestrian safety application stored in a terminal according to an embodiment of the present invention.

[0034] Referring to FIG. 1, the pedestrian safety application (100) includes an information acquisition unit (110), a user status judgment unit (120), a collision risk judgment unit (130), and a warning output unit (140).

[0035] The pedestrian safety application (100) performs a function of recognizing an obstacle in front and warning the user of the risk of collision with the obstacle in front when the user walks while looking at the terminal.

[0036] The pedestrian safety application (100) can be downloaded and installed on a terminal and run, or can be run in a built-in state on the terminal.

[0037] The information acquisition unit (110) acquires multiple sensor information and driving information from the terminal and the electronic device connected to the terminal and provides the information to the user status judgment unit (120).

[0038] The sensor information includes at least one of a user's face direction image provided from the terminal's front camera, a user's walking direction image provided from the terminal's rear camera, acceleration sensor information, gyroscope sensor information, and GPS sensor information.

[0039] The operating information includes the terminal's application operating information and display operating information.

[0040] In addition, the information acquisition unit (110) can use sensor information provided from sensors of electronic devices connected to the terminal in addition to sensors provided in the terminal.

[0041] The electronic device includes an electronic device that is connected to a terminal and can provide sensor information to the terminal or output a warning alarm based on alarm information provided by the terminal. For example, the electronic device may be at least one of Bluetooth earphones, smart glasses, a smartwatch, a portable camera, or a black box.

[0042] When the information acquisition unit (110) fails to acquire sensor information and driving information, it generates information reception error alarm information including the sensor information and driving information that was not acquired and provides it to the warning output unit (140).

[0043] Figure 2 is a drawing for explaining the user status judgment unit illustrated in Figure 1.

[0044] Referring to FIG. 2, the user status determination unit (120) includes a first information determination unit (121), a gaze status determination unit (122), a terminal movement status determination unit (123), and a terminal status determination unit (124).

[0045] The user status determination unit (120) determines whether the user is using the terminal while walking based on the sensor information and driving information acquired from the information acquisition unit (110) and sets the user status to either a normal state or a warning activation state.

[0046] If the first information judgment unit (121) cannot recognize the user's eyes based on the first image of the user's face direction provided from the front camera of the terminal, the first information judgment unit (121) generates first error alarm information and provides the first error alarm information to the warning output unit (140), causing the warning output unit (140) to output the first error alarm warning.

[0047] An example of a case where the user's eyes cannot be recognized based on the first image is when the user covers the front camera when holding the terminal, so that the user's eyes are not included in the image, and the user's status cannot be determined based on the front camera image.

[0048] The first information judgment unit (121) generates first error alarm information when the user's eyes cannot be identified from the front camera image.

[0049] The gaze state determination unit (122) generates the user's current gaze state information and gaze state duration information on the gaze state duration based on the detection of the user's gaze through analysis of the user's eye shape based on the terminal's operating information and the first image of the user's face direction.

[0050] Specifically, the gaze state determination unit (122) determines whether the terminal is currently running an application and the terminal's display is turned on based on operation information including the terminal's application operation information and display operation information.

[0051] If it is determined from the operating information that the terminal is currently running an application and that the terminal's display is turned on, the gaze state determination unit (122) recognizes the user's gaze with the first image and determines whether the user is gazing at the terminal, thereby generating terminal gaze state information, and measures the duration of the terminal's gaze state from the time the user started gazing at the terminal to the present, thereby generating gaze state duration.

[0052] Terminal gaze status information includes information on whether the user is currently 'gazing' or 'not' at the terminal, and gaze status maintenance information includes information on the duration of the user's terminal gaze.

[0053] Here, the determination of whether the user's gaze is recognized from the first image and whether the user is looking at the terminal can be made using any of the known deep learning-based CNN technology and other artificial intelligence-based image recognition technologies.

[0054] Meanwhile, if the terminal is not currently running an application or the terminal display is turned off, the gaze status determination unit (122) does not determine whether the user is gazing at the terminal.

[0055] The terminal movement status determination unit (123) determines the movement status including acceleration information of the terminal based on the acceleration sensor information and GPS sensor information of the terminal, and determines whether the user is currently walking, thereby generating terminal movement status information.

[0056] Specifically, the terminal movement status determination unit (123) determines whether the terminal's movement speed is within the movement speed setting range based on acceleration sensor information and GPS sensor information. Based on acceleration sensor information and GPS sensor information, it generates terminal movement status information indicating whether the user is in one of the following movement types: the first movement type of walking with the terminal, the second movement type of running, the third movement type of moving in a vehicle, and the fourth movement type of stationary.

[0057] The terminal status determination unit (124) sets the terminal status to one of the basic status and the warning activation status based on the user's terminal gaze status information, gaze status duration information, and terminal movement status information.

[0058] Terminal status changes can occur at preset status check intervals. For example, if the status check interval is set to 1 second, the terminal status can change every second.

[0059] Specifically, the terminal status setting unit (124) sets the terminal status to a warning activation state when the user is currently looking at the terminal from the terminal gaze status information, the gaze status duration is longer than a preset duration judgment standard time from the gaze status duration information, and the user is in the first movement form of walking with the terminal based on the terminal movement status information, and otherwise sets the terminal status to a normal state.

[0060] If the terminal status determination unit (124) determines the terminal status based solely on the terminal gaze state or solely on the gaze state duration, a warning about an obstacle, which is a risk factor, may be unnecessarily generated. Therefore, the terminal status determination unit (124) of the present invention determines the terminal status based on both the terminal gaze state and the gaze state duration.

[0061] Since displaying a warning to a user while the user can look at the terminal while also observing the surrounding environment only provides the user with unnecessary information, the present invention determines the warning activation status of the terminal by considering the duration of the terminal's gaze state in order to determine the concentration of the user's gaze state in order to solve the above problem.

[0062] In addition, the terminal status setting unit (124) can adjust the duration judgment standard time, which is the judgment standard for the gaze state duration, based on the terminal movement speed derived from the acceleration sensor information and GPS sensor information.

[0063] Since each user's walking speed varies, the faster the terminal's moving speed (as determined by acceleration and GPS sensor data), the shorter the duration determination time. The slower the terminal's moving speed, the longer the duration determination time. Accordingly, by considering the user's walking speed, unnecessary warnings and alarms can be minimized and the risk of collisions can be prevented in a customized manner.

[0064] FIG. 3 is a drawing for explaining another type of user status judgment unit illustrated in FIG. 1.

[0065] Referring to FIG. 3, the user status determination unit (120) includes a first sensor information determination unit (126) and a terminal status determination unit (127).

[0066] The user status judgment unit (120) determines whether the user is using the terminal while walking based on the terminal status judgment criteria and sets the user status to either a normal status or a warning activation status.

[0067] The user status determination unit illustrated in the above-described FIG. 2 determines the terminal status by determining the user's gaze state based on an image from a terminal held by the user or an electronic device connected to the terminal, but the user status determination unit of the present form illustrated in FIG. 3 determines the terminal status based on information other than the image.

[0068] The user status judgment unit (120) judges the usage status of the terminal based on application operation information, display operation information, and sensor information provided by the terminal.

[0069] Sensor information includes GPS sensor information, acceleration sensor information, and gyroscope sensor information.

[0070] When used for the first time or upon request by the user, the terminal status judgment criteria setting unit (121) sets the terminal status judgment criteria based on content testing.

[0071] When using a terminal, the user's walking pattern can be divided into a first walking pattern, where the user uses the terminal while observing their surroundings, and a second walking pattern, where the user focuses on the terminal and does not observe their surroundings. The user can switch between the first and second walking patterns at any time while using the terminal.

[0072] The movement speed of the terminal is slower and the shaking of the terminal is smaller than in the first walking form, where the user uses the terminal while looking around, and in the second walking form, where the user focuses on the terminal and does not look around.

[0073] Specifically, the terminal status judgment criterion setting unit (121) can set terminal status judgment criteria including the terminal's movement speed range, the terminal's tilt angle range, and the terminal's shake range based on the user's response to the test content provided through the user interface.

[0074] Specifically, the terminal status judgment criterion setting unit (121) provides a guide for setting the terminal status judgment criterion to the terminal through a user interface. Accordingly, when the terminal status judgment criterion setting unit (121) sets the terminal status judgment criterion, a user interface may be provided to the terminal to request the user to locate in a place without obstacles, and in response, a selection of either completed or incomplete may be requested through the user interface.

[0075] Afterwards, when the user's 'complete' response is received through the user interface, the user interface outputs the test content while displaying the start of the setup test on the terminal screen and receives a response to the test content from the user.

[0076] For example, the user interface could display the lyrics of the national anthem on the terminal during a setup test market, and ask the user to input how many parts of the national anthem lyrics there are incorrectly while walking, or ask the user to type the lyrics of the national anthem while walking.

[0077] The terminal status judgment criterion setting unit (121) calculates test output information for the terminal's average moving speed, maximum / minimum moving speed error, average tilt angle of the terminal, maximum / minimum tilt angle error, average shaking degree of the terminal, and maximum / minimum shaking error based on the terminal's GPS information, acceleration sensor information, and gyroscope sensor information during the test time until a response is input by the user after the start of the setting test.

[0078] The terminal status judgment criterion setting unit (121) creates or updates the terminal status judgment criterion when the user uses the terminal while walking, based on the test output information.

[0079] The terminal status judgment criterion setting unit (121) sets the terminal's moving speed range based on the terminal's average moving speed and maximum / minimum error, calculates the terminal's tilt angle range based on the terminal's average tilt angle and maximum / minimum error, and calculates the terminal's shake range based on the terminal's average shake information and maximum / minimum error, thereby setting the terminal status judgment criterion.

[0080] According to the terminal status judgment criteria above, situations where the user is not walking and concentrating on the terminal can be excluded, such as when the user is using the terminal while riding in a vehicle, when the user is walking while holding the terminal in his or her hand, or when the user is using the terminal while observing the surroundings. Here, the terminal tilt angle refers to the angle at which the user tilts the terminal while using it while walking.

[0081] The terminal status determination unit (127) sets the terminal status to either a normal status or a warning activation status based on terminal status determination criteria, driving information, and sensor information.

[0082] The operating information required for setting the terminal status includes the terminal's application operating information and display operating information, and the sensor information required for determining the terminal status includes GPS sensor information, acceleration sensor information, and gyroscope sensor information.

[0083] The terminal status determination unit (127) checks whether the application is currently running on the terminal and the display is turned on based on the application operation information and the display operation information.

[0084] First, if the application is not running or the display is turned off, the terminal status determination unit (127) determines the terminal status as normal.

[0085] Second, the terminal status determination unit (127) derives the moving speed of the terminal based on GPS information and acceleration sensor information, determines whether the moving speed is within the moving speed range, and determines the terminal status as normal if the moving speed is outside the moving speed range.

[0086] Third, the terminal state determination unit (127) derives the inclination of the terminal from the gyroscope sensor information, determines whether the inclination of the terminal is within the inclination angle range, and determines the terminal state as normal if the inclination of the terminal is outside the inclination angle range.

[0087] Fourth, the terminal status determination unit (127) calculates the acceleration of the terminal from the acceleration sensor information, determines whether the acceleration of the terminal is within the shaking range of the terminal, and determines the terminal status as normal if the acceleration of the terminal is outside the shaking range of the terminal.

[0088] If the terminal status determination unit (127) examines the first to fourth items and determines that any one of the terminal statuses is normal, the terminal status is determined to be normal.

[0089] If the terminal status determination unit (127) examines the first to fourth items and determines that the terminal status is not normal, the terminal status is set to warning activation status.

[0090] Fig. 4 is a drawing for explaining the collision risk judgment unit illustrated in Fig. 1, and Fig. 5 is an exemplary drawing for explaining the judgment method of the collision risk judgment unit illustrated in Fig. 4.

[0091] Referring to FIG. 4, the collision risk judgment unit (130) includes a second information judgment unit (131), an obstacle judgment unit (132), a risk factor judgment unit (133), a risk area filter setting unit (134), and a risk factor alarm information generation unit (134).

[0092] When the terminal status is in the warning activation state, the collision risk judgment unit (130) recognizes obstacles, judges risk factors among the obstacles, and generates risk factor alarm information.

[0093] If the second information judgment unit (131) cannot judge an object based on at least one front image among the second image in the walking direction, which is an image from the rear camera of the terminal, and the front image received from smart glasses connected to the terminal, it generates second error alarm information and provides the second error alarm information to the warning output unit (140), thereby causing the warning output unit (140) to output a second error alarm warning.

[0094] That is, if the rear camera of the terminal or the front camera of the smart glasses connected to the terminal is not working, or if the user has taken the rear camera of the terminal away, the obstacle cannot be recognized based on the second image, which is the rear camera image in the walking direction. Accordingly, the second sensor information status determination unit (131), if obstacle recognition is not possible from the second image, which is the rear camera image, generates second error alarm information and provides it to the warning output unit (140).

[0095] Referring to Fig. 5, the obstacle determination unit (132) analyzes the image information of the walking direction among the sensor information to detect an obstacle.

[0096] The obstacle determination unit (132) performs filtering on a specific area based on object recognition technology using deep learning, determines whether each filtered area is an object area, and determines whether it is an obstacle, and determines the boundary or lower boundary (L1) of the obstacle. Currently, many object recognition technologies using deep learning are known, and the obstacle determination unit (132) can borrow any of the currently known technologies.

[0097] In Fig. 5, the obstacle determination unit (132) can recognize object A (A) and object B (B).

[0098] The risk factor determination unit (133) is configured with two vertical risk boundary lines (L3) and a risk reference line (L2) on an image or photograph. By applying a risk area filter (F), an obstacle included in or crossing the risk area filter can be determined as a risk factor. Here, the solid line represents the boundary between the background wall and the floor.

[0099] In Fig. 5, object B is an obstacle that is included within or crosses the risk boundary line (L3) of two risk area filters (F), so object B (B) is judged as a risk factor, but object A (A) is an obstacle that is neither included within nor crosses the risk area filter (F), so object A (A) is not judged as a risk factor.

[0100] Figure 6 is a drawing for explaining how the risk area filter setting unit adjusts the risk boundary line, and Figure 7 is a drawing for explaining how the risk area filter setting unit adjusts the risk baseline.

[0101] Referring to FIGS. 6 and 7, the risk area filter setting unit (134) can receive an adjustment input for the risk filter area from the user and set the risk filter area.

[0102] In general, since users who use their terminals while walking often hold their terminals at a specific angle, the risk baseline (L2) and risk boundary line (L3) may be set differently for each user.

[0103] Referring to Fig. 6, a user can directly select a risk boundary line (L3) located on the terminal and change the position of the risk boundary line (L3) by dragging.

[0104] As shown in (A) of Fig. 6, when the user touches the danger boundary line (L3) with his or her finger to select it, and then drags the danger boundary line (L3) in a lateral direction, and places the user's finger at a desired location as shown in (B) of Fig. 6, the danger boundary line (L3) is changed to that location.

[0105] Referring to Figure 7, a user can directly select a risk reference line (L2) located on the terminal and change the position of the risk reference line (L2) by dragging.

[0106] As shown in (A) of Fig. 7, when the user touches the risk reference line (L2) with his / her finger and selects it, drags it up and down, and places his / her finger at the desired location as shown in (B) of Fig. 7, the risk reference line (L2) is changed.

[0107] As described above, the user can adjust the positions of the risk baseline (L2) and risk boundary line (L3) while looking at the device while walking, allowing the user to set a risk filter area optimized for their walking situation. This reduces the load on the device.

[0108] Unlike the above, the risk area filter setting unit (134) can also change the height of the risk reference line (L2) and the width of the risk boundary line (L3) in the risk area filter based on acceleration sensor information and gyroscope sensor information that change according to the user's terminal angle.

[0109] Referring again to FIG. 4, the risk factor alarm information generation unit (135) generates risk factor alarm information when the boundary or lower boundary (L1) of a risk factor obstacle reaches the risk reference line (L2) and provides it to the warning output unit (140). The warning alarm output from the warning output unit (140) can prevent the user from being struck by the risk factor.

[0110] The warning output unit (140) generates a warning alarm through a warning means provided in at least one of the terminal and the electronic device connected to the terminal based on at least one of sensor information reception error alarm information, first error alarm information, second error alarm information, and risk factor alarm information.

[0111] The warning means may include at least one means that can make the user aware of the warning alarm, in addition to displaying a warning message on the terminal screen or generating sound or vibration.

[0112] The warning output unit (140) can transmit a message that sensor information cannot be acquired from multiple sensors using a warning means based on sensor information reception error alarm information.

[0113] The warning output unit (140) can use a warning means based on the first error alarm information to transmit a message to the user that the user's eyes cannot be identified from the front camera image or that GPS information cannot be received.

[0114] The warning output unit (140) can transmit a message that an object cannot be determined based on the second image in the walking direction, which is an image from the rear camera of the terminal, using a warning means based on the second error alarm information.

[0115] The warning output unit (140) can transmit a message to prevent the user from being struck by a risk factor using a warning means based on the risk factor alarm information.

[0116] Meanwhile, the collision risk judgment unit (130) can be operated when the terminal status is in a warning activation state, but can also be operated regardless of the terminal status through user settings.

[0117] In the specific embodiments of the present invention described above, components included in the invention are expressed in the singular or plural form depending on the specific embodiment presented. However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present invention is not limited to singular or plural components. Even components expressed in the plural form may be composed of singular elements, or even components expressed in the singular form may be composed of plural elements.

[0118] While the detailed description of the present invention has described specific embodiments, it should be understood that various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the scope of the claims described below, but also by equivalents thereof.

Claims

1. In the pedestrian safety application stored on the terminal, An information acquisition unit that acquires sensor information and driving information from a terminal; A user status determination unit that determines whether a user is using a terminal while walking based on the above sensor information and driving information and sets the terminal status to either a normal status or a warning activation status; When the above terminal status is in a warning activation state, a collision risk judgment unit that recognizes the shape of an obstacle in the walking direction based on the sensor information, determines whether the obstacle is a hazard, and generates hazard alarm information; and A pedestrian safety application stored in a terminal, including a warning output unit that generates an alarm warning through a warning means provided in the terminal based on the above risk alarm information.

2. In paragraph 1, The above information acquisition department, A pedestrian safety application stored in a terminal, characterized in that when sensor information cannot be acquired from the terminal, information reception error alarm information is generated and provided to the warning output unit.

3. In paragraph 1, The above user status judgment unit is, A gaze state determination unit that determines the user's terminal gaze state based on a first image of the user's face direction and measures the duration of the gaze state; A terminal movement status determination unit that determines the movement status of the terminal based on the terminal's acceleration sensor information and GPS sensor information; and A pedestrian safety application stored in a terminal, characterized in that it includes a terminal status setting unit that sets the terminal status to one of a normal status and a warning activation status based on the terminal gaze status, the duration of the gaze status, and the terminal movement status.

4. In paragraph 3, The above terminal status setting section, The terminal state is set by comparing the duration of the above gaze state with the preset duration judgment standard, A pedestrian safety application stored in a terminal, characterized in that the above duration judgment criterion is adjusted based on the terminal movement speed derived from the terminal's acceleration sensor information and GPS sensor information.

5. In paragraph 3, A pedestrian safety application stored in a terminal, characterized in that the terminal status determination unit sets the terminal status based on the terminal gaze status, the duration of the gaze status, and the terminal movement status for each preset status inspection unit time.

6. In paragraph 3, The above user status judgment unit is, A pedestrian safety application stored in a terminal, characterized in that it further includes a first sensor information judgment unit that generates first error alarm information when the user has removed the front camera.

7. In paragraph 1, The above collision risk judgment unit is, An obstacle determination unit that determines whether there is an obstacle and the lower boundary of the obstacle in the second image in the walking direction based on object recognition technology using deep learning; A risk factor determination unit that applies a risk area filter to the second image and determines whether the obstacle determined by the obstacle determination unit is a risk factor; and A pedestrian safety application stored in a terminal, characterized in that it includes a risk factor alarm information generation unit that generates risk factor alarm information and provides it to the warning output unit when the lower boundary of the above obstacle reaches the risk reference line, which is the lower boundary of the risk area filter.

8. In paragraph 7, The above collision risk judgment unit is, A pedestrian safety application stored in a terminal, characterized in that it further includes a second information judgment unit that generates second error alarm information when the rear camera of the user is damaged.

9. In paragraph 1, The above warning output section, A pedestrian safety application stored in a terminal, characterized in that it generates a warning alarm using a warning means provided in an electronic device connected to the terminal.

10. In paragraph 1, The above warning output section, A pedestrian safety application stored in a terminal, characterized in that it generates a warning alarm using at least one warning means among displaying a warning message on a terminal screen, generating a sound, and generating a vibration.

11. In the terminal where the pedestrian safety application is stored, Information acquisition unit that acquires sensor information and driving information; A user status determination unit that determines whether a user is using a terminal while walking based on the above sensor information and driving information and sets the terminal status to either a normal status or a warning activation status; When the above terminal status is in a warning activation state, a collision risk judgment unit that recognizes the shape of an obstacle in the walking direction based on the sensor information, determines whether the obstacle is a hazard, and generates hazard alarm information; and A terminal storing a pedestrian safety application, including a warning output unit that generates an alarm warning through a warning means provided in the terminal based on the above risk alarm information.

Images