Electronic device and method for providing feedback for safe walking

The integration of electronic device sensors and vehicle ADAS systems provides real-time obstacle detection and alerts, addressing the challenge of user distraction and enhancing safety during walking near vehicles.

KR103043897B1Active Publication Date: 2026-11-06SAMSUNG ELECTRONICS CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
KR1020210003408
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-01-11
Publication Date
2026-11-06
Estimated Expiration
2041-01-11

AI Technical Summary

Technical Problem

Existing systems fail to effectively provide feedback to users on potential obstacles while walking near vehicles, especially when using electronic devices that distract the user from their surroundings, and do not integrate seamlessly with vehicle safety systems for enhanced collision risk assessment.

Method used

An electronic device equipped with sensors, cameras, and communication modules that detect obstacles, determine collision risks, and provide alerts through wearable devices or vehicles, integrating with vehicle ADAS systems for comprehensive safety support.

Benefits of technology

Enhances user safety by providing real-time obstacle detection and alerts, ensuring safe walking near vehicles by leveraging both electronic device and vehicle sensors and communication systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112021003353258-PAT00004_ABST
    Figure 112021003353258-PAT00004_ABST
Patent Text Reader

Abstract

Various embodiments of the present invention disclose a method and apparatus comprising a vision camera, a sensor module, a communication module, an output module, a memory, and a processor operatively connected to the vision camera, the sensor module, the communication module, the output module, or the memory, wherein the processor is connected to an external electronic device through the communication module, receives location information of the external electronic device from the external electronic device to determine an expected travel path, recognizes an obstacle on the expected travel path based on at least one of the location information of the external electronic device, an image acquired from the vision camera, a sensor value acquired from the sensor module, or information collected from another device existing on the road through the communication module, determines a collision risk between the external electronic device and the obstacle, and is configured to provide a risk alert through the output module. Various embodiments are possible.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] Various embodiments of the present invention disclose a method and apparatus for providing feedback to support safe walking for a user while moving in a vehicle. Background Technology

[0002] With the development of digital technology, various types of electronic devices such as mobile communication terminals, PDAs (personal digital assistants), electronic notebooks, smartphones, tablet PCs (personal computers), and wearable devices are widely used. For example, electronic devices can be connected to wireless earphones that can be worn on a user's ears to output the sound of music or videos through the wireless earphones. Alternatively, electronic devices can be connected to wearable display devices (e.g., AR glasses, smart glasses) to provide various content applying extended reality (XR) technologies, such as virtual reality (VR), augmented reality (AR), and / or mixed reality (MR), through the wearable display devices.

[0003] Meanwhile, vehicles are evolving not only to improve their performance as a means of transportation but also to enhance various functions related to safe driving. For example, the development of advanced driver assistance systems (ADAS) or autonomous vehicles is actively underway. For instance, vehicles can acquire various information from the outside through wireless communication modules, process the acquired information to support safe driving, and provide information to enable autonomous driving. The problem to be solved

[0004] In various embodiments, a method and apparatus may be disclosed for detecting surrounding obstacles through a sensor of an electronic device, connecting the electronic device and the vehicle using short-range communication when a user enters within a certain distance from the vehicle, identifying an expected movement path based on information obtained from the vehicle's ADAS, the electronic device's APAS (advanced pedestrian assistance systems), and the electronic device's sensor, or recognizing surrounding obstacles and providing a warning notification to the user through the electronic device or the vehicle based on a collision with the obstacle. means of solving the problem

[0005] An electronic device according to various embodiments of the present invention comprises a vision camera, a sensor module, a communication module, an output module, a memory, and a processor operatively connected to the vision camera, the sensor module, the communication module, the output module, or the memory. The processor may be configured to be connected to an external electronic device through the communication module, receive location information of the external electronic device from the external electronic device to determine an expected travel path, recognize an obstacle on the expected travel path based on at least one of the location information of the external electronic device, an image acquired from the camera, a sensor value acquired from the sensor module, or information collected from another device existing on the road through the communication module, determine a collision risk between the external electronic device and the obstacle, and provide a risk alert through the output module.

[0006] An electronic device according to various embodiments of the present invention comprises a sensor module, a vision camera, a communication module, a location recognition module, a memory, and a processor operatively connected to the sensor module, the vision camera, the communication module, the location recognition module, or the memory. The processor may be configured to acquire location information of the electronic device using the location recognition module, identify an expected travel path based on the location information of a vehicle stored in the memory and the acquired location information, recognize an obstacle on the expected travel path based on at least one of the location information of the electronic device, an image acquired from the vision camera, or a sensor value acquired from the sensor module, provide a danger alert based on the recognized obstacle, and, when the electronic device moves within a certain distance of the vehicle, connect with the vehicle and provide a danger alert through the vehicle.

[0007] A method of operation of an electronic device mounted on a vehicle according to various embodiments of the present invention may include: connecting to an external electronic device through a communication module of the electronic device; receiving location information of the external electronic device from the external electronic device and determining an expected travel path; recognizing an obstacle on the expected travel path based on at least one of the location information of the external electronic device, an image acquired from a vision camera of the electronic device, a sensor value acquired from a sensor module of the electronic device, or information collected from another device existing on the road through the communication module; and determining a collision risk between the external electronic device and the obstacle and providing a risk notification through an output module of the electronic device. Effects of the invention

[0008] According to various embodiments, a user can be protected from surrounding dangerous obstacles by utilizing various information obtained from electronic devices as well as various sensors mounted on the vehicle.

[0009] According to various embodiments, until the electronic device and the vehicle are connected, the electronic device directly recognizes surrounding dangerous obstacles, determines whether there is a collision with the surrounding dangerous obstacles, and issues a warning; and when the electronic device and the vehicle are connected, through the exchange of information between the electronic device and the vehicle, the electronic device or the vehicle can respectively warn of the risk of collision regarding dangerous obstacles around the user.

[0010] According to various embodiments, when a user is using an electronic device and is unable to look directly at their surroundings, the electronic device recognizes the situation, and if it is determined that the user may be exposed to the risk of an accident, sensors of surrounding devices are activated, and feedback can be provided to the user when the risk is detected.

[0011] According to various embodiments, when a user gets on or off a vehicle, an advanced pedestrian assistance system (APAS) is executed through an electronic device carried by the user, and an advanced driver assistance system (ADAS) of the vehicle is executed to detect risk factors from all directions with the user as the central point. Brief explanation of the drawing

[0012] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 2 is a diagram illustrating a network environment between an electronic device and a vehicle according to various embodiments. FIG. 3 is a configuration diagram illustrating the internal configuration of an electronic device according to various embodiments. FIG. 4 is a flowchart illustrating a method for providing user feedback for safe walking through information exchange between an electronic device and a vehicle according to various embodiments. FIG. 5 is a flowchart illustrating the operation method of an electronic device according to various embodiments. FIGS. 6a and 6b are drawings illustrating an example of providing user feedback through an electronic device or vehicle according to various embodiments. FIG. 7 is a flowchart illustrating a method of providing user feedback in a vehicle according to various embodiments. FIG. 8 is a diagram illustrating an example of providing user feedback through an electronic device or vehicle based on a change in the position of an electronic device according to various embodiments. FIG. 9 is a diagram illustrating an example of providing user feedback through a plurality of electronic devices connected to a vehicle according to various embodiments. FIG. 10 is a flowchart illustrating a method of providing user feedback through a vehicle and a plurality of electronic devices according to various embodiments. FIGS. 11a and FIGS. 11b are drawings illustrating an example of guiding collision risk through an electronic device or vehicle according to various embodiments. Specific details for implementing the invention

[0013] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments.

[0014] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0015] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0016] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0017] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0018] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0019] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0020] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0021] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0022] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0023] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0024] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0025] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0026] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0027] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0028] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0029] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0030] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0031] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0032] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0033] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0034] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0035] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0036] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0037] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0038] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0039] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0040] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0041] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0042] FIG. 2 is a diagram illustrating a network environment between an electronic device and a vehicle according to various embodiments.

[0043] Referring to FIG. 2, an electronic device (e.g., the electronic device (101) of FIG. 1) according to various embodiments may support safe walking of a user while the user moves in a direction of movement (210) by using at least one of a sensor module (e.g., the sensor module (176) of FIG. 1), a vision camera (e.g., the camera module (180) of FIG. 1), or a microphone (e.g., the input module (150) of FIG. 1) when the user exceeds a certain distance (230) from the vehicle (201). According to various embodiments, the electronic device (101) may include a smartphone, a tablet PC, and / or a laptop. The direction of movement (210) may mean the direction of movement toward the vehicle (201). The vehicle (201) may include a walking support device (or system) that supports safe walking of a user similar to the electronic device (101).

[0044] For example, the electronic device (101) can determine (or predict, calculate) an expected travel path to the vehicle (201) based on the location information of the vehicle (201) and the location information of the electronic device (101). The location information of the vehicle (201) may be stored in the memory of the electronic device (101) (e.g., the memory (130) of FIG. 1). For example, a user may turn off the engine of the vehicle (201) and move while carrying the electronic device (101). The electronic device (101) may store the location of the vehicle (201) at the time of disembarking from the vehicle (201) or at the time of boarding the vehicle (201). Additionally, the user's previous travel path while moving away from the vehicle (201) may be stored in the memory (130). The previous travel path may be calculated based on the location information of the electronic device (101) acquired in real time or periodically when a location-based service is activated (or turned on) on the electronic device (101). The electronic device (101) may transmit location information of the electronic device (101) to the vehicle (201) while moving away from the vehicle (201). The electronic device (101) may obtain the predicted travel path based on at least one of the location information of the vehicle (201), the previous travel path, or the location information of the electronic device (101) stored in the memory (130). The electronic device (101) may obtain (or receive) location information of the electronic device (101) in real time from a location information system (or satellite) through a location recognition module. The location recognition module may be a GPS (global positioning system) module or a GNSS module (e.g., the communication module (190) of FIG. 1), or it may be a module that recognizes location using surrounding base station signals, Wi-Fi, or beacon signals.

[0045] The electronic device (101) can recognize obstacles existing on the predicted path of movement and determine whether there is a collision with the obstacles. The obstacles may obstruct or endanger the safe walking of the user. For example, the obstacles may include utility poles, traffic lights, railroad tracks, signboards, other users, bicycles, motorcycles, automobiles, etc. The electronic device (101) can acquire the current location of the electronic device (101) in real time and acquire map information within a set distance (e.g., 1m, 3m, 5m) based on the current location. The map information may include high definition (HD) map information (or dynamic map). The HD map information refers to a 3D map with centimeter (cm) level precision and can be used for autonomous driving. The HD map information may include various information related to objects on the road (e.g., terrain, features) as 3D digital information (e.g., terrain information, spatial information, location information, 3D map information reflecting 3D spatial mapping information). Objects included in HD map information may refer to actual objects in the real world (e.g., terrain information, feature information, etc.).

[0046] The electronic device (101) can recognize obstacles (e.g., immovable obstacles (e.g., terrain, features)) based on HD map information and determine whether to collide with the obstacles. Alternatively, the electronic device (101) can recognize obstacles using at least one of a lidar, an ultrasonic sensor, an accelerometer, or a geomagnetic sensor. Lidar is an acronym for light detection and ranging or laser imaging, detection and ranging, and may refer to a sensor that emits a laser at a target (e.g., an object (e.g., terrain (river, mountain, sea), or feature (e.g., road, building, sign)) and detects distance, direction, speed, temperature, material distribution, and concentration characteristics by measuring the time and intensity taken for the emitted laser to return. Lidar may be included in an electronic device (101) or attached to the electronic device (101) as an accessory. An ultrasonic sensor may emit ultrasound (or an ultrasonic signal) and be used for distance measurement through the time it takes for the emitted ultrasound to be reflected back. An accelerometer may be a sensor that detects the speed of the electronic device (101). A geomagnetic sensor may be a sensor that detects the direction of the electronic device (101), e.g., yaw, roll, and pitch.

[0047] According to various embodiments, the electronic device (101) can recognize obstacles using a V2X (vehicle to everything) communication module (e.g., the communication module (190) of FIG. 1). V2X may be a communication technology that exchanges information with other vehicles and objects with infrastructure built on the road through a wired or wireless network. V2X may include all forms of communication technology applicable to vehicles on the road, such as vehicle-to-vehicle communication (V2V), vehicle-to-infra (V2I), vehicle-to-pedestrian (V2P), or vehicle-to-nomadic devices (V2N). The electronic device (101) can collect various information from IoT (internet of things) devices present on the road through the communication module (190).

[0048] The electronic device (101) can recognize obstacles using a time of flight (TOF) sensor (or camera), a vision camera, or a microphone. For example, the electronic device (101) can recognize obstacles located within a first distance (251) using a microphone, recognize obstacles located within a second distance (253) using a vision camera, recognize obstacles located within a third distance (255) using a short-range communication method, and recognize obstacles located within a fourth distance (257) using a LiDAR. The first distance (251) may be shorter than the second distance (253), the second distance (253) may be shorter than the third distance (255), and the third distance (255) may be shorter than the fourth distance (257). Such examples are provided only to aid in understanding the invention and do not limit the invention by description.

[0049] The electronic device (101) can recognize obstacles located nearby by analyzing sounds input from a microphone. The electronic device (101) can identify what the obstacle is by analyzing images acquired from a vision camera. The vision camera is a camera equipped with a vision sensor that can recognize or detect images acquired through the camera to identify target objects (e.g., obstacles). The electronic device (101) can calculate the distance to the obstacle or calculate the speed at which the obstacle approaches based on information acquired (or collected) from the sensor module (176), camera module (180), and communication module (190). The electronic device (101) can determine whether a collision will occur with the recognized obstacle based on the distance or speed to the obstacle.

[0050] According to various embodiments, the electronic device (101) may provide obstacle guidance information or obstacle collision risk when a collision with an obstacle is expected. The obstacle guidance information indicates what the obstacle is and may include, for example, at least one of text (e.g., railroad tracks, traffic lights, bicycles), images, or videos. The electronic device (101) may display the obstacle guidance information or obstacle collision risk on a display (e.g., the display module (160) of FIG. 1). Alternatively, the electronic device (101) may provide the obstacle guidance information or obstacle collision risk through vibration or sound. The electronic device (101) may determine whether a wearable device (e.g., earphones, AR glasses) is connected and provide the obstacle guidance information or obstacle collision risk through the wearable device.

[0051] The electronic device (101) may provide obstacle collision risk in different or distinct ways depending on the risk level (e.g., collision, danger, warning). For example, if the risk level is collision, the risk level may be high; if the risk level is danger, the risk level may be medium; and if the risk level is warning, the risk level may be low. The electronic device (101) may provide obstacle collision risk differently depending on the risk level. For example, if the risk level is high, the electronic device (101) may provide obstacle collision risk with a red color, strong vibration (or multiple times, quickly), and a loud sound (quickly). If the risk level is medium, the electronic device (101) may provide obstacle collision risk with an orange color, medium vibration (or multiple times, slowly), and a medium sound (slowly). If the risk level is low, the electronic device (101) may provide obstacle collision risk with a yellow color, low vibration, and a quiet sound. The above examples are for the purpose of facilitating understanding of the invention and are not intended to limit the invention for illustrative purposes only.

[0052] When the user is using the electronic device (101) (e.g., standing still or walking) or is unable to look directly at the surroundings, the electronic device (101) can recognize the user's dangerous situation. If the electronic device (101) determines that the user may be exposed to the risk of an accident, the sensors of the surrounding devices are activated, and when the risk is detected, the risk can be fed back to the user.

[0053] According to various embodiments, the electronic device (101) can determine the usage status of the electronic device (101) using a sensor module (176) or a camera module (180). The electronic device (101) can determine whether the user is looking at the electronic device (101) because it may be dangerous if the user walks while looking at the electronic device (101). The electronic device (101) can determine the usage status of the electronic device (101) based on sensor information obtained from a sensor module (176), such as an illuminance sensor, a proximity sensor, a motion sensor, an accelerometer, or a gyroscope, or an image obtained from a camera module (180). When the user walks while looking at the electronic device (101), the sensing values ​​(e.g., speed, rotation value) may continuously change. Alternatively, the front camera of the electronic device (101) may face the user, and the rear camera of the electronic device (101) may continuously change the scene during movement, so the acquired image may continuously change.

[0054] For example, the electronic device (101) can determine that the user's gaze direction is toward the electronic device (101) when the posture of the electronic device corresponds to a set posture (e.g., a posture that appears when a user uses the electronic device while walking), when a touch input is detected, or when a specific application is executed. When the user's gaze direction is toward the electronic device (101), the electronic device (101) can provide obstacle guidance information or an obstacle collision risk.

[0055] The user may continue moving in the direction where the vehicle (201) is located while carrying the electronic device (101). When the electronic device (101) continues moving in the direction of movement (210) and enters within a certain distance (230) where it can be connected (or paired) with the vehicle (201), it may be wirelessly connected to the vehicle (201). For example, the electronic device (101) may be connected to the vehicle (201) via short-range wireless communication such as RFID (Radio-Frequency Identification), Bluetooth, Low Power Bluetooth, Wi-Fi, Wi-Fi Direct, or UWB (Ultra Wide Band). The electronic device (101) may be connected to the vehicle (201) by transmitting subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196) of FIG. 1 to the vehicle (201). The electronic device (101) is connected to the vehicle (201) at least once and may include, for example, the electronic device of a user who has access to the vehicle (201) (e.g., the owner, family, or acquaintance of the vehicle (201). When wirelessly connected to the vehicle (201), the electronic device (101) can turn on the power of the vehicle (201), control the vehicle (201) to start, and transmit the predicted travel route.

[0056] When the vehicle (201) is connected to the electronic device (101), it can receive location information of the electronic device (101) from the electronic device (101) and determine an expected travel path. The expected travel path can be determined based on the location information of the vehicle (201) and the location information of the electronic device (101). Before the vehicle (201) is turned off, the vehicle (201) can acquire (or receive) the location information of the vehicle (201) in real time from a location information system (or satellite) through a location recognition module. The location recognition module may be a GPS or GNSS module (e.g., the communication module (360) of FIG. 3) or a module that recognizes location using surrounding base station signals, Wi-Fi, or beacon signals. The vehicle (201) may acquire and store the location information of the electronic device (101) while the electronic device (101) is moving away from the vehicle (201), and may use it when determining an expected travel path. Alternatively, the vehicle (201) may receive an expected travel path from the electronic device (101). The vehicle (201) may recognize obstacles on the expected travel path based on advanced driver assistance systems (ADAS). For example, the vehicle (201) may recognize surrounding obstacles based on at least one of a communication module, a vision camera, or a sensor module.

[0057] For example, a vehicle (201) can recognize obstacles located in front of and behind the vehicle (201) within a first distance (271) via an ultrasonic sensor. The vehicle (201) can recognize obstacles located within a second distance (273) via a camera (e.g., front camera, vision camera), lidar, laser, or radar. Alternatively, the vehicle (201) can recognize obstacles located within a third distance (275) by communicating with surrounding electronic devices, surrounding vehicles, and surrounding infrastructure via short-range communication such as V2X or UWB. The first distance (271) may be shorter than the second distance (273), and the second distance (273) may be shorter than the third distance (275). The above examples are provided only to aid in understanding the invention and are not intended to limit the invention by description.

[0058] According to various embodiments, the vehicle (201) may transmit the recognized obstacle information to the electronic device (101). The obstacle information may include location information of the obstacle or speed information of the obstacle. Alternatively, the vehicle (201) may calculate the distance between the electronic device (101) and the obstacle or the speed of the obstacle based on the current location of the electronic device (101) regarding the recognized obstacle. In this case, the obstacle information may include the distance between the electronic device (101) and the obstacle or the speed of the obstacle. The vehicle (201) may determine what the obstacle is from HD map information or an acquired image and transmit the obstacle information with the type of obstacle (e.g., person, animal, object) further included.

[0059] According to various embodiments, the vehicle (201) may receive location information (e.g., current location) of the electronic device (101) from the electronic device (101) in real time, periodically, or optionally. The vehicle (201) may receive obstacle information recognized by the electronic device (101) from the electronic device (101). The obstacle information may receive at least one of the obstacle location information, obstacle speed information, obstacle type information, or distance information between the obstacle and the electronic device (101). The vehicle (201) may determine whether to collide with an obstacle based on at least one of the obstacle information recognized by the vehicle (201), the obstacle information received from the electronic device (101), or the location information of the electronic device (101).

[0060] When a collision with an obstacle is anticipated for a user, the vehicle (201) may provide an obstacle collision risk through an output module. For example, the output module may include the vehicle's (201) lights or horn. The vehicle (201) may provide an obstacle collision risk by turning on the lights or sounding the horn. For example, the vehicle (201) may provide different directions of lights based on the location of the obstacle or the location information of the electronic device (101). The vehicle (201) may turn on the left light when the obstacle is located to the left of the vehicle (201), and turn on the right light when the obstacle is located to the right of the vehicle (201). Alternatively, the vehicle (201) may turn on the left light when the electronic device (101) is located to the left of the vehicle (201), and turn on the right light when the electronic device (101) is located to the right of the vehicle (201). The vehicle (201) can provide a different direction of light and sound a horn based on the location of an obstacle or the location information of the electronic device (101).

[0061] According to various embodiments, the vehicle (201) may provide different directions of light, intensity, or horn volumes depending on the risk level. For example, if the risk level is collision, the vehicle (201) may determine the risk to be high and output a light of strong intensity or a horn of loud sound. If the risk level is high, the vehicle (201) may output different directions of light. If the risk level is danger, the vehicle (201) may determine the risk to be medium and output a light of medium intensity or a horn of medium sound. If the risk level is medium, the vehicle (201) may output a light in one direction. If the risk level is warning, the vehicle (201) may determine the risk to be low and output a light of weak intensity or a horn of quiet sound. Alternatively, if the lights of the vehicle (201) can output different colors, the vehicle (201) may output different colors of light depending on the risk level. The above examples are intended only to aid in understanding the invention and are not intended to limit the invention for the sake of explanation.

[0062] FIG. 3 is a configuration diagram illustrating the internal configuration of an electronic device according to various embodiments.

[0063] Referring to FIG. 3, a vehicle according to various embodiments (e.g., vehicle (201) of FIG. 2) may include (or be equipped with) a walking assistance device (or system) (301) (or electronic device (101) of FIG. 1). The walking assistance device (301) may include a vision camera (310), a sensor module (320), a processor (330), a memory (340), an output module (350), and a communication module (360). The walking assistance device (301) may be similar to the electronic device (101) of FIG. 1 and may further include other components not shown in FIG. 3 (e.g., a display module (160)).

[0064] A vision camera (310) or a sensor module (320) may be placed inside or outside the vehicle (201). The vision camera (310) may be installed on the front, rear, or side of the vehicle (201) to photograph the exterior of the vehicle (201) within a set distance (e.g., 10 m) depending on the field of view or resolution. The vision camera (310) may be a camera including a vision sensor. The sensor module (320) may be the same or similar as the sensor module (176) of FIG. 1 and may include at least one of an ultrasonic sensor, lidar, radar, laser, accelerometer, or geomagnetic sensor. Lidar, radar, or laser may refer to a sensor that emits a laser (or radio wave) at a target (e.g., an object (e.g., terrain (river, mountain, sea), or feature (e.g., road, building, sign)) and detects distance, direction, speed, temperature, material distribution, and concentration characteristics by measuring the time and intensity taken for the emitted laser (or radio wave) to return. An acceleration sensor may be a sensor that detects the speed of the vehicle (201). A geomagnetic sensor may be a sensor that detects the direction (or steering angle) of the vehicle (201), e.g., yaw, roll, pitch.

[0065] The output module (350) may include a light or a horn of the vehicle (201). The output module (350) may control the light or a horn of the vehicle (201) according to the control of the processor (330). The light of the vehicle (201) may include a first light positioned on the right side of the vehicle (201) or a second light positioned on the left side of the vehicle (201).

[0066] The communication module (360) can obtain (or receive) location information or map information of the walking support device (301) from an external server (e.g., a location providing system, a navigation system). The map information may be HD map information. The communication module (360) can communicate with at least one of an electronic device (e.g., the electronic device (101) of FIG. 1), a surrounding electronic device, a surrounding vehicle, or surrounding infrastructure through short-range communication such as V2X or UWB. The electronic device (101) is connected to the vehicle (201) at least once and may include, for example, an electronic device of a user who has access to the vehicle (201) (e.g., the owner, family, or acquaintance of the vehicle (201)). The communication module (360) can receive the current location or expected travel path of the electronic device (101) from the electronic device (101). The communication module (360) can collect information from at least one of the surrounding electronic device, a surrounding vehicle, or surrounding infrastructure. The communication module (360) can transmit at least one of the location information of the walking support device (301), map information, the current location of the electronic device (101), the expected movement path, or collected information to the processor (330).

[0067] The processor (330) can perform the same or similar role as the processor (120) of FIG. 1. For example, the processor (330) can acquire an image captured by the vision camera (310) or a sensor value measured by the sensor module (320). The processor (330) can analyze the image to recognize what an obstacle (or target object) is. For example, the processor (330) can recognize whether the obstacle is a person, an animal, or an object. The processor (330) can use the sensor value to acquire location information of the vehicle (201) (e.g., the current location of the vehicle (201), speed information of the vehicle (201), steering angle information of the vehicle (201), or distance information between the vehicle (201) and the obstacle. The processor (330) can acquire at least one of location information, map information, the current location of the electronic device (101), the expected travel path, or collected information from the communication module (360). The processor (120) can store the acquired information in memory (340).

[0068] The processor (330) can analyze acquired information to recognize obstacles and determine whether there is a collision with the obstacles. The processor (330) can determine whether there is a collision with the obstacles based on the location information of the recognized obstacles and the electronic device (101). If a collision between the user and the obstacle is expected, the processor (330) can provide an obstacle collision risk through the output module (350). For example, the processor (330) can provide an obstacle collision risk by turning on the lights of the vehicle (201) or sounding the horn. The processor (330) can provide different directions, intensities, or horn sizes depending on the risk level. For example, the processor (330) can drive either a first light positioned on the right side of the vehicle or a second light positioned on the left side of the vehicle, or drive the first light and the second light differently, or drive the first light, the second light, and the horn. The pattern driving the first light and the second light may mean a pattern of flashing light.

[0069] For example, if the risk level is collision, the processor (330) determines the risk to be high and may output a light of high intensity or a loud horn. If the risk level is high, the processor (330) may output different directions of light or flashing patterns. If the risk level is danger, the processor (330) determines the risk to be medium and may output a light of medium intensity or a horn of medium sound. If the risk level is medium, the processor (330) may output the light in one direction. If the risk level is warning, the processor (330) determines the risk to be low and may output a light of weak intensity or a horn of low sound. Or, if the light of the vehicle (201) can output a color, the processor (330) may output different colors of light depending on the risk level.

[0070] An electronic device according to various embodiments of the present invention (e.g., walking assistance device (301) of FIG. 3) comprises a vision camera (e.g., vision camera (310) of FIG. 3), a sensor module (e.g., sensor module (320) of FIG. 3), a communication module (e.g., communication module (360) of FIG. 3), an output module (e.g., output module (350) of FIG. 3), a memory (e.g., memory (340) of FIG. 3), and a processor (e.g., processor (330) of FIG. 3) operatively connected to the vision camera, the sensor module, the communication module, the output module, or the memory, wherein the processor is connected to an external electronic device through the communication module, receives location information of the external electronic device from the external electronic device to determine an expected movement path, recognizes obstacles on the expected movement path based on at least one of the location information of the external electronic device, an image acquired from the camera, a sensor value acquired from the sensor module, or information collected from other devices existing on the road through the communication module, and determines the risk of collision between the external electronic device and the obstacles. It can be configured to provide a risk alert through the above output module based on a judgment.

[0071] The above electronic device may be mounted on a vehicle (e.g., vehicle (201) of FIG. 2) and configured to turn on the power of the vehicle or turn on the ignition of the vehicle according to the control of the external electronic device.

[0072] The output module includes a horn, a first light positioned on the right side of the vehicle, or a second light positioned on the left side of the vehicle, and the processor may be configured to drive the first light or the second light differently or output a horn through the horn based on at least one of the location information of the external electronic device or the location of the obstacle.

[0073] The processor may be configured to provide different directions of the vehicle's lights, light intensity, or horn volume output through the horn depending on the risk level of the risk notification.

[0074] The processor recognizes an obstacle located within a first distance from the electronic device through an ultrasonic sensor, recognizes an obstacle located within a second distance from the electronic device through a vision camera or lidar, recognizes an obstacle located within a third distance from the electronic device through V2X (vehicle to everything) or UWB (ultra wide band) communication, and the first distance may be set to be shorter than the second distance and the second distance may be set to be shorter than the third distance.

[0075] The processor may be configured to calculate the distance between the electronic device and the recognized obstacle or the speed of the obstacle based on the current location of the external electronic device, determine whether a collision with the obstacle occurs based on at least one of the distance between the electronic device and the obstacle or the speed of the obstacle, and provide a risk alert based on the result of the determination.

[0076] The processor may be configured to transmit obstacle information including the recognized obstacle to the external electronic device, receive obstacle information recognized by the external electronic device from the external electronic device, and determine the risk of collision between the external electronic device and the obstacle based on the recognized obstacle and the received obstacle information.

[0077] The processor may be configured to acquire HD (high definition) map information from a server and to determine the risk of collision between the external electronic device and the obstacle based further on the acquired map information.

[0078] An electronic device according to various embodiments of the present invention (e.g., the electronic device (101) of FIG. 1) comprises a sensor module (e.g., the sensor module (176) of FIG. 1), a vision camera (e.g., the camera module (180) of FIG. 1), a communication module (e.g., the communication module (190) of FIG. 1), a position recognition module, a memory (e.g., the memory (130) of FIG. 1), and a processor (e.g., the processor (120) of FIG. 1) operatively connected to the sensor module, the vision camera, the communication module, the position recognition module, or the memory, wherein the processor acquires position information of the electronic device using the position recognition module, identifies an expected movement path based on the position information of a vehicle stored in the memory and the acquired position information, recognizes an obstacle on the expected movement path based on at least one of the position information of the electronic device, an image acquired from the vision camera, or a sensor value acquired from the sensor module, provides a danger alert based on the recognized obstacle, and when the electronic device moves within a certain distance of the vehicle, the vehicle and It can be connected and configured to provide risk alerts through the vehicle.

[0079] The memory further stores a previous travel path obtained based on the position change of the electronic device while moving away from the vehicle, and the processor may be configured to determine the expected travel path based further on the previous travel path.

[0080] The processor may be configured to provide the danger alert through at least one of a display module, a haptic module, or a speaker when not connected to the vehicle, and to provide the danger alert through the electronic device or the vehicle when connected to the vehicle.

[0081] The processor may recognize an obstacle located within a first distance from the electronic device using a microphone of the electronic device, recognize an obstacle located within a second distance from the electronic device using a vision camera, recognize an obstacle located within a third distance from the electronic device using V2X (vehicle to everything) or UWB (ultra wide band) communication, recognize an obstacle located within a fourth distance from the electronic device using lidar, and the first distance may be set to be shorter than the second distance, the second distance may be shorter than the third distance, and the third distance may be set to be shorter than the fourth distance.

[0082] The processor may be configured to receive obstacle information from the vehicle when connected to the vehicle, determine whether there is a collision between the electronic device and the obstacle based on the recognized obstacle or the received obstacle information, and provide the risk alert based on the result of the determination.

[0083] The above processor may be configured to acquire HD (high definition) map information from a server and to determine whether there is a collision between the electronic device and an obstacle based further on the acquired map information.

[0084] The processor may be configured to track the direction of the user's gaze, and when the direction of the user's gaze is toward the electronic device, provide the danger alert through at least one of the display module, the haptic module, or the speaker, and when the direction of the user's gaze is not toward the electronic device, provide the danger alert through the vehicle.

[0085] The processor may be configured to determine that the user's gaze direction is toward the electronic device when the posture of the electronic device corresponds to a set posture, a touch input is detected, or a specific application is executed.

[0086] The processor may be configured to determine whether it is connected to a wearable device, and if it is connected to the wearable device, to provide a risk alert through the wearable device.

[0087] The above processor may be configured to provide different risk alerts based on the risk level or the type of obstacle.

[0088] FIG. 4 is a flowchart (400) illustrating a method of providing user feedback for safe walking through information exchange between an electronic device and a vehicle according to various embodiments.

[0089] Referring to FIG. 4, in operation 401, an electronic device according to various embodiments (e.g., the electronic device (101) of FIG. 1) can determine the user's expected movement path. The expected movement path may refer to the path taken by a user carrying the electronic device (101) while moving toward a vehicle (e.g., the vehicle (201) of FIG. 2) or while moving away from (or away from) the vehicle (201). The electronic device (101) may calculate the expected movement path based on the location information of the vehicle (201) stored in memory (e.g., the memory (130) of FIG. 1) and the current location of the electronic device (101). Alternatively, the electronic device (101) may calculate the expected movement path by further considering a previous movement path. The previous movement path may be calculated based on the change in the position of the electronic device (101) over time while moving away from the vehicle (201) after the electronic device (101) has stored the location information of the vehicle (201).

[0090] In operation 403, the electronic device (101) may recognize obstacles on the predicted path of movement. The obstacles may hinder or endanger the safe walking of the user. For example, the obstacles may refer to utility poles, traffic lights, railroad tracks, billboards, other users, bicycles, motorcycles, cars, etc. The electronic device (101) may acquire the current location of the electronic device (101) in real time and acquire map information within a set distance (e.g., 1m, 3m, 5m) based on the current location. The map information may include HD map information. The electronic device (101) may acquire sensor values ​​(or sensor data) from at least one of a LiDAR, an accelerometer, or a geomagnetic sensor. The electronic device (101) may acquire images from a TOF sensor (or camera) or a vision camera (e.g., the camera module (180) of FIG. 1) or acquire sound from a microphone (e.g., the input module (150) of FIG. 1). The electronic device (101) can recognize obstacles based on at least one of the sensor value, the image, the sound, or the map information. According to various embodiments, the electronic device (101) can recognize obstacles using short-range communication such as V2X or UWB. The electronic device (101) can collect various information from IoT devices present on the road through a communication module (190).

[0091] In operation 405, the electronic device (101) can determine whether it is located in close proximity to the vehicle (201). If it is located in close proximity to the vehicle (201) (e.g., within a certain distance (e.g., within a certain distance (230) of FIG. 2), the electronic device (101) can wirelessly connect with the vehicle (201). If the electronic device (101) is not located in close proximity to the vehicle (201), it may not be able to connect with the vehicle (201). If the electronic device (101) is located in close proximity to the vehicle (201), it can perform operation 407, and if it is not located in close proximity to the vehicle (201), it can perform operation 415.

[0092] In operation 407, when located in close proximity to the vehicle (201), the electronic device (101) can be connected to the vehicle (201). For example, the electronic device (101) can be connected to the vehicle (201) via short-range wireless communication such as Bluetooth, low-power Bluetooth, Wi-Fi, Wi-Fi Direct, or UWB. The electronic device (101) may include the electronic device of a user who has access to the vehicle (201) at least once, for example, the owner, family, or acquaintance of the vehicle (201). The electronic device (101) can be connected to the vehicle (201) by transmitting subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196) of FIG. 1 to the vehicle (201). When wirelessly connected to the vehicle (201), the electronic device (101) can control the power of the vehicle (201) or the vehicle (201) to be turned on.

[0093] In operation 409, the electronic device (101) can transmit the current location of the electronic device (101) to the vehicle (201). The electronic device (101) can transmit the current location of the electronic device (101) in real time, periodically, or optionally. Alternatively, the electronic device (101) can transmit the expected travel path along with the current location of the electronic device (101) to the vehicle (201). The electronic device (101) can obtain the current location (or location information) of the electronic device (101) through a location recognition module. The location recognition module may be a GPS module or a GNSS module (e.g., the communication module (190) of FIG. 1), or a module that recognizes location using surrounding base station signals, Wi-Fi, or beacon signals.

[0094] In operation 410, the vehicle (201) may determine an expected movement path based on the current location of the electronic device (101) and the current location of the vehicle (201). The vehicle (201) may include the walking assistance device (301) of FIG. 3. The vehicle (201) may determine the expected movement path directly or receive an expected movement path from the electronic device (101). If the vehicle (201) receives an expected movement path from the electronic device (101), operation 410 may be omitted. This is merely an implementation issue and the invention is not limited by the description.

[0095] In operation 411, the vehicle (201) can detect (or recognize) surrounding obstacles based on ADAS. The vehicle (201) can recognize surrounding obstacles based on at least one of the current location of the electronic device (101), the expected travel path, sensor values ​​(e.g., obtained from the sensor module (320)), images (e.g., obtained from the sensor module (320)), or collected information (e.g., obtained from the communication module (360) of FIG. 3). For example, the vehicle (201) can recognize obstacles located in front of and behind the vehicle (201) within a first distance (e.g., the first distance (271) of FIG. 2) through an ultrasonic sensor (e.g., the sensor module (320) of FIG. 3). The vehicle (201) can recognize obstacles located within a second distance (e.g., the second distance (273) in Fig. 2) through a front camera, a vision camera (e.g., the vision camera (310) in Fig. 3), a lidar, a laser, or a radar. The vehicle (201) can recognize obstacles located within a third distance (e.g., the third distance (275) in Fig. 2) by communicating with surrounding electronic devices, surrounding vehicles, and surrounding infrastructure through short-range communication such as V2X or UWB (e.g., the communication module (360) in Fig. 3).

[0096] In operation 413, the vehicle (201) may transmit the recognized obstacle information to the electronic device (101). The obstacle information may include location information of the obstacle or speed information of the obstacle. Alternatively, the vehicle (201) may calculate the distance between the electronic device (101) and the obstacle or the speed of the obstacle based on the current location of the electronic device (101) regarding the recognized obstacle. In this case, the obstacle information may include the distance between the electronic device (101) and the obstacle or the speed of the obstacle. The vehicle (201) may determine what the obstacle is from HD map information or acquired images and transmit the obstacle information with the type of obstacle (e.g., person, animal, object) further included. Alternatively, depending on the implementation, the vehicle (201) may not transmit the recognized obstacle information to the electronic device (101). If the vehicle (201) does not transmit the recognized obstacle information to the electronic device (101), operation 413 may be omitted. This is merely an implementation issue and the invention is not limited by the description.

[0097] In operation 415, the electronic device (101) can determine whether there is a collision with an obstacle. If it is not located in close proximity to the vehicle (201), the electronic device (101) can determine whether there is a collision with an obstacle without being connected to the vehicle (201). The electronic device (101) can determine whether there is a collision with an obstacle by calculating the distance to the obstacle or by calculating the speed at which the obstacle approaches before being connected to the vehicle (201). When the electronic device (101) is connected to the vehicle (201), it can determine whether there is a collision with an obstacle by further considering the obstacle information received from the vehicle (201). The electronic device (101) can calculate the distance between the electronic device (101) and the obstacle recognized by the electronic device (101), or the obstacle recognized by the vehicle (201), or calculate the speed at which the obstacle approaches. The electronic device (101) can determine whether there is a collision with the recognized obstacle based on the distance or speed to the obstacle.

[0098] In operation 417, the electronic device (101) may notify of a risk based on a collision with the obstacle. For example, the obstacle guidance information indicates what the obstacle is and may include at least one of, for example, text (e.g., railroad tracks, traffic lights, bicycles), images, or videos. The electronic device (101) may display the obstacle guidance information or the obstacle collision risk on a display (e.g., the display module (160) of FIG. 1). Alternatively, the electronic device (101) may provide the obstacle guidance information or the obstacle collision risk through vibration or sound. The electronic device (101) may provide vibration regarding the obstacle guidance information or the obstacle collision risk by controlling the haptic module (179) of FIG. 1, or output sound regarding the obstacle guidance information or the obstacle collision risk (e.g., warning sound, voice) through the sound output module (155). The electronic device (101) may determine whether a wearable device (e.g., earphones, AR glasses) is connected and provide the obstacle guidance information or the obstacle collision risk through the wearable device.

[0099] According to various embodiments, the electronic device (101) can determine the usage status of the electronic device (101) using a sensor module (176) or a camera module (180). The electronic device (101) can determine whether the user is looking at the electronic device (101) because it may be dangerous if the user walks while looking at the electronic device (101). The electronic device (101) can determine that the user's gaze direction is toward the electronic device (101) if the posture of the electronic device (101) corresponds to a set posture (e.g., a posture that appears when the user uses the electronic device while walking), if a touch input is detected, or if a specific application is executed. If the user's gaze direction is toward the electronic device (101), the electronic device (101) can provide obstacle guidance information or an obstacle collision risk.

[0100] In operation 416, the vehicle (201) can determine whether to collide with an obstacle. The vehicle (201) can determine whether to collide with an obstacle based on the location information of the recognized obstacle and the electronic device (101). The vehicle (201) can receive obstacle information recognized by the electronic device (101) or receive location information of the electronic device (101) from the electronic device (101). The received obstacle information may receive at least one of the location information of the obstacle, the speed information of the obstacle, the type information of the obstacle, or the distance information between the obstacle and the electronic device (101). The vehicle (201) can determine whether to collide with an obstacle based on at least one of the obstacle information recognized by the vehicle (201), the obstacle information received from the electronic device (101), or the location information of the electronic device (101).

[0101] In operation 418, the vehicle (201) may provide a warning based on a collision with the obstacle. If the vehicle (201) anticipates a collision with the obstacle, it may provide an obstacle collision risk through an output module. For example, the output module may include the vehicle's (201) lights or horn. The vehicle (201) may provide an obstacle collision risk by turning on the lights or sounding the horn. The vehicle (201) may provide different directions of the lights, intensity, or horn volumes depending on the risk level.

[0102] According to various embodiments, when the electronic device (101) and the vehicle (201) are not connected, only operations 415 and 417 are performed, and when the electronic device (101) and the vehicle (201) are connected, operations 415 and 417 may be performed simultaneously, sequentially, or only in one of the entities with operations 416 and 418. For example, when the electronic device (101) and the vehicle (201) are connected and the user's gaze direction is toward the electronic device (101), operations 415 and 417 are performed, and when the user's gaze direction is not toward the electronic device (101), operations 416 and 418 may be performed. Alternatively, regardless of the user's gaze direction, operations 415 and 417 may be performed simultaneously or sequentially with operations 416 and 418. This is merely an implementation issue and the invention is not limited by the description.

[0103] FIG. 5 is a flowchart (500) illustrating the operation method of an electronic device according to various embodiments.

[0104] Referring to FIG. 5, in operation 501, a processor (e.g., processor (120) of FIG. 1) of an electronic device (e.g., electronic device (101) of FIG. 1) according to various embodiments may obtain an expected travel path. The expected travel path may refer to a path taken while a user carrying the electronic device (101) moves toward a vehicle (e.g., vehicle (201) of FIG. 2) or moves away from the vehicle (201). The user may turn off the engine of the vehicle (201) and move while carrying the electronic device (101). The processor (120) may store location information of the vehicle (201) in a memory (e.g., memory (130) of FIG. 1) at the time of disembarking from the vehicle (201) or at the time of boarding the vehicle (201). The processor (120) may calculate the expected travel path based on the location information of the vehicle (201) stored in the memory (130) and the current location of the electronic device (101). The processor (120) can obtain the current location of the electronic device (101) through a location recognition module. The location recognition module may be a GPS module or a GNSS module (e.g., the communication module (190) of FIG. 1), or a module that recognizes location using surrounding base station signals, Wi-Fi, or beacon signals. Additionally, the processor (120) can calculate a previous path based on the change in the location of the electronic device (101) over time while moving away from the vehicle (201) after storing the location information of the vehicle (201). The processor (120) can store the previous path in memory (130) along with the location information of the vehicle (201). The processor (120) can calculate the predicted path by further considering the previous path.

[0105] In operation 503, the processor (120) may recognize obstacles on the predicted path of movement. The obstacles may hinder or endanger the safe walking of the user. For example, the obstacles may refer to utility poles, traffic lights, railroad tracks, signboards, other users, bicycles, motorcycles, cars, etc. The processor (120) may acquire the current location of the electronic device (101) in real time and obtain map information within a set distance (e.g., 1m, 3m, 5m) based on the current location from a map information providing server (e.g., server (108) of FIG. 1). The map information may include HD map information. The HD map information refers to a 3D stereoscopic map with centimeter (cm) level precision and can be used for autonomous driving. Objects included in the HD map information may refer to actual objects in the real world (e.g., terrain information, feature information, etc.). The processor (120) may recognize obstacles (e.g., immovable obstacles (e.g., terrain, feature)) based on the HD map information. The processor (120) can recognize obstacles based on sensor values ​​obtained through a sensor module (e.g., the sensor module (176) of FIG. 1). The sensor module (176) may include at least one of a lidar, an accelerometer, or a geomagnetic sensor. The lidar may be included in the electronic device (101) or attached to the electronic device (101) in the form of an accessory.

[0106] The processor (120) can recognize obstacles using V2X or UWB communication. V2X may be a communication technology that exchanges information with other vehicles and objects with infrastructure built on the road through a wired or wireless network. Alternatively, the processor (120) may recognize obstacles using a TOF sensor (or camera), a vision camera, or a microphone. For example, the processor (120) may recognize obstacles located nearby by analyzing sound input from a microphone. The processor (120) may be able to identify what the obstacle is by analyzing an image acquired from a vision camera. The processor (120) may transmit sound or images to an AI (artificial intelligence) or an external server (e.g., the server (108) in FIG. 1) for sound analysis or image analysis, and obtain the analyzed results. The processor (120) may calculate the distance to the obstacle or calculate the speed at which the obstacle is approaching based on at least one of map information, sensor values, images, or sounds.

[0107] In operation 505, the processor (120) can determine whether it is connected to the vehicle (201). If the processor (120) is located in close proximity to the vehicle (201) (e.g., within a certain distance (e.g., within a certain distance (230) of FIG. 2), it can be wirelessly connected to the vehicle (201). The processor (120) can be connected to the vehicle (201) by transmitting subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196) of FIG. 1 to the vehicle (201). The connection with the vehicle (201) can be performed before or after operation 501 or operation 503. If the processor (120) is not connected to the vehicle (201), it can perform operation 507, and if it is connected to the vehicle (201), it can perform operation 511.

[0108] In operation 507, when not connected to the vehicle (201), the processor (120) can determine whether there is a collision with an obstacle. For example, the processor (120) can calculate the speed of the user using location information received from a location provider server and sensor values ​​obtained from a sensor module (176). Since the user is carrying an electronic device (101), the speed of the user can be interpreted as the speed of the electronic device (101). The processor (120) can determine whether there is a collision with an obstacle based on at least one of the distance between the recognized obstacle and the electronic device (101), the speed of the electronic device (101), or the current location of the electronic device (101).

[0109] In operation 509, the processor (120) may provide a danger alert based on the result of the above determination. If the vehicle (201) is not located in close proximity, the processor (120) may determine whether there is a collision with an obstacle without a connection to the vehicle (201) and provide a danger alert. For example, the processor (120) may calculate the speed at which an obstacle approaches the electronic device (101) based on at least one of the distance between the recognized obstacle and the electronic device (101), the speed of the recognized obstacle, or the speed of the electronic device (101). The processor (120) may display the distance to the obstacle or the speed of the obstacle through a display (e.g., the display module (160) of FIG. 1). The processor (120) may identify the obstacle based on a vision camera, collected information, or sensor values, and provide obstacle guidance information based on the identified one. The obstacle guidance information indicates what the obstacle is, and may mean, for example, a person, animal, means of transportation, object, building, or pillar. The obstacle guidance information may include at least one of text, an image (e.g., an icon image of an obstacle), or a video. Alternatively, the processor (120) may provide obstacle guidance information or an obstacle collision risk through vibration or sound.

[0110] According to various embodiments, the processor (120) may display a different danger alert depending on the type of obstacle. For example, the processor (120) may distinguish whether the obstacle is a movable object (e.g., person, car, motorcycle, bicycle) (e.g., dynamic notation) or an immovable object (e.g., utility pole, railroad track, traffic light) (e.g., static notation). The processor (120) may provide at least one of the following as a danger alert: height of the obstacle (e.g., stairs, driveway, bump), warning indication based on the situation (e.g., roadway, railroad track, traffic light), estimated time of collision with the obstacle (e.g., countdown), color change according to the danger level (e.g., collision, danger, warning), size of the obstacle, or weight of the obstacle. For example, if the danger level is collision, the danger level may be high; if the danger level is danger, the danger level may be medium; and if the danger level is warning, the danger level may be low. The processor (120) may provide different obstacle collision risks depending on the danger level. For example, if the risk level is high, the processor (120) may provide a risk of collision with an obstacle using a red color, strong vibration, and a loud sound. If the risk level is medium, the processor (120) may provide a risk of collision with an obstacle using an orange color, medium vibration, and a medium sound. If the risk level is low, the processor (120) may provide a risk of collision with an obstacle using a yellow color, weak vibration, and a soft sound.

[0111] According to various embodiments, the processor (120) determines whether a wearable device (e.g., earphones, watch) is connected to the electronic device (101) and can provide obstacle guidance information or obstacle collision risk through the wearable device. For example, if the earphones and watch are connected, the processor (120) can output sound through the earphones and provide visual indication and vibration through the watch. The processor (120) can control the wearable device to send a danger alert command to the wearable device.

[0112] According to various embodiments, the processor (120) can determine the usage status of the electronic device (101) using a sensor module (176) or a camera module (180). The processor (120) can determine the usage status of the electronic device (101) based on sensor information obtained from the sensor module (176) or an image obtained from the camera module (180). When a user walks while looking at the electronic device (101), the sensing values ​​(e.g., speed, rotation value) may continuously change. Alternatively, the front camera of the electronic device (101) may be facing the user, and the rear camera of the electronic device (101) may continuously change the scene while moving, so the acquired image may continuously change. The processor (120) can determine that the user's gaze direction is toward the electronic device (101) when the posture of the electronic device (101) corresponds to a set posture, when a touch input is detected, or when a specific application is executed.

[0113] According to various embodiments, the processor (120) may provide a danger alert when the user's gaze direction is toward the electronic device (101). When the user's gaze direction is toward the electronic device (101), the processor (120) may display the danger alert through the display module (160), and when the user's gaze direction is not toward the electronic device (101), may provide a vibration for the danger alert through the haptic module (179) or output a sound for the danger alert through the sound output module (155).

[0114] When connected to the vehicle (201), in operation 511, the processor (120) can obtain obstacle information from the vehicle (201). When the processor (120) is located in close proximity to the vehicle (201), it can connect to the vehicle (201) via the communication module (190) and receive obstacle information from the vehicle (201). The obstacle information may include location information of the obstacle or speed information of the obstacle. Alternatively, the vehicle (201) may calculate the distance between the electronic device (101) and the obstacle or the speed of the obstacle based on the current location of the electronic device (101) regarding the recognized obstacle. At this time, the obstacle information may include the distance between the electronic device (101) and the obstacle or the speed of the obstacle. The vehicle (201) may determine what the obstacle is from HD map information or an acquired image and transmit the obstacle information by further including the type of obstacle (e.g., person, animal, object). According to various embodiments, depending on the implementation, the vehicle (201) may not transmit the recognized obstacle information to the electronic device (101). If the vehicle (201) does not transmit the recognized obstacle information to the electronic device (101), operation 511 may be omitted. This is merely an implementation issue and the invention is not limited by the description.

[0115] In operation 513, the processor (120) can track the direction of the user's gaze. The processor (120) can determine that the user's gaze direction is toward the electronic device (101) when the posture of the electronic device (101) corresponds to a set posture, when a touch input is detected, or when a specific application is executed. The posture of the electronic device (101) can be obtained based on sensor values ​​obtained using a gyroscope sensor. The processor (120) can determine that the user's gaze direction is toward the electronic device (101) when the posture of the electronic device (101) is determined to be a posture of using the electronic device (101) while walking based on sensor values ​​obtained by an accelerometer sensor. The processor (120) can determine that the user's gaze direction is toward the electronic device (101) when a touch input is detected while the user is determined to be walking. For example, in the case of a music application, the user's gaze direction may not be directed toward the electronic device (101) while music is playing. However, in the case of a specific application (e.g., a game application, a YouTube application), the user's gaze direction may be directed toward the electronic device (101) while the application is running. The processor (120) can determine whether the currently running application corresponds to a specific application based on the application identifier or type set in the application.

[0116] In operation 515, the processor (120) can determine whether there is a collision with an obstacle. If the processor (120) is connected to the vehicle (201), it can determine whether there is a collision with an obstacle by further considering obstacle information received from the vehicle (201). The processor (120) can calculate the distance between the electronic device (101) and the obstacle recognized by the electronic device (101), or between the obstacle recognized by the vehicle (201) and the electronic device (101), or calculate the speed at which the obstacle is approaching. The processor (120) can determine whether there is a collision with the recognized obstacle based on the distance or speed to the obstacle. Since operation 515 is identical or similar to operation 507 described above, a detailed description may be omitted.

[0117] In operation 517, the processor (120) may provide a danger alert through the vehicle (201) or the electronic device (101) based on the direction of gaze. For example, the processor (120) may provide a danger alert through the display module (160), the haptic module (179), or the sound output module (155) when the user's gaze direction is toward the electronic device (101). Alternatively, the processor (120) may control the provision of a danger alert through the vehicle (201) when the user's gaze direction is not toward the electronic device (101). Alternatively, the processor (120) may control the provision of a danger alert through the electronic device (101) and the vehicle (201) when the user's gaze direction is toward the electronic device (101). Alternatively, the processor (120) can control the provision of a danger alert through a wearable device and a vehicle (201) connected to the electronic device (101) when the user's gaze direction is toward the electronic device (101).

[0118] FIGS. 6a and 6b are drawings illustrating an example of providing user feedback through an electronic device or vehicle according to various embodiments.

[0119] FIG. 6a is a diagram illustrating an example of providing a danger alert through an electronic device or vehicle according to a change in the position of the electronic device.

[0120] Referring to FIG. 6a, an electronic device according to various embodiments (e.g., the electronic device (101) of FIG. 1) can support safe walking of a user while the user moves in a first direction of movement (615) by using at least one of a sensor module (e.g., the sensor module (176) of FIG. 1), a camera (e.g., the camera module (180) of FIG. 1), or a microphone (e.g., the input module (150) of FIG. 1). For example, when the user is at a distance of a first distance (601) from a vehicle (e.g., the vehicle (201) of FIG. 2), the electronic device (101) can recognize an obstacle located within a first distance (251) using a microphone, recognize an obstacle located within a second distance (253) using a vision camera, recognize an obstacle located within a third distance (255) using a short-range communication method, and recognize an obstacle located within a fourth distance (257) using a lidar. The first distance (251) is shorter than the second distance (253), the second distance (253) is shorter than the third distance (255), and the third distance (255) may be shorter than the fourth distance (257). While the electronic device (101) is at a distance of the first distance (601) from the vehicle (201), it can recognize an obstacle, determine whether there is a collision (610) with the recognized obstacle, and provide a danger alert based on the result of the determination.

[0121] While the electronic device (101) is separated from the vehicle (201) by a first distance (601), the electronic device (101) can track the user's gaze direction and determine that the user's gaze direction is directed toward the electronic device (101) when the posture of the electronic device (101) corresponds to a set posture, a touch input is detected, or a specific application is executed. The electronic device (101) can provide a danger alert based on the gaze direction. When the user's gaze direction is directed toward the electronic device (101), the electronic device (101) can provide a danger alert through a display module (160), a haptic module (179), or an acoustic output module (155). Alternatively, when the user's gaze direction is not directed toward the electronic device (101), the electronic device (101) can control the provision of a danger alert through a wearable device connected to the electronic device (101).

[0122] While the user continues to move in the second direction of movement (635), if the user is separated from the vehicle (201) by a second distance (603), the electronic device (101) can determine whether it is possible to connect with the vehicle (201). When the electronic device (101) enters within a certain distance where it can be connected (or paired) with the vehicle (201), it can be connected to the vehicle (201) via short-range wireless communication. While the electronic device (101) is separated from the vehicle (201) by a second distance (603), it can recognize an obstacle, determine whether there is a collision with the recognized obstacle (630), and provide a danger alert based on the determination result. When the electronic device (101) is connected to the vehicle (201), it can receive obstacle information from the vehicle (201), determine whether there is a collision with the obstacle (630) based on the obstacle information recognized by the electronic device (101) and the received obstacle information, and provide a danger alert based on the determination result. While the electronic device (101) is at a distance of 201 and 603, it can track the user's gaze direction and provide a danger alert based on the gaze direction.

[0123] When the vehicle (201) is separated from the electronic device (101) by a second distance (603) and connected to the electronic device (101), it receives the user's expected travel path or the current location of the electronic device (101) from the electronic device (101), recognizes an obstacle to determine whether there is an obstacle collision (630), and provides a danger alert based on the determination result. The second distance (603) may be the distance at which the vehicle (201) can be connected to the electronic device (101). The vehicle (201) can calculate the distance between the obstacle and the electronic device (101) based on the information received from the electronic device (101), calculate the speed of the obstacle, and determine whether there is an obstacle collision. The vehicle (201) can provide a danger alert by turning on the lights or sounding the horn.

[0124] When the vehicle (201) is at a third distance (605), the electronic device (101) or the vehicle (201) may recognize an obstacle and determine whether there is an obstacle collision (650), and provide a danger alert based on the determination result. When the vehicle (201) is at a third distance (605), the electronic device (101) may track the user's gaze direction and provide a danger alert based on the gaze direction. For example, the electronic device (101) may control the vehicle (201) to provide a danger alert when the user's gaze direction is not directed toward the electronic device (101). The electronic device (101) may control the electronic device (101) and the vehicle (201) to provide a danger alert when the user's gaze direction is directed toward the electronic device (101). Alternatively, the electronic device (101) can be controlled to provide a danger alert through a wearable device and a vehicle (201) connected to the electronic device (101) when the user's gaze direction is toward the electronic device (101).

[0125] When the vehicle (201) is separated from the electronic device (101) by a third distance (605), it may receive the user's expected travel path or the current location of the electronic device (101) from the electronic device (101), recognize an obstacle to determine whether there is an obstacle collision (630), and provide a danger alert based on the determination result. The vehicle (201) may provide a danger alert by turning on the lights or sounding the horn.

[0126] FIG. 6b is a diagram illustrating an example of providing a risk alert through an electronic device.

[0127] Referring to FIG. 6b, the electronic device (101) can determine an expected travel path (670) based on the location information of the vehicle (201) and the current location of the electronic device (101). The location information of the vehicle (201) or a previous travel path may be stored in the memory of the electronic device (101) (e.g., the memory (130) in FIG. 1). The electronic device (101) can calculate the expected travel path (670) by further considering the previous travel path. The electronic device (101) can recognize obstacles on the expected travel path (670) and determine whether there is a collision with the recognized obstacles. For example, the electronic device (101) can recognize obstacles (e.g., people) based on map information based on the current location of the electronic device (101), sensor values ​​obtained from a sensor module (e.g., the sensor module (176) in FIG. 1), and information obtained through short-range wireless communication. The electronic device (101) can recognize an obstacle located within a first distance (251) using a microphone, recognize an obstacle located within a second distance (253) using a vision camera, and recognize an obstacle located within a third distance (255) using a short-range communication method.

[0128] In the following description, the recognized obstacle will be described as an electronic device (102). The electronic device (101) can communicate with another user's electronic device (102) via short-range wireless communication (e.g., V2X, UWB) or can recognize the electronic device (102) via LiDAR. The electronic device (101) can calculate (or predict) the expected movement path (690) of the electronic device (102) based on the distance from the electronic device (102) or the speed of the electronic device (102). The electronic device (101) can determine whether there is a collision (691) with the electronic device (102) based on at least one of the expected movement path (670), the speed of the electronic device (101), the expected movement path (690) of the electronic device (102), the distance from the electronic device (102), or the speed of the electronic device (102). The electronic device (101) can provide a risk alert based on the result of the determination.

[0129] FIG. 7 is a flowchart (700) illustrating a method of providing user feedback in a vehicle according to various embodiments.

[0130] Referring to FIG. 7, in operation 701, a processor (e.g., processor (330) of FIG. 3) of a vehicle (e.g., vehicle (201) of FIG. 2) according to various embodiments may be connected to at least one electronic device (e.g., electronic device (101) of FIG. 1). The vehicle (201) may include (or be equipped with) a walking assistance device (or system) (301). The processor (330) may store information about the electronic device (101) connected at least once in memory (e.g., memory (340) of FIG. 3). For example, the memory (340) may store information on the electronic device (101) of the first user who is the owner of the vehicle (201), information on the second electronic device (e.g., electronic device (102) of FIG. 1) of the second user (e.g., mother) who is a family member of the first user, information on the third electronic device (e.g., electronic device (104) of FIG. 1) of the third user (e.g., father), and information on the fourth electronic device of the fourth user (e.g., friend) who is an acquaintance of the first user. The at least one electronic device may include at least one of the first user's electronic device (101), the second electronic device (102), the third electronic device (104), and the fourth electronic device. The processor (330) may be connected to the at least one electronic device via short-range wireless communication (e.g., Bluetooth, UWB) through a communication module (e.g., communication module (360) of FIG. 3). For convenience of explanation, the at least one electronic device will be described as the electronic device (101) below. However, the present invention is not limited by the description.

[0131] In operation 703, the processor (330) can turn on the ignition of the vehicle (201). When the processor (330) is connected to the electronic device (101), it can receive a command (or instruction) from the electronic device (101) to turn on the power of the vehicle (201) or to turn on the ignition of the vehicle (201). The processor (330) can turn on the power of the vehicle (201) and turn on the ignition according to the command of the electronic device (101).

[0132] In operation 705, the processor (330) may receive location information from at least one electronic device (e.g., electronic device (101)) to determine the user's expected travel path. The expected travel path may be determined (or predicted, calculated) based on the location information of the vehicle (201) and the location information of the electronic device (101) to the vehicle (201). Alternatively, the processor (330) may receive the user's expected travel path along with the location information of the electronic device (101) (e.g., current location) from the electronic device (101) via a communication module (360). The processor (330) may receive the location information of the electronic device (101) in real time, periodically, or optionally. Alternatively, the processor (330) may further receive the previous travel path of the electronic device (101) and use it to determine the expected travel path. The above previous movement path can be calculated based on the location information of the electronic device (101) while the electronic device (101) is moving away from the vehicle (201).

[0133] Although the drawing indicates that operation 705 is performed after operation 703, operation 705 may be performed after being connected to the electronic device (101) (e.g., operation 701). Operation 705 may also be performed simultaneously with operation 701 or operation 703. This is merely an implementation issue and does not limit the invention.

[0134] In operation 707, the processor (330) can recognize obstacles. The processor (330) can recognize obstacles on the predicted path of travel based on ADAS. For example, the processor (330) can recognize surrounding obstacles based on at least one of a communication module (360), a camera (e.g., vision camera (310) of FIG. 3), or a sensor module (e.g., sensor module (320) of FIG. 3). For example, the processor (330) can acquire map information (e.g., HD map information) from a map information providing server (e.g., server (108) of FIG. 1) and recognize obstacles located in front of and behind the vehicle (201) through an ultrasonic sensor, or recognize obstacles through a camera (e.g., front camera, vision camera), lidar, laser, or radar. Alternatively, the processor (330) can recognize obstacles located around the vehicle (201) by obtaining information from surrounding electronic devices, surrounding vehicles, and surrounding infrastructure through short-range communication such as V2X or UWB. The processor (330) can recognize obstacles based on at least one of map information, sensor values, images, or collected information. The processor (330) can transmit the recognized obstacle information to the electronic device (101). The obstacle information may include location information of the obstacle or speed information of the obstacle.

[0135] In operation 709, the processor (330) can obtain user gaze information from at least one electronic device (e.g., electronic device (101)). The electronic device (101) can determine whether the user is looking at the electronic device (101) because it may be dangerous if the user walks while looking at the electronic device (101). The electronic device (101) can determine whether the user's gaze direction is toward the electronic device (101) based on sensor information obtained from a sensor module (176), such as a light sensor, a proximity sensor, a motion sensor, an accelerometer, or a gyroscope, or an image obtained from a camera module (180). The processor (330) can receive user gaze information from the electronic device (101) in real time, periodically, or optionally.

[0136] Although the drawing indicates that operation 709 is performed after operation 707, operation 709 may be performed after operation 703. Alternatively, operation 709 may be performed simultaneously with operation 705. This is merely an implementation issue and does not limit the present invention.

[0137] In operation 711, the processor (330) can determine whether there is a collision with an obstacle. The processor (330) can determine whether there is a collision with an obstacle based on the location information of the recognized obstacle or the electronic device (101). The processor (330) can receive obstacle information recognized by the electronic device (101) from the electronic device (101). The obstacle information can receive at least one of the location information of the obstacle, the speed information of the obstacle, the type information of the obstacle, or the distance information between the obstacle and the electronic device (101). The processor (330) can determine whether there is a collision with an obstacle based on at least one of the obstacle information recognized by the vehicle (201), the obstacle information received from the electronic device (101), or the location information of the electronic device (101).

[0138] In operation 713, the processor (330) can notify of a risk of collision with an obstacle based on line of sight information. If the processor (330) anticipates a collision with an obstacle, it can provide a risk of collision with an obstacle through an output module. For example, the output module may include the lights or horn of the vehicle (201). The processor (330) can provide a risk of collision with an obstacle by turning on the lights or sounding the horn. For example, the processor (330) can provide different directions of the lights based on the location of the obstacle or the location information of the electronic device (101). The processor (330) can turn on the left light if the obstacle is located to the left of the vehicle (201), and turn on the right light if the obstacle is located to the right of the vehicle (201). Alternatively, the processor (330) may turn on the left light when the electronic device (101) is located to the left of the vehicle (201), and turn on the right light when the electronic device (101) is located to the right of the vehicle (201). The processor (330) may provide different directions of light and sound the horn based on the location of an obstacle or the location information of the electronic device (101).

[0139] According to various embodiments, the processor (330) may provide different directions of light, intensity, or horn volumes depending on the risk level. For example, if the risk level is collision, the processor (330) may determine the risk to be high and output a light of strong intensity or a horn of loud sound. If the risk level is high, the processor (330) may output different directions of light. If the risk level is danger, the processor (330) may determine the risk to be medium and output a light of medium intensity or a horn of medium sound. If the risk level is medium, the processor (330) may output a light in one direction. If the risk level is warning, the processor (330) may determine the risk to be low and output a light of weak intensity or a horn of quiet sound. Alternatively, if the lights of the vehicle (201) can output different colors, the processor (330) may output different colors of light depending on the risk level.

[0140] FIG. 8 is a diagram illustrating an example of providing user feedback through an electronic device or vehicle based on a change in the position of an electronic device according to various embodiments.

[0141] Referring to FIG. 8, an electronic device according to various embodiments (e.g., the electronic device (101) of FIG. 1) can implement APAS (advanced pedestrian assistance systems), and a vehicle (e.g., the vehicle (201) of FIG. 2) can implement ADAS to detect risk factors in all directions with the user as the central point. To this end, the electronic device (101) can obtain an expected travel path while moving toward the vehicle (201), recognize obstacles on the expected travel path, and notify of the recognized obstacle collision risk. The first expected travel path (801) may refer to a path where the user carrying the electronic device (101) moves toward the direction where the vehicle (201) is located. The second expected travel path (803) may refer to a path where the user carrying the electronic device (101) moves toward the direction away from the vehicle (201). The second expected travel path (803) may be the same as or different from the first expected travel path (801). The electronic device (101) can obtain a first predicted movement path (801) and, while moving sequentially from a first position (810) to a third position (830) on the first predicted movement path (801), recognize obstacles and notify of the recognized obstacle collision risk. The first position (810) may be further from the vehicle (201) than the second position (820), and the second position (820) may be further from the vehicle (201) than the third position (830). At the first position (810), the electronic device (101) can recognize obstacles located within a first distance (251) using a microphone, recognize obstacles located within a second distance (253) using a vision camera, and recognize obstacles located within a third distance (255) using a short-range communication method. At the first position (810), since the electronic device (101) cannot be connected to the vehicle (201), the electronic device (101) can independently recognize an obstacle on the first expected path (801), determine whether there is a collision with the recognized obstacle, and provide a danger alert.

[0142] The electronic device (101) can detect an obstacle at the second location (820), determine whether there is a collision with the detected obstacle, and provide a danger alert. When the electronic device (101) moves away from the vehicle (201) by a certain distance (805) at the second location (820), it can independently detect an obstacle on the first expected path (801), determine whether there is a collision with the detected obstacle, and provide a danger alert. When the electronic device (101) moves within a certain distance (805) of the vehicle (201) at the second location (820), it can move in conjunction with the vehicle (201) to determine whether there is a collision with an obstacle and provide a danger alert. When the electronic device (101) moves within a certain distance (805) of the vehicle (201), it can connect with the vehicle (201) to turn on the power of the vehicle (201) or turn on the ignition of the vehicle (201). The electronic device (101) can transmit a first expected travel path (801) or location information of the electronic device (101) (e.g., current location) to the vehicle (201) and receive obstacle information from the vehicle (201). The obstacle information may include information about obstacles recognized by the vehicle (201). The electronic device (101) can transmit obstacle information recognized by the electronic device (101) to the vehicle (201). When the vehicle (201) is connected to the electronic device (101), it can determine whether there is a collision with an obstacle based on the location information of the electronic device (101), the speed of the electronic device (101), the recognized obstacle, or the obstacle information received from the electronic device (101) and provide a danger alert.

[0143] The electronic device (101) can detect an obstacle at a third location (830), determine whether there is a collision with the detected obstacle, and provide a danger alert. At the third location (830), since the electronic device (101) is connected to the vehicle (201), the electronic device (101) can determine whether there is a collision with an obstacle in conjunction with the vehicle (201) and provide a danger alert. The electronic device (101) can determine whether there is a collision with an obstacle and provide a danger alert based on the location information of the electronic device (101), the speed of the electronic device (101), the detected obstacle, or the obstacle information received from the vehicle (201). The vehicle (201) can determine whether there is a collision with an obstacle and provide a danger alert based on the location information of the electronic device (101), the speed of the electronic device (101), the detected obstacle, or the obstacle information received from the electronic device (101).

[0144] The electronic device (101) can obtain a second predicted movement path (803) and, while moving sequentially to a fourth position (850) through a sixth position (870) on the second predicted movement path (803), recognize an obstacle and provide a warning of the recognized obstacle collision risk. The fourth position (850) is closer to the vehicle (201) than the fifth position (8650), and the fifth position (860) may be closer to the vehicle (201) than the sixth position (870). At the fourth position (850), since the electronic device (101) is connected to the vehicle (201), the electronic device (101) can recognize an obstacle on the second predicted movement path (803) in conjunction with the vehicle (201), determine whether there is a collision with the recognized obstacle, and provide a warning of the risk.

[0145] The electronic device (101) can detect an obstacle at a fourth location (840), determine whether to collide with the detected obstacle, and provide a danger alert. The electronic device (101) can transmit the obstacle information detected by the electronic device (101), the current location of the electronic device (101), and the second predicted travel path (803) to the vehicle (201). The vehicle (201) can determine whether to collide with an obstacle based on the location information of the electronic device (101), the speed of the electronic device (101), the detected obstacle, or the obstacle information received from the electronic device (101), and provide a danger alert. The vehicle (201) can detect an obstacle on the second predicted travel path (803) and transmit the detected obstacle information to the electronic device (101). The electronic device (101) can determine whether there is a collision with an obstacle based on the location information of the electronic device (101), the speed of the electronic device (101), the recognized obstacle, or obstacle information received from the vehicle (201) and provide a danger alert.

[0146] When the electronic device (101) is located within a certain distance (805) of the vehicle (201) at the fifth position (860), it can determine whether there is a collision with an obstacle in conjunction with the vehicle (201) and provide a danger alert. When the electronic device (101) moves away from the certain distance (805) of the vehicle (201) at the fifth position (860), it can independently recognize an obstacle on the second expected movement path (803), determine whether there is a collision with the recognized obstacle, and provide a danger alert. When the electronic device (101) moves away from the certain distance (805) of the vehicle (201) at the fifth position (860), it can turn off the power of the vehicle (201) or turn off the engine of the vehicle (201). The electronic device (101) can transmit a second expected travel path (801) or location information of the electronic device (101) (e.g., current location) to the vehicle (201) and receive obstacle information from the vehicle (201). The electronic device (101) can determine whether to collide with an obstacle based on the location information of the electronic device (101), the speed of the electronic device (101), the recognized obstacle, or the obstacle information received from the vehicle (201) and provide a danger alert.

[0147] The electronic device (101) can detect an obstacle at the sixth position (870), determine whether to collide with the detected obstacle, and provide a danger alert. At the sixth position (870), since the electronic device (101) is not connected to the vehicle (201), the electronic device (101) can independently determine whether to collide with an obstacle and provide a danger alert. The electronic device (101) can determine whether to collide with an obstacle and provide a danger alert based on the location information of the electronic device (101), the speed of the electronic device (101), or the detected obstacle.

[0148] FIG. 9 is a diagram illustrating an example of providing user feedback through a plurality of electronic devices connected to a vehicle according to various embodiments.

[0149] Referring to FIG. 9, a vehicle according to various embodiments (e.g., vehicle (201) of FIG. 2) may be connected to a plurality of electronic devices to provide a warning of the risk of collision with an obstacle. For example, the plurality of electronic devices may be electronic devices that have been connected to the vehicle (201) at least once, and may include, for example, a first electronic device (e.g., electronic device (101) of FIG. 1), a second electronic device (e.g., electronic device (102) of FIG. 1), and a third electronic device (e.g., electronic device (104) of FIG. 1). Hereinafter, the electronic device (101) will be described as the first electronic device (101), the electronic device (102) as the second electronic device (102), and the electronic device (104) as the third electronic device (104). For example, the first electronic device (101) may be the electronic device of the first user (e.g., the owner of the vehicle (201)), the second electronic device (102) may be the electronic device of the second user (e.g., family 1 of the first user), and the third electronic device (104) may be the electronic device of the third user (e.g., family 2 of the first user).

[0150] The first electronic device (101) can recognize obstacles located within a first distance (251) using a microphone, recognize obstacles located within a second distance (253) using a vision camera, recognize obstacles located within a third distance (255) using a short-range communication method, and recognize obstacles located within a fourth distance (257) using a lidar. The second electronic device (102) and the third electronic device (104) can also recognize obstacles in the same or similar way as the first electronic device (101).

[0151] The first electronic device (101) may obtain a first expected travel path (911) based on location information of the vehicle (201) or location information of the first electronic device (101) at a first location (940), recognize an obstacle on the first expected travel path (911), determine whether there is a collision (913) with the recognized obstacle, and provide a danger alert based on the determination result. When the first electronic device (101) enters within a certain distance of the vehicle (201), it may transmit the first expected travel path (911) and location information of the first electronic device (101) to the vehicle (201) and receive first obstacle information from the vehicle (201). The vehicle (201) may recognize an obstacle based on the first expected travel path (911) and location information of the first electronic device (101) and transmit the recognized first obstacle information to the first electronic device (101). The first electronic device (101) can determine whether there is a collision (913) with a recognized obstacle by further considering the first obstacle information. The vehicle (201) can provide a different direction of light based on the location of the obstacle or the location information of the first electronic device (101). The vehicle (201) can turn on the left light or sound the horn as a danger alert to the first electronic device (101).

[0152] The second electronic device (102) may obtain a second expected travel path (931) based on the location information of the vehicle (201) or the location information of the second electronic device (102) at the second location (930), recognize an obstacle on the second expected travel path (931), determine whether there is a collision (933) with the recognized obstacle, and provide a danger alert based on the result of the determination. The second expected travel path (931) may be the same as or different from the first expected travel path (911). When the second electronic device (102) enters within a certain distance of the vehicle (201), it may transmit the second expected travel path (931) and the location information of the second electronic device (102) to the vehicle (201) and receive second obstacle information from the vehicle (201). The vehicle (201) can recognize an obstacle based on the second expected path (931) and the location information of the second electronic device (102), and transmit the recognized second obstacle information to the second electronic device (102). The second electronic device (102) can determine whether there is a collision (933) with the recognized obstacle by further considering the second obstacle information. The vehicle (201) can provide a different direction of light based on the location of the obstacle or the location information of the second electronic device (102). The vehicle (201) can turn on the left light or sound the horn as a danger alert to the second electronic device (102).

[0153] The third electronic device (104) may obtain a third expected travel path (951) based on the location information of the vehicle (201) or the location information of the third electronic device (104) at a third location (950), recognize an obstacle on the third expected travel path (951), determine whether there is a collision (953) with the recognized obstacle, and provide a danger alert based on the determination result. The third expected travel path (951) may be the same as or different from the second expected travel path (931) or the first expected travel path (911). When the third electronic device (104) enters within a certain distance of the vehicle (201), it may transmit the third expected travel path (951) and the location information of the third electronic device (104) to the vehicle (201) and receive third obstacle information from the vehicle (201). The vehicle (201) can recognize an obstacle based on the third expected path (951) and the location information of the third electronic device (104), and transmit the recognized third obstacle information to the third electronic device (104). The third electronic device (104) can determine whether there is a collision (953) with the recognized obstacle by further considering the third obstacle information. The vehicle (201) can provide a different direction of light based on the location of the obstacle or the location information of the third electronic device (104). The vehicle (201) can turn on the right light or sound the horn as a danger alert to the third electronic device (104).

[0154] FIG. 10 is a flowchart illustrating a method of providing user feedback through a vehicle and a plurality of electronic devices according to various embodiments.

[0155] Referring to FIG. 10, a first electronic device (1010) according to various embodiments (e.g., the electronic device (101) of FIG. 1) can determine a first expected movement path. The first expected movement path may refer to a path taken by a first user possessing the first electronic device (1010) while moving toward a vehicle (e.g., the vehicle (201) of FIG. 2) or while moving away from (or moving away from) the vehicle (201). The first electronic device (1010) can calculate the first expected movement path based on at least one of the location information of the vehicle (201) stored in memory (e.g., the memory (130) of FIG. 1) and the current location of the first electronic device (1010). The first electronic device (1010) may further store a previous movement path in memory (130). The above previous movement path can be calculated based on the change in position of the first electronic device (1010) over time while moving away from the vehicle (201) after the first electronic device (1010) has stored the position information of the vehicle (201). The first electronic device (1010) can determine the first expected movement path by further considering the above previous movement path.

[0156] In operation 1003, the first electronic device (1010) can recognize obstacles on the first expected path of movement. The obstacles may hinder or endanger the safe walking of the user. For example, the obstacles may refer to utility poles, traffic lights, railroad tracks, signboards, other users, bicycles, motorcycles, cars, etc. The first electronic device (1010) can acquire the current location of the first electronic device (1010) in real time through a location recognition module and can acquire map information within a set distance (e.g., 1m, 3m, 5m) based on the current location from a map information providing server (e.g., server (108) of FIG. 1). The first electronic device (1010) can recognize an obstacle based on at least one of the map information, a sensor value obtained from a sensor module (e.g., sensor module (176) of FIG. 1), an image obtained from a vision camera (e.g., camera module (180) of FIG. 1) or a sound obtained from a microphone (e.g., input module (150) of FIG. 1).

[0157] In operation 1005, the first electronic device (1010) can determine whether it is located in close proximity to the vehicle (201). If it is located in close proximity to the vehicle (201) (e.g., within a certain distance (e.g., within a certain distance (230) of FIG. 2), the first electronic device (1010) can wirelessly connect with the vehicle (201). If the first electronic device (1010) is not located in close proximity to the vehicle (201), it may not be able to connect with the vehicle (201). If the first electronic device (1010) is located in close proximity to the vehicle (201), it can perform operation 1007, and if it is not located in close proximity to the vehicle (201), it can perform operation 1015.

[0158] When located in close proximity to the vehicle (201), in operation 1007, the first electronic device (1010) can be connected to the vehicle (201). For example, the first electronic device (1010) can be connected to the vehicle (201) via short-range wireless communication such as Bluetooth, low-power Bluetooth, Wi-Fi, Wi-Fi Direct, or UWB. The first electronic device (1010) can be connected to the vehicle (201) by transmitting subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196) of FIG. 1 to the vehicle (201). Once wirelessly connected to the vehicle (201), the first electronic device (1010) can control the vehicle (201) to turn on the power or start the vehicle (201).

[0159] In operation 1009, the first electronic device (1010) can transmit location information of the first electronic device (1010) to the vehicle (201). The first electronic device (1010) can transmit the current location of the first electronic device (1010) to the vehicle (201) in real time, periodically, or optionally. Alternatively, the first electronic device (1010) may transmit the first expected travel path along with the current location of the first electronic device (1010).

[0160] In operation 1002, a second electronic device (1020) according to various embodiments (e.g., the electronic device (102) of FIG. 1) may determine a second expected movement path. The second expected movement path may refer to a path taken by a second user possessing the second electronic device (1020) while moving toward the direction where the vehicle (201) is located or while moving away from (or away from) the vehicle (201). The second expected movement path may be the same as or different from the first expected movement path. The second electronic device (1020) may calculate the second expected movement path based on at least one of the location information of the vehicle (201) stored in memory (e.g., the memory (130) of FIG. 1) and the current location of the second electronic device (1020). The second electronic device (1020) may calculate the second expected movement path by further considering the previous movement path stored in memory (130). The above previous movement path can be calculated based on the change in position of the second electronic device (1020) over time while moving away from the vehicle (201) after the second electronic device (1020) has stored the location information of the vehicle (201).

[0161] In operation 1004, the second electronic device (1020) can recognize an obstacle on the second expected path. The second electronic device (1020) can acquire the current location of the second electronic device (1020) in real time and acquire map information within a set distance (e.g., 1m, 3m, 5m) based on the current location. The second electronic device (1020) can recognize the obstacle based on at least one of the map information, a sensor value acquired from a sensor module (e.g., sensor module (176) of FIG. 1), an image acquired from a vision camera (e.g., camera module (180) of FIG. 1), or a sound acquired from a microphone (e.g., input module (150) of FIG. 1).

[0162] In operation 1006, the second electronic device (1020) can determine whether it is located in close proximity to the vehicle (201). If it is located in close proximity to the vehicle (201) (e.g., within a certain distance (e.g., within a certain distance (230) of FIG. 2), the second electronic device (1020) can wirelessly connect with the vehicle (201). If the second electronic device (1020) is not located in close proximity to the vehicle (201), it may not be able to connect with the vehicle (201). If the second electronic device (1020) is located in close proximity to the vehicle (201), it can perform operation 1008, and if it is not located in close proximity to the vehicle (201), it can perform operation 1016.

[0163] When located in close proximity to the vehicle (201), in operation 1008, the second electronic device (1020) can be connected to the vehicle (201). For example, the second electronic device (1020) can be connected to the vehicle (201) via short-range wireless communication such as Bluetooth, low-power Bluetooth, Wi-Fi, Wi-Fi Direct, or UWB. The second electronic device (1020) can be connected to the vehicle (201) by transmitting subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196) of FIG. 1 to the vehicle (201). Once wirelessly connected to the vehicle (201), the second electronic device (1020) can control the vehicle (201) to turn on the power or start the vehicle (201).

[0164] In operation 1012, the second electronic device (1020) can transmit the second location information of the second electronic device (1020) to the vehicle (201). The second electronic device (1020) can transmit the current location of the second electronic device (1020) to the vehicle (201) in real time, periodically, or optionally. Alternatively, the second electronic device (1020) may transmit the second expected travel path along with the current location of the second electronic device (1020).

[0165] Although the drawings illustrate that operations 1001 to 1009 are performed before operations 1002 to 1012, operations 1002 to 1012 may be performed before operations 1001 to 1009, or operations 1002 to 1012 may be performed simultaneously with operations 1001 to 1009. The examples are provided only to aid in understanding the invention and do not limit the invention.

[0166] In operation 1010, the vehicle (201) can determine a first expected travel path and a second expected travel path. The vehicle (201) can determine the first expected travel path based on the current location of the first electronic device (1010) and the location information of the vehicle (201). The vehicle (201) can determine the second expected travel path based on the current location of the second electronic device (1020) and the location information of the vehicle (201). Alternatively, the vehicle (201) may receive the first expected travel path from the first electronic device (1010) or receive the second expected travel path from the second electronic device (1020). If the vehicle (201) receives the expected travel paths from the first electronic device (1010) and the second electronic device (1020), respectively, operation 1010 may be omitted.

[0167] In operation 1011, the vehicle (201) can detect (or recognize) surrounding obstacles based on ADAS. The vehicle (201) may include the walking assistance device (301) of FIG. 3. The vehicle (201) can recognize surrounding obstacles of the first electronic device (1010) based on at least one of the current location of the first electronic device (1010), the first expected movement path, sensor values ​​(e.g., obtained from the sensor module (320)), images (e.g., obtained from the sensor module (320)), or collected information (e.g., obtained from the communication module (360) of FIG. 3). Additionally, the vehicle (201) can recognize obstacles around the second electronic device (1020) based on at least one of the current location of the second electronic device (1020), the second expected path of movement, sensor values ​​(e.g., obtained from the sensor module (320)), images (e.g., obtained from the sensor module (320)), or collected information (e.g., obtained from the communication module (360) of FIG. 3).

[0168] In operation 1013, the vehicle (201) may transmit first obstacle information, including surrounding obstacles of the first electronic device (1010), to the first electronic device (1010). The first obstacle information may include location information or speed information of obstacles located (or existing) around the first electronic device (1010). Alternatively, the vehicle (201) may calculate the distance between the first electronic device (1010) and the obstacle or the speed of the obstacle based on the current location of the first electronic device (1010) regarding the recognized obstacle. At this time, the first obstacle information may include the distance between the first electronic device (1010) and the obstacle or the speed of the obstacle. The vehicle (201) may determine what the obstacle is from HD map information or acquired images and transmit the first obstacle information with additional information including the type of obstacle (e.g., person, animal, object).

[0169] In operation 1014, the vehicle (201) may transmit second obstacle information, including surrounding obstacles of the second electronic device (1020), to the second electronic device (1020). The second obstacle information may include location information or speed information of obstacles located (or existing) around the second electronic device (1020). The second obstacle information may be the same or different from the first obstacle information. Alternatively, the vehicle (201) may calculate the distance between the second electronic device (1020) and the obstacle or the speed of the obstacle based on the current location of the second electronic device (1020) regarding the recognized obstacle. In this case, the second obstacle information may include the distance between the second electronic device (1020) and the obstacle or the speed of the obstacle. The vehicle (201) may determine what the obstacle is from HD map information or acquired images and transmit the second obstacle information with additional information including the type of obstacle (e.g., person, animal, object).

[0170] In operation 1015, the first electronic device (1010) can determine whether there is a collision with an obstacle. If it is not located in close proximity to the vehicle (201), the first electronic device (1010) can determine whether there is a collision with an obstacle without being connected to the vehicle (201). The first electronic device (1010) can determine whether there is a collision with an obstacle by calculating the distance to the obstacle or by calculating the speed at which the obstacle approaches before being connected to the vehicle (201). When the first electronic device (1010) is connected to the vehicle (201), it can determine whether there is a collision with an obstacle by further considering the first obstacle information received from the vehicle (201). The first electronic device (1010) can calculate the distance between the first electronic device (1010) and the obstacle recognized by the first electronic device (1010), or the obstacle recognized by the vehicle (201), or calculate the speed at which the obstacle approaches. The first electronic device (1010) can determine whether there is a collision with the recognized obstacle based on the distance or speed to the obstacle.

[0171] In operation 1017, the first electronic device (1010) may notify of a danger based on a collision with the obstacle. For example, the obstacle guidance information may include at least one of text, an image, or a video, for example, indicating what the obstacle is. The first electronic device (1010) may display the obstacle guidance information or the obstacle collision risk on a display (e.g., the display module (160) of FIG. 1), provide vibrations regarding the obstacle guidance information or the obstacle collision risk by controlling the haptic module (179) of FIG. 1, or output a sound (e.g., a warning sound, voice) regarding the obstacle guidance information or the obstacle collision risk through the sound output module (155).

[0172] In operation 1016, the second electronic device (1020) can determine whether there is a collision with an obstacle. If it is not located in close proximity to the vehicle (201), the second electronic device (1020) can determine whether there is a collision with an obstacle without being connected to the vehicle (201). The second electronic device (1020) can determine whether there is a collision with an obstacle by calculating the distance to the obstacle or by calculating the speed at which the obstacle approaches before being connected to the vehicle (201). When the second electronic device (1020) is connected to the vehicle (201), it can determine whether there is a collision with an obstacle by further considering the second obstacle information received from the vehicle (201). The second electronic device (1020) can calculate the distance between the second electronic device (1020) and the obstacle recognized by the second electronic device (1020), or the obstacle recognized by the vehicle (201), or calculate the speed at which the obstacle approaches. The second electronic device (1020) can determine whether there is a collision with the recognized obstacle based on the distance or speed from the obstacle.

[0173] In operation 1018, the second electronic device (1020) may notify of a danger based on a collision with the obstacle. For example, the obstacle guidance information may include at least one of text, an image, or a video, for example, indicating what the obstacle is. The second electronic device (1020) may display the obstacle guidance information or the obstacle collision risk on a display (e.g., the display module (160) of FIG. 1), control the haptic module (179) of FIG. 1 to provide vibrations regarding the obstacle guidance information or the obstacle collision risk, or output a sound (e.g., a warning sound, voice) regarding the obstacle guidance information or the obstacle collision risk through the sound output module (155).

[0174] Although the drawings describe operations 1015 and 1017 as being performed before operations 1016 and 1018, operations 1016 and 1018 may be performed before operations 1015 and 1017, or simultaneously. The examples are provided only to aid in understanding the invention and do not limit the invention.

[0175] In operation 1019, the vehicle (201) can determine whether there is a collision between the first electronic device (1010) and an obstacle, or whether there is a collision between the second electronic device (1020) and an obstacle. For example, the vehicle (201) can determine whether there is a collision with the recognized obstacle based on the location information of the recognized obstacle or the first electronic device (1010). The vehicle (201) can receive information about the obstacle recognized by the electronic device (101) from the first electronic device (1010), or receive location information of the first electronic device (1010). The received obstacle information may receive at least one of the location information of the obstacle, the speed information of the obstacle, the type information of the obstacle, or the distance information between the obstacle and the first electronic device (1010). The vehicle (201) can determine whether to collide with an obstacle based on at least one of obstacle information recognized by the vehicle (201), obstacle information received from the first electronic device (1010), or location information of the first electronic device (1010).

[0176] Alternatively, the vehicle (201) may determine whether to collide with the recognized obstacle based on the location information of the recognized obstacle or the second electronic device (1020). The vehicle (201) may receive obstacle information recognized by the electronic device (101) from the second electronic device (1020) or receive location information of the second electronic device (1020). The received obstacle information may receive at least one of the obstacle location information, obstacle speed information, obstacle type information, or distance information between the obstacle and the second electronic device (1020). The vehicle (201) may determine whether to collide with the obstacle based on at least one of the obstacle information recognized by the vehicle (201), obstacle information received from the second electronic device (1020), or location information of the second electronic device (1020).

[0177] In operation 1021, the vehicle (201) may provide a warning based on a collision with the obstacle. If a collision is expected between the first electronic device (1010) or the second electronic device (1020) and the obstacle, the vehicle (201) may turn on its lights or sound a horn. The vehicle (201) may provide a warning of a collision with the obstacle by turning on its lights or sounding its horn. For example, the vehicle (201) may turn on the left light or sound the horn as a warning of danger to the first electronic device (1010). The vehicle (201) may turn on the right light or sound the horn as a warning of danger to the second electronic device (1020). The vehicle (201) may provide different directions of the lights, intensity, or horn volumes depending on the level of danger.

[0178] FIGS. 11a and FIGS. 11b are drawings illustrating an example of guiding collision risk through an electronic device or vehicle according to various embodiments.

[0179] FIG. 11a is a drawing illustrating an example of guiding collision risk through an electronic device or vehicle according to various embodiments.

[0180] Referring to FIG. 11a, the first situation (1110) may be a state in which the first obstacle recognition radius (1113) of an electronic device (e.g., the electronic device (101) of FIG. 1) and the second obstacle recognition radius (1115) of a vehicle (e.g., the vehicle (201) of FIG. 2) partially overlap, so that the electronic device (101) and the vehicle (201) interact with each other to recognize obstacles and provide feedback on the risk of collision with obstacles. The electronic device (101) may recognize obstacles within the first obstacle recognition radius (1113) using a microphone (e.g., the acoustic output module (155) of FIG. 1), a sensor module (e.g., the sensor module (176) of FIG. 1), a camera (e.g., the camera module (180) of FIG. 1), or a communication module (e.g., the communication module (190) of FIG. 1). The vehicle (201) can detect obstacles within a second obstacle detection radius (1115) using a camera (e.g., vision camera (310) of FIG. 3), a sensor module (e.g., sensor module (320) of FIG. 3), or a communication module (e.g., communication module (360) of FIG. 3). The electronic device (101) can obtain an expected travel path (1107) based on location information of the electronic device (101) (e.g., location of user (1101)) or location information of the vehicle (201). The electronic device (101) can detect obstacles on the expected travel path (1107) and determine a collision risk (1130). The electronic device (101) can collect information from another vehicle (1103) or bicycle (1105) through a communication module (190) and calculate an expected travel path (1109) of the bicycle (1105) based on the collected information. The electronic device (101) can determine the collision risk (1130) based on the expected movement path (1170) of the electronic vehicle (101) and the expected movement path (1109) of the bicycle (1105).

[0181] According to various embodiments, the electronic device (101) determines whether the user (1101)’s gaze is directed toward the electronic device (101) and can provide guidance on the collision risk (1130) through the electronic device (101) or the vehicle (201) based on the user’s gaze. The electronic device (101) can determine that the user (1101)’s gaze direction is directed toward the electronic device (101) when the posture of the electronic device (101) corresponds to a set posture (e.g., a posture that appears when the user uses the electronic device while walking), when a touch input is detected, or when a specific application is executed. When the user (1101)’s gaze direction is directed toward the electronic device (101), the electronic device (101) can provide guidance on the collision risk (1130) through the vehicle (201). Alternatively, if the user (1101) is looking toward the electronic device (101), the electronic device (101) may provide a warning of a collision risk (1130) through a wearable device connected to the electronic device (101). Alternatively, if the user is looking toward the electronic device (101), the electronic device (101) may display a danger warning through a display module (160), and if the user is not looking toward the electronic device (101), may provide a vibration for the danger warning through a haptic module (179) or output a sound for the danger warning through a sound output module (155).

[0182] FIG. 11b is a drawing illustrating another example of guiding collision risk through an electronic device or vehicle according to various embodiments.

[0183] Referring to FIG. 11b, the second situation (1150) may be a state in which an obstacle is recognized and the risk of collision with the obstacle is fed back when the first obstacle recognition radius (1113) of the electronic device (101) and the second obstacle recognition radius (1115) of the vehicle (201) completely overlap. The electronic device (101) may recognize an obstacle within the first obstacle recognition radius (1113) using a microphone (e.g., acoustic output module (155) of FIG. 1), a sensor module (e.g., sensor module (176) of FIG. 1), a camera (e.g., camera module (180) of FIG. 1), or a communication module (e.g., communication module (190) of FIG. 1). The vehicle (201) can recognize an obstacle (1153) within a second obstacle recognition radius (1115) using a camera (e.g., vision camera (310) of FIG. 3), a sensor module (e.g., sensor module (320) of FIG. 3), or a communication module (e.g., communication module (360) of FIG. 3). The obstacle (1153) is not included in the first obstacle recognition radius (1113) but is included in the second obstacle recognition radius (1115) and can be recognized by the vehicle (201). The vehicle (201) can transmit information about the obstacle (1153) (or obstacle information) to an electronic device (101).

[0184] The electronic device (101) can determine the risk of collision (1151) with an obstacle (1153) based on the recognized obstacle and obstacle information received from the vehicle (201). The electronic device (101) can control the vehicle (201) to notify the risk of collision (1151) through the vehicle (201). For example, the vehicle (201) can provide the risk of collision (1151) by turning on the lights or sounding the horn (1157). Alternatively, the electronic device (101) can display obstacle guidance information (1155) regarding the risk of collision (1151) through the display module (160). Alternatively, the electronic device (101) can provide a vibration regarding the risk notification through the haptic module (179) or output a sound regarding the risk notification through the sound output module (155) based on the user's gaze direction, if the user's gaze direction is not directed toward the electronic device (101).

[0185] A method of operation of an electronic device (e.g., a walking support device (301) of FIG. 3) mounted on a vehicle (e.g., a vehicle (201) of FIG. 2) according to various embodiments of the present invention may include: connecting to an external electronic device (e.g., an electronic device (101) of FIG. 1) through a communication module of the electronic device (e.g., a communication module (360) of FIG. 3); receiving location information of the external electronic device from the external electronic device to determine an expected movement path; recognizing an obstacle on the expected movement path based on at least one of the location information of the external electronic device, an image obtained from a vision camera of the electronic device (e.g., a vision camera (310) of FIG. 3), a sensor value obtained from a sensor module of the electronic device (e.g., a sensor module (320) of FIG. 3), or information collected from another device existing on the road through the communication module; and determining a collision risk between the external electronic device and the obstacle and providing a risk notification through an output module of the electronic device.

[0186] The vehicle includes a horn, a first light positioned on the right side of the vehicle, or a second light positioned on the left side of the vehicle, and the provided operation may include driving either the first light or the second light as a danger alert based on at least one of the location of the external electronic device or the location of the obstacle, driving the first light and the second light in a different pattern, or driving the first light, the second light, and the horn.

[0187] The various embodiments of the present invention disclosed in this specification and drawings are provided merely as specific examples to facilitate the explanation of the technical content of the invention and to aid in understanding the invention, and are not intended to limit the scope of the invention. Accordingly, the scope of the present invention should be interpreted to include all modifications or variations derived based on the technical concept of the invention, in addition to the embodiments disclosed herein. Explanation of the symbols

[0188] 101: Electronic devices 120: Processor 130: Memory 176: Sensor Module 190: Communication module 201: Vehicle 301: Walking assistance device

Claims

Claim 1 In an electronic device mounted on a vehicle, a vision camera; a sensor module including lidar and an ultrasonic sensor; a communication module; an output module; and a memory. The system includes a processor operatively connected to the vision camera, the sensor module, the communication module, the output module, or the memory, wherein the processor, when it is determined that the distance between the electronic device and an external electronic device held by a user is within a specified distance, connects to the external electronic device through the communication module; when connected to the external electronic device, receives location information of the external electronic device from the external electronic device, determines an expected movement path based on the current location of the electronic device and the location information of the external electronic device, recognizes obstacles on the expected movement path based on the determined expected movement path, the location information of the external electronic device, an image acquired from the vision camera, a sensor value acquired from the sensor module, and information collected from other devices existing on the expected movement path through the communication module, transmits information regarding the recognized obstacles to the external electronic device, determines the risk of collision between the external electronic device and the obstacles, and provides a risk alert through the output module, wherein the obstacle recognition involves recognizing obstacles located within a first distance between the electronic device and the expected movement path through the ultrasonic sensor, and recognizing obstacles located between the electronic device and the expected An electronic device that recognizes an obstacle located within a second distance on a travel path and recognizes an obstacle located within a third distance on an expected travel path through V2X (vehicle to everything) or UWB (ultra wide band) communication, wherein the first distance is shorter than the second distance and the second distance is shorter than the third distance. Claim 2 In claim 1, the processor is an electronic device configured to turn on the power of the vehicle or turn on the ignition of the vehicle according to the control of the external electronic device. Claim 3 In claim 1, the output module comprises a horn, a first light positioned on the right side of the vehicle, or a second light positioned on the left side of the vehicle, and the processor is an electronic device configured to drive the first light or the second light differently or output a horn through the horn based on at least one of the location information of the external electronic device or the location of the obstacle. Claim 4 In claim 1, the processor is an electronic device configured to provide different directions of the vehicle's lights, light intensity, or horn volume output through the horn according to the risk level of the risk notification. Claim 5 delete Claim 6 An electronic device configured such that, in claim 1, the processor calculates the distance between the electronic device and the recognized obstacle or the speed of the obstacle based on the current location of the external electronic device, determines whether there is a collision with the obstacle based on at least one of the distance between the electronic device and the obstacle or the speed of the obstacle, and provides a risk alert based on the determination result. Claim 7 An electronic device configured such that, in claim 1, the processor transmits obstacle information including the recognized obstacle to the external electronic device, receives obstacle information recognized by the external electronic device from the external electronic device, and determines a collision risk between the external electronic device and the obstacle based on the recognized obstacle and the received obstacle information. Claim 8 In claim 7, the processor is an electronic device configured to acquire HD (high definition) map information from a server and to determine the risk of collision between the external electronic device and the obstacle based further on the acquired map information. Claim 9 In an electronic device, a sensor module; a vision camera; a communication module; a position recognition module; a memory; and includes a processor operatively connected to the sensor module, the vision camera, the communication module, the location recognition module, or the memory, wherein the processor acquires location information of the electronic device using the location recognition module, transmits the location information of the electronic device to an external electronic device mounted on the vehicle, identifies an expected travel path based on the location information of the external electronic device stored in the memory and the acquired location information, recognizes obstacles on the expected travel path based on the identified expected travel path, the location information of the electronic device, an image acquired from the vision camera, and a sensor value acquired from the sensor module, provides a danger alert based on the recognized obstacles, and when the electronic device moves within a certain distance of the vehicle, connects with the vehicle and provides a danger alert through the vehicle, wherein the obstacle recognition recognizes obstacles located within a first distance between the electronic device and the expected travel path using the microphone of the electronic device, recognizes obstacles located within a second distance between the electronic device and the expected travel path using the vision camera, and connects with the electronic device using V2X (vehicle to everything) or UWB (ultra wide band) communication An electronic device that recognizes an obstacle located within a third distance on the predicted movement path and uses lidar to recognize an obstacle located within a fourth distance on the predicted movement path with respect to the electronic device, wherein the first distance is shorter than the second distance, the second distance is shorter than the third distance, and the third distance is shorter than the fourth distance. Claim 10 In claim 9, the memory further stores a previous travel path obtained based on a change in the position of the electronic device while moving away from the vehicle, and the processor is configured to determine the expected travel path based further on the previous travel path. Claim 11 In claim 9, the processor is configured to provide the danger alert through at least one of a display module, a haptic module, or a speaker when not connected to the vehicle, and to provide the danger alert through the electronic device or the vehicle when connected to the vehicle. Claim 12 delete Claim 13 In claim 9, the processor is an electronic device configured to receive obstacle information from the vehicle when connected to the vehicle, determine whether there is a collision between the electronic device and the obstacle based on the recognized obstacle or the received obstacle information, and provide the risk alert based on the result of the determination. Claim 14 In paragraph 13, the processor is an electronic device configured to acquire HD (high definition) map information from a server and to determine whether there is a collision between the electronic device and an obstacle based further on the acquired map information. Claim 15 An electronic device configured such that, in claim 9, the processor tracks the direction of a user's gaze, provides the danger alert through at least one of a display module, a haptic module, or a speaker when the direction of the user's gaze is toward the electronic device, and provides the danger alert through the vehicle when the direction of the user's gaze is not toward the electronic device. Claim 16 In claim 15, the processor is an electronic device configured to determine that the user's gaze direction is toward the electronic device when the posture of the electronic device corresponds to a set posture, or when a touch input is detected, or when a specific application is executed, at least one of which is the case. Claim 17 In claim 9, the processor is an electronic device configured to determine whether it is connected to a wearable device, and if it is connected to the wearable device, to provide a risk alert through the wearable device. Claim 18 In paragraph 9, the processor is an electronic device configured to provide different risk alerts based on the risk level or type of obstacle. Claim 19 A method of operation of an electronic device mounted on a vehicle, wherein when it is determined that the distance between the electronic device and an external electronic device held by a user is within a specified distance, the operation of connecting to the external electronic device through the communication module of the electronic device; when connected to the external electronic device, the operation of receiving location information of the external electronic device from the external electronic device; the operation of determining an expected travel path based on the current location of the electronic device and the location information of the external electronic device; the operation of recognizing obstacles on the expected travel path based on the determined expected travel path, the location information of the external electronic device, an image acquired from the vision camera of the electronic device, a sensor value acquired from the sensor module of the electronic device, and information collected from other devices existing on the expected travel path through the communication module; and the operation of transmitting information regarding the recognized obstacles to the external electronic device. The method includes an operation of determining a collision risk between the external electronic device and the obstacle and providing a risk notification through the output module of the electronic device, wherein the operation of recognizing the obstacle is an operation of recognizing an obstacle located within a first distance between the electronic device and the expected travel path through an ultrasonic sensor, recognizing an obstacle located within a second distance between the electronic device and the expected travel path through a vision camera or lidar, and recognizing an obstacle located within a third distance between the electronic device and the expected travel path through V2X (vehicle to everything) or UWB (ultra wide band) communication, wherein the first distance is set to be shorter than the second distance and the second distance is set to be shorter than the third distance. Claim 20 In claim 19, the vehicle comprises a horn, a first light positioned on the right side of the vehicle, or a second light positioned on the left side of the vehicle, and the provided operation comprises driving either the first light or the second light as a danger alert based on at least one of the location of the external electronic device or the location of the obstacle, driving the first light and the second light in a different pattern, or driving the first light, the second light, and the horn.

Citation Information

Patent Citations

  • Degree of risk determination system, server, and computer program

    JP2018124812A

  • Method and device for providing advanced pedestrian assistance system to protect pedestrian preoccupied with smartphone

    KR1020200095338A

  • Method for probiding smart key service and electronic device thereof

    KR1020190024169A

  • A mobile device capable of pedestrian assistant monitoring

    KR1020200120819A

  • Pedestrian feedback system including bollard for preventing traffic accident of smombie and method for the same

    KR1020200129361A