System

Abstract

To provide a system for allowing a bicycle user to use a road without anxiety by providing a means for previously avoiding a contact accident of a bicycle with a pedestrian or another vehicle.SOLUTION: A system comprising: means for acquiring position information; means for acquiring a forward video with a camera; means for detecting an accident-prone area using the acquired position information; means for analyzing the acquired video in real time and detecting an obstacle; means for issuing a voice alert when an accident-prone area and an obstacle are detected; and means for automatically reducing a speed of a bicycle with respect to the obstacle.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Bicycle-related accidents remain common, and the main causes of these accidents are a lack of awareness of traffic rules and the lack of safety features on bicycles themselves. Bicycles, in particular, are difficult to equip with advanced safety devices like automobiles, and require careful balance, making them at high risk of collision. Therefore, systems are needed to enable cyclists to use roads safely. Specific measures are also needed to prevent collisions with pedestrians and other vehicles. [Means for solving the problem]

[0005] The system of the present invention includes a means for acquiring location information and capturing forward-facing video using a camera. It also provides a means for detecting accident-prone areas using the acquired location information and a means for analyzing the acquired video in real time to detect obstacles. It also includes a means for issuing an audio alert when an accident-prone area or obstacle is detected. It also includes a means for automatically slowing the bicycle when an obstacle is encountered, ensuring the user's safety. It also includes a means for transmitting location information to a server, which then returns information about accident-prone areas, and a means for analyzing video using a generative AI to calculate the relative speed and distance of an object. It also includes a means for providing specific deceleration modes for each obstacle (slow mode for pedestrians, left-hand mode for cyclists, and flashing light alert mode for cars), and a means for automatically slowing down only when it is confirmed that the user has both hands on the handlebars. This approach significantly improves the safety of cyclists and significantly reduces the risk of accidents.

[0006] "Location information" is data indicating a specific geographic location obtained using technology such as GPS.

[0007] A "camera" is a device that captures light and produces image or video data.

[0008] "Accident hotspots" are regions or areas where traffic accidents frequently occur based on historical data.

[0009] "Video" is digital data containing visual information captured by a camera.

[0010] "Real-time analysis" is a technology that processes acquired data either simultaneously with acquisition or within a very short time, and obtains analytical results.

[0011] An "obstacle" is any object or entity that may impede the progress of a bicycle, including pedestrians, other bicycles, and vehicles.

[0012] A "voice alert" is a notification method that uses voice to warn users of danger and caution.

[0013] "Automatic deceleration" is a control action in which the system reduces the vehicle's speed at its own discretion.

[0014] A "server" is a computer system that is connected to a network and provides data and services in response to requests from a large number of clients.

[0015] "Generative AI" is an algorithm that uses artificial intelligence technology to analyze input information and generate specific results.

[0016] "Relative velocity" refers to the difference in velocity between two objects, and is an indicator for comparing the velocity of moving objects in particular.

[0017] "Handlebar" means a device with a grip for controlling the direction of travel of a bicycle.

[0018] A "motor control device" is an electronic device that controls a vehicle's motor and adjusts its speed and direction.

[0019] "Crawl mode" is a driving mode in which the vehicle speed is temporarily reduced significantly.

[0020] The "left driving mode" is a driving mode in which the vehicle drives to the left.

[0021] "Light flashing alert mode" is a warning method that alerts other vehicles and pedestrians by flashing the bicycle's lights. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] ---

[0044] The present invention relates to a bicycle navigation system, and provides a support system for cyclists to ride safely. This system has functions to detect traffic accident hot spots and automatically slow down when encountering obstacles, and can provide specific measures for users to avoid accidents.

[0045] Explanation of program processing

[0046] 1. System Configuration

[0047] The system mainly consists of the following components:

[0048] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[0049] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[0050] User: An individual who uses a bicycle.

[0051] 2. Specific Program Processing

[0052] Location information acquisition and analysis

[0053] User:

[0054] Before getting on your bike, launch the navigation app on your device.

[0055] Device:

[0056] When the system starts up, it enables the GPS function and begins obtaining the current location.

[0057] The acquired current location information is sent to the server.

[0058] server:

[0059] Based on the received location information, the system compares it with a database to obtain information on nearby accident-prone areas.

[0060] The acquired information is returned to the device.

[0061] Device:

[0062] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[0063] Examples:

[0064] - A voice notification will say, "There is an accident-prone area ahead. Please be careful."

[0065] Video analysis and hazard detection

[0066] Device:

[0067] Uses the smartphone camera to capture images of the area ahead.

[0068] The acquired video is analyzed in real time using generative AI.

[0069] Detects the presence of obstacles such as pedestrians, bicycles, and cars.

[0070] Examples:

[0071] - If a pedestrian suddenly approaches ahead, it will be detected through video analysis.

[0072] Auto-slowdown and alert features

[0073] Device:

[0074] If a rapidly approaching obstacle is detected based on video analysis, the system instructs the bicycle's motor control device to automatically slow down.

[0075] Select a specific deceleration mode depending on the type of obstacle.

[0076] Slow down mode for pedestrians.

[0077] Use left-side avoidance mode for other bicycles.

[0078] For automobiles, the alert mode is set by flashing lights.

[0079] User:

[0080] You will be instructed to make sure you keep both hands on the wheel while slowing down.

[0081] Device:

[0082] Only with user confirmation will the motor controller gradually reduce the bike's speed.

[0083] Specific examples

[0084] Situation: Approaching an intersection while riding a bicycle

[0085] User:

[0086] Get on your bike and launch the app.

[0087] Device:

[0088] The current location is obtained using GPS and sent to the server.

[0089] server:

[0090] Information on accident-prone areas is acquired and sent to the terminal.

[0091] Device:

[0092] A voice message will be displayed saying, "Be careful before approaching the intersection."

[0093] A camera captures images of the area ahead and analyzes them using generative AI.

[0094] Video analysis detects sudden approach of pedestrians.

[0095] Device:

[0096] A creep mode command is sent to the motor controller and the user is instructed to keep both hands on the steering wheel.

[0097] Once it is confirmed that you are holding the steering wheel, a creep mode is activated and the vehicle gradually slows down.

[0098] In this way, the system of the present invention helps users to ride bicycles safely and significantly reduces the risk of accidents.

[0099] The processing flow will be explained below.

[0100] ---

[0101] Step 1:

[0102] User: Turns on the smartphone and launches the bicycle navigation app.

[0103] Device: When the app is launched, enable GPS and camera functions.

[0104] Step 2:

[0105] Device: Uses GPS to obtain current location information.

[0106] Device: Sends the acquired current location information to the server.

[0107] Step 3:

[0108] Server: Based on the received location information, the server compares it with a database of accident-prone areas to obtain information on nearby accident-prone locations.

[0109] Server: Returns the accident-prone location information obtained as a result of the matching to the terminal.

[0110] Step 4:

[0111] Terminal: Analyzes the accident-prone location information returned from the server and determines whether the current location is close to an accident-prone location.

[0112] Terminal: When approaching a nearby accident hotspot, the device will issue a voice alert to the user.

[0113] Step 5:

[0114] Device: The smartphone camera captures real-time images of the road ahead.

[0115] Device: The acquired video is temporarily stored in the internal memory, and analysis begins using the generation AI.

[0116] Step 6:

[0117] Device: Analyzes video using generative AI and detects obstacles such as pedestrians, bicycles, and cars.

[0118] Terminal: Calculates the distance and relative speed of detected obstacles.

[0119] Step 7:

[0120] Terminal: Based on the video analysis results, evaluate whether there is an approaching obstacle.

[0121] Terminal: Prepares to take specific action (slow mode, drive left mode, flash lights) if assessed as dangerous.

[0122] Step 8:

[0123] Device: Select slow-walk mode for pedestrians, drive left mode for cyclists, and flashing light alert mode for cars.

[0124] Terminal: Tells the motor controller to slow down or flash lights.

[0125] Step 9:

[0126] Device: A sensor checks whether the user has both hands on the steering wheel.

[0127] Terminal: If the driver is confirmed to be holding the steering wheel, the motor control unit will automatically begin a gradual deceleration process.

[0128] Step 10:

[0129] Terminal: Ensures that the bicycle's speed has been reduced to a safe level for obstacles.

[0130] User: Keep both hands on the steering wheel while slowing down and ensure safe driving.

[0131] This allows the system to automatically take appropriate measures before the user gets into a dangerous situation, supporting safe cycling.

[0132] Example 1

[0133] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0134] Systems designed to help cyclists ride safely face challenges due to their inadequate ability to detect frequent traffic accidents and obstacles. In particular, they lack the ability to obtain real-time location information, provide information about accident-prone areas and obstacles, and automatically adjust speed. They also need a way to avoid unnecessary slowdowns by obtaining user confirmation.

[0135] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0136] In this invention, the server includes means for acquiring location information, means for acquiring video of the area ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time and detecting obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle speed toward an obstacle, and means for executing the speed reduction when the user agrees to the speed reduction. This makes it possible to detect accident-prone areas and obstacles and provide appropriate alerts and deceleration to ensure safe cycling for cyclists.

[0137] "Means of obtaining location information" refers to a function that measures the current location using a smartphone, GPS device, etc. and obtains the data.

[0138] "Means for obtaining images of the front using a camera" refers to a function that collects images of the front of the bicycle in real time using a smartphone, camera device, etc.

[0139] "Means for detecting accident-prone areas" is a function that identifies areas where traffic accidents have occurred frequently in the past based on location information and extracts the necessary data.

[0140] "Means for analyzing and detecting obstacles in real time" refers to a function that instantly analyzes the images captured by the camera and identifies obstacles such as pedestrians, bicycles, and automobiles.

[0141] "Means for issuing audio alerts" is a function that issues an audio warning to the user when an accident-prone area or obstacle is detected.

[0142] "Means for automatically reducing the speed of the bicycle" is a function that automatically controls the speed of the bicycle in response to detected obstacles, ensuring safety.

[0143] The "means for slowing down when the user agrees to slow down" is a function that notifies the user of the need to slow down and reduces the speed only if the user complies with the instruction.

[0144] "Generative AI" is an artificial intelligence that performs real-time image analysis based on video data acquired by a camera to determine the type and location of obstacles.

[0145] This invention provides a bicycle navigation system to support cyclists in riding safely. In particular, it has the function of detecting traffic accident hotspots and automatically slowing down the vehicle when encountering obstacles, aiming to significantly reduce the risk of accidents. This system consists of three main elements: a server, a terminal, and a user.

[0146] System Components

[0147] server:

[0148] The server maintains a database of areas prone to traffic accidents and provides location information to the terminal. The server receives the location information and returns the information on areas prone to traffic accidents to the terminal.

[0149] Device:

[0150] The terminal is a mobile device such as a smartphone, and uses GPS and camera functions. When the system is started, the terminal begins acquiring its current location and acquires location information and images of the area ahead in real time. The acquired images are then analyzed using a generative AI model to detect obstacles and issue instructions for automatic deceleration.

[0151] User:

[0152] The user is an individual who rides a bicycle. Before riding, the user launches the navigation app and follows the alerts and instructions from the device.

[0153] Hardware and software used

[0154] GPS Module:

[0155] It is built into the device and acquires the current location.

[0156] camera:

[0157] It is built into the terminal and captures images of what is in front of it.

[0158] Generative AI models:

[0159] It runs on the device, analyzes images captured by the camera in real time, and detects obstacles.

[0160] Motor Controller:

[0161] A device for automatically adjusting the speed of a bicycle.

[0162] Specific Examples of the Invention

[0163] Situation: Approaching an intersection while riding a bicycle

[0164] User:

[0165] The user gets on the bicycle and starts the navigation app on the device. The user drives the bicycle following the instructions of the app.

[0166] Device:

[0167] When the navigation app is launched, it uses the GPS module to obtain the current location and sends that information to the server. Based on the information on accident-prone areas returned from the server, it notifies the user by voice, "There is an accident-prone area ahead. Please be careful." It also uses a camera to obtain images of the area ahead and analyzes them in real time using a generative AI model. Based on the analysis results, if the device detects a pedestrian approaching quickly, it instructs the motor control device to enter slow-down mode.

[0168] User:

[0169] When instructed to slow down, the user ensures they keep both hands on the steering wheel, then follows further instructions from the device to drive safely.

[0170] In this way, the system of the present invention combines real-time location information with video analytics to provide specific alerts and slow-down instructions to help cyclists avoid accidents, thereby increasing cyclist safety and significantly reducing the risk of accidents.

[0171] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0172] Step 1:

[0173] The user launches a navigation app. The input is the user tapping the navigation app on their smartphone. The output is the navigation app launching.

[0174] Specific operation:

[0175] The user taps the navigation app icon on the smartphone home screen.

[0176] Step 2:

[0177] The device uses the GPS module to obtain its current location. The input is satellite data received by the device's GPS module. The output is the current location data.

[0178] Specific operation:

[0179] The smartphone obtains the current location information from satellites and sends that information to the location management module within the app.

[0180] Step 3:

[0181] The current location information acquired by the device is sent to the server. The input is the current location data. The output is the location information data sent to the server.

[0182] Specific operation:

[0183] The GPS information is sent to the server using the terminal's network module.

[0184] Step 4:

[0185] Based on the location information received by the server, information on accident-prone areas is retrieved from the database. The input is the current location data sent from the device. The output is information on accident-prone areas.

[0186] Specific operation:

[0187] The server queries the database and retrieves the results.

[0188] Step 5:

[0189] The server returns the information about accident-prone areas it has acquired to the terminal. The input is the information about accident-prone areas acquired by the server. The output is the information about accident-prone areas that is sent to the terminal.

[0190] Specific operation:

[0191] The server obtains information about areas prone to accidents from the database and transmits it to the terminal via the network.

[0192] Step 6:

[0193] The terminal issues a voice alert to the user based on the returned information on accident-prone areas. The input is the accident-prone area information sent from the server. The output is the issuance of a voice alert.

[0194] Specific operation:

[0195] The device will notify you with a voice message saying, "There is an accident-prone area ahead. Please be careful."

[0196] Step 7:

[0197] The device uses the smartphone camera to capture images of the front. The input is the camera image data. The output is the captured front image data.

[0198] Specific operation:

[0199] The camera continuously captures images of the area ahead and stores the data in the device's memory.

[0200] Step 8:

[0201] The image captured by the device is input into a generative AI model and analyzed in real time. The input is the image data from the camera. The output is the analyzed obstacle data.

[0202] Specific operation:

[0203] The device inputs the video data into a generative AI model, which then analyzes the data to identify the type and location of obstacles.

[0204] Step 9:

[0205] The device detects obstacles based on the analysis results and instructs the bicycle's motor control device to slow down. The input is the analysis result from the generative AI model. The output is a command to the motor control device to slow down.

[0206] Specific operation:

[0207] If a pedestrian is detected, the device will issue an alert saying "Pedestrian detected, bicycle speed set to slow mode" and send a command to the motor control device to slow down.

[0208] Step 10:

[0209] Verify that the user keeps both hands on the steering wheel when instructed to slow down. The input is an audio alert. The output is the user gripping the steering wheel.

[0210] Specific operation:

[0211] The user follows the instruction "Please hold the steering wheel with both hands" and performs an operation to confirm that he or she is holding the steering wheel.

[0212] Step 11:

[0213] The terminal gradually slows down after the user confirms. The input is the user's confirmation. The output is a decrease in the bicycle's speed.

[0214] Specific operation:

[0215] The device detects the user's confirmation and begins to safely slow down, gradually reducing the bicycle's speed.

[0216] The bicycle navigation system of the present invention is implemented through the above steps. The specific operations and data inputs and outputs at each step ensure that the system functions accurately and safely.

[0217] (Application example 1)

[0218] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0219] There is a need to reduce the risk of traffic accidents while riding a bicycle and ensure safety. However, current bicycle navigation systems and automated driving assistance systems do not fully fulfill their functions of detecting accident-prone areas in real time or automatically detecting and slowing down obstacles. Another problem is the lack of user awareness in situations where the risk of an accident is high. In these circumstances, it is necessary to develop a reliable accident avoidance and deceleration system.

[0220] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0221] In this invention, the server includes means for acquiring location information, means for acquiring video of the road ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time to detect obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle when approaching an obstacle, means for analyzing the camera video with a generative AI model to detect objects, means for confirming that the handlebars of the bicycle are being held, and means for controlling the bicycle's slow-down mode. This makes it possible to detect traffic accident risks while riding a bicycle in real time and to appropriately slow down the bicycle.

[0222] "Location information" is data that indicates a specific location or coordinates using technology such as GPS.

[0223] A "camera" is a device that receives light and records and displays images or videos.

[0224] An "accident hotspot" is a location in a particular area or section of road where traffic accidents occur frequently.

[0225] A "generative AI model" is a computer program that has been trained using machine learning algorithms and is used to automate specific tasks.

[0226] An "obstacle" is any object that impedes the progress of a bicycle or vehicle, such as a pedestrian, a car, or another bicycle.

[0227] "Audio alert" is a function that transmits a warning message by voice from an audio output device.

[0228] "Automatic deceleration" is a function that automatically reduces the speed of a bicycle or vehicle when the system detects an obstacle.

[0229] A "server" is a computer system that provides services to other devices and systems over a network.

[0230] "Crawl mode" is a driving mode in which a vehicle or bicycle is controlled to a speed significantly slower than normal.

[0231] "Video analysis" is a technology in which a computer processes images and videos captured by a camera to extract and recognize specific information.

[0232] "Confirm that you are holding the handlebars" is a function that uses sensors to detect how the user is holding the handlebars of a bicycle or vehicle.

[0233] This invention relates to a driving assistance system for bicycles or autonomous vehicles. The system acquires location information, captures images of the road ahead with a camera, and analyzes this information in real time to detect accident-prone areas and obstacles, and issues audio alerts and automatically slows down the vehicle. A specific implementation method for this system will be described below.

[0234] 1. System Configuration

[0235] The system includes the following main components:

[0236] server

[0237] Devices (smartphones and in-car cameras)

[0238] User (bicyclist or vehicle driver)

[0239] 2. System Operation

[0240] Location information acquisition and analysis

[0241] User: Before getting on a bike or vehicle, launch the navigation app on your device.

[0242] Device:

[0243] Enable the GPS function to obtain your current location.

[0244] The acquired location information is sent to the server.

[0245] server:

[0246] The received location information is compared with a database to obtain information on nearby accident-prone areas.

[0247] The acquired information is returned to the device.

[0248] Device:

[0249] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[0250] Example: A voice message will be sent saying, "There is an accident-prone area ahead. Please be careful."

[0251] Video analysis and hazard detection

[0252] Device:

[0253] Use an onboard camera to capture images of the vehicle ahead.

[0254] The captured video is analyzed in real time using a generative AI model.

[0255] Detects the presence of obstacles such as pedestrians, bicycles, and vehicles.

[0256] Example: If a pedestrian suddenly approaches ahead, this is detected through video analysis.

[0257] Example prompt: "Analyze the video in front of the vehicle in real time to detect pedestrians and other obstacles."

[0258] Auto-slowdown and alert features

[0259] Device:

[0260] If the system detects a rapidly approaching obstacle based on video analysis, it will instruct the motor control device of the bicycle or vehicle to automatically slow down.

[0261] Select a specific deceleration mode depending on the type of obstacle.

[0262] Slow down mode for pedestrians.

[0263] Use left-side avoidance mode for other bicycles.

[0264] For automobiles, the alert mode is set by flashing lights.

[0265] User:

[0266] You will be instructed to keep both hands on the steering wheel while slowing down.

[0267] Device:

[0268] Only when the motor controller determines that the user is still holding the handlebars will it gradually reduce the speed of the bicycle or vehicle.

[0269] Hardware and Software

[0270] Hardware used:

[0271] GPS Modules

[0272] In-car camera

[0273] Software used:

[0274] OpenCV (camera image analysis)

[0275] Generative AI model (obstacle detection)

[0276] Specific examples

[0277] Situation: Vehicle approaches a busy intersection

[0278] 1. The user launches the app and enables the GPS function.

[0279] 2. The device sends its current location to the server and obtains data on accident-prone areas.

[0280] 3. The server sends the acquired data back to the device, and the device issues a voice notification saying, "You are approaching an accident-prone area. Please be careful."

[0281] 4. The onboard camera captures images of the road ahead and performs real-time analysis using a generative AI model.

[0282] 5. If the device detects an obstacle such as a pedestrian or another vehicle, it will automatically switch to creep mode and safely slow down.

[0283] In this way, this system helps users to ride bicycles and vehicles safely, significantly reducing the risk of accidents.

[0284] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0285] Step 1:

[0286] User: Before getting on a bike or vehicle, launch the navigation app on your device.

[0287] Input: None

[0288] Output: App launch

[0289] What happens: A user taps to launch an app on their smartphone or vehicle's infotainment system.

[0290] Step 2:

[0291] Device: Enable GPS to obtain your current location.

[0292] Input: None

[0293] Output: Current location information (latitude and longitude)

[0294] Specific operation: The device obtains current location data using the built-in GPS module.

[0295] Step 3:

[0296] Device: Sends the acquired location information to the server.

[0297] Input: Current location information (output of step 2)

[0298] Output: Sending completion message or data to the server

[0299] What it does: Your device sends its current location data to a server via an internet connection.

[0300] Step 4:

[0301] Server: The received location information is compared with a database to obtain information on nearby accident-prone areas.

[0302] Input: Current location

[0303] Output: Accident hotspot information

[0304] Specific operation: The server accesses the database and queries for data on accident-prone areas that match the current location information.

[0305] Step 5:

[0306] Server: Returns the acquired information on accident-prone areas to the terminal.

[0307] Input: Accident hotspot information

[0308] Output: Accident-prone areas information (response data sent to the device)

[0309] Specific operations: The server constructs the search results and sends the data back to the device.

[0310] Step 6:

[0311] Terminal: Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[0312] Input: Accident hotspot information

[0313] Output: Audio alert

[0314] Specific operation: The device uses the built-in speaker to notify the user by voice, "There is an accident-prone area ahead. Please be careful."

[0315] Step 7:

[0316] Terminal: Uses the onboard camera to capture images of the area ahead.

[0317] Input: None

[0318] Output: Video data (live feed)

[0319] Specific operation: The camera captures images of the area in front of the vehicle and acquires real-time video data.

[0320] Step 8:

[0321] Terminal: The captured video is analyzed in real time using a generative AI model.

[0322] Input: Video data

[0323] Output: Obstacle detection results (pedestrians, bicycles, vehicles, etc.)

[0324] Specific operation: The acquired video data is input into a generative AI model, which then detects obstacles such as pedestrians and vehicles.

[0325] Step 9:

[0326] Terminal: Based on video analysis, if a rapidly approaching obstacle is detected, the device instructs the motor control device of the bicycle or vehicle to automatically slow down.

[0327] Input: Obstacle detection result

[0328] Output: Deceleration instruction

[0329] Specific operation: Sends a command to the motor control device to reduce speed depending on the danger level of the obstacle.

[0330] Step 10:

[0331] Device: Check that the user is holding the wheel, and only then will creep mode be activated.

[0332] Input: User's hand position data (steering wheel sensor)

[0333] Output: Crawl mode execution command

[0334] Specific operation: A sensor installed on the handle detects the position of the user's hands and, based on that information, instructs the motor control device to enter slow-motion mode.

[0335] Sample prompt: "Analyze the video in front of the vehicle in real time to detect pedestrians and other obstacles."

[0336] Through these steps, a system that reduces the risk of accidents while riding a bicycle or vehicle and supports safe driving will be efficiently realized.

[0337] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0338] ---

[0339] This invention combines an emotion recognition engine with a bicycle navigation system to improve the safety and comfort of cyclists. In addition to detecting areas prone to traffic accidents and automatically slowing down when encountering obstacles, this system can recognize the user's emotions in real time and take appropriate action based on that data.

[0340] Explanation of program processing

[0341] 1. System Configuration

[0342] The system mainly consists of the following components:

[0343] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[0344] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[0345] User: An individual who uses a bicycle.

[0346] Emotion engine: A software module for recognizing user emotions and analyzing data.

[0347] 2. Specific Program Processing

[0348] Location information acquisition and analysis

[0349] User:

[0350] Before getting on your bike, launch the navigation app on your device.

[0351] Device:

[0352] When the system starts up, it enables the GPS function and begins obtaining the current location.

[0353] The acquired current location information is sent to the server.

[0354] server:

[0355] Based on the received location information, the system compares it with a database to obtain information on nearby accident-prone areas.

[0356] The acquired information is returned to the device.

[0357] Device:

[0358] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[0359] Video analysis and hazard detection

[0360] Device:

[0361] Uses the smartphone camera to capture images of the area ahead.

[0362] The acquired video is temporarily stored in internal memory, and analysis begins using the generating AI.

[0363] Device:

[0364] Generative AI analyzes the image and detects obstacles such as pedestrians, bicycles, and cars.

[0365] Calculates the distance and relative speed of detected obstacles.

[0366] Emotion Recognition and Its Applications

[0367] Device:

[0368] An emotion engine is activated that recognizes emotions from the user's facial expressions, voice, etc.

[0369] An emotion engine assesses the user's stress level, anxiety, and fear.

[0370] Device:

[0371] If the user exhibits high stress levels, the bike speed is automatically adjusted.

[0372] If the user is feeling anxious or scared, the warning alert will be intensified.

[0373] Examples:

[0374] Situation 1: If the user is feeling stressed, the system will detect this through its emotion engine and automatically slow down the speed.

[0375] A voice message will say, "Relax, we'll slow down a bit."

[0376] Situation 2: If the user is feeling fear, the system will issue a detailed warning alert.

[0377] A voice message will be displayed saying, "There is an imminent danger ahead, please slow down immediately."

[0378] Auto-slowdown and alert features

[0379] Device:

[0380] If a rapidly approaching obstacle is detected based on video analysis, the system instructs the bicycle's motor control device to automatically slow down.

[0381] Select a specific deceleration mode depending on the type of obstacle.

[0382] Slow down mode for pedestrians.

[0383] Left-hand driving mode for other bicycles.

[0384] For automobiles, the alert mode is set by flashing lights.

[0385] User:

[0386] You will be instructed to make sure you keep both hands on the wheel while slowing down.

[0387] Device:

[0388] Only with user confirmation will the motor controller gradually reduce the bike's speed.

[0389] Specific examples

[0390] Situation: Approaching an intersection while riding a bicycle

[0391] User:

[0392] Get on your bike and launch the app.

[0393] Device:

[0394] The current location is obtained using GPS and sent to the server.

[0395] server:

[0396] Information on accident-prone areas is acquired and sent to the terminal.

[0397] Device:

[0398] A voice message will be displayed saying, "Be careful before approaching the intersection."

[0399] A camera captures images of the area ahead and analyzes them using generative AI.

[0400] Video analysis detects sudden approach of pedestrians.

[0401] Device:

[0402] A creep mode command is sent to the motor controller and the user is instructed to keep both hands on the steering wheel.

[0403] Once it is confirmed that you are holding the steering wheel, a creep mode is activated and the vehicle gradually slows down.

[0404] Device:

[0405] The emotion engine analyzes the user's emotions and if it detects high stress levels, it will instruct them to slow down further.

[0406] A voice will inform you, "Relax, we'll slow down further."

[0407] In this way, the system of the present invention increases the safety and comfort of the user and significantly reduces the risk of accidents.

[0408] The processing flow will be explained below.

[0409] ---

[0410] Step 1:

[0411] User: Turns on the smartphone and launches the bicycle navigation app.

[0412] Device: When the app is launched, enable GPS and camera functions.

[0413] Step 2:

[0414] Device: Uses GPS to obtain current location information.

[0415] Device: Sends the acquired current location information to the server.

[0416] Step 3:

[0417] Server: Based on the received location information, the server compares it with a database of accident-prone areas to obtain information on nearby accident-prone locations.

[0418] Server: Returns the accident-prone location information obtained as a result of the matching to the terminal.

[0419] Step 4:

[0420] Terminal: Analyzes the accident-prone location information returned from the server and determines whether the current location is close to an accident-prone location.

[0421] Terminal: When approaching a nearby accident hotspot, the device will issue a voice alert to the user.

[0422] Step 5:

[0423] Device: The smartphone camera captures real-time images of the road ahead.

[0424] Device: The acquired video is temporarily stored in the internal memory, and analysis begins using the generation AI.

[0425] Step 6:

[0426] Device: Analyzes video using generative AI and detects obstacles such as pedestrians, bicycles, and cars.

[0427] Terminal: Calculates the distance and relative speed of detected obstacles.

[0428] Step 7:

[0429] Terminal: Based on the video analysis results, evaluate whether there is an approaching obstacle.

[0430] Terminal: Prepares to take specific action (slow mode, drive left mode, flash lights) if assessed as dangerous.

[0431] Step 8:

[0432] Device: Select slow-walk mode for pedestrians, drive left mode for cyclists, and flashing light alert mode for cars.

[0433] Terminal: Tells the motor controller to slow down or flash lights.

[0434] Step 9:

[0435] Device: Runs the emotion engine, analyzes the user's facial expressions and voice, and recognizes their emotional state in real time.

[0436] Device: When the user's stress level, anxiety, or fear is detected, the device analyzes the information and determines the necessary response.

[0437] Step 10:

[0438] Device: If it detects high stress levels in the user, it will issue instructions to automatically slow down the bike.

[0439] Device: A voice prompt tells the user to "Relax, we'll slow down a bit."

[0440] Step 11:

[0441] Device: If the emotion engine detects that the user is anxious or scared, it will increase the intensity of the warning alert.

[0442] Device: A voice message will be displayed saying "There is an imminent danger ahead, please slow down immediately."

[0443] Step 12:

[0444] Terminal: The motor control unit follows the instructions and gradually reduces the speed of the bicycle.

[0445] User: Keep both hands on the steering wheel while slowing down and ensure safe driving.

[0446] In this way, the system monitors the user's emotional state in real time and takes appropriate action as needed, supporting safe cycling.

[0447] Example 2

[0448] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0449] While conventional bicycle navigation systems provide basic risk avoidance based on location information and obstacle detection, they are unable to take the user's emotional state into account and therefore are unable to fully enhance the user's safety and comfort. In particular, when the user is experiencing high levels of stress, impatience, or fear, it is difficult for the system to respond appropriately, potentially increasing the risk of an accident.

[0450] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0451] In this invention, the server includes means for acquiring location information, means for acquiring video of the area ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time to detect obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle when approaching an obstacle, and means for recognizing the user's emotions and adjusting the bicycle speed and warning mode based on the user's emotional state. This allows for appropriate responses according to the user's emotional state, improving the user's safety and comfort.

[0452] A "means for obtaining location information" is a device or system that has the ability to determine current location using GPS or other location information services.

[0453] "Means for obtaining forward images using a camera" refers to a device or system that has the function of capturing and recording images of the area in front of the bicycle using a smartphone or dedicated camera.

[0454] The "means for detecting accident-prone areas" is a system that has the function of identifying areas where traffic accidents are likely to occur by comparing location information with a pre-set database.

[0455] "Means for analyzing captured video in real time and detecting obstacles" refers to a system that uses video processing technology and generative AI models to recognize obstacles such as pedestrians, bicycles, and cars within the video.

[0456] The "means for issuing audio alerts" refers to a system that has the function of issuing audio messages to alert the user based on detected accident hotspots and obstacles.

[0457] "Means for automatically slowing down the bicycle speed" refers to a system that operates the motor control device to slow down the bicycle speed depending on the type and distance of an obstacle detected.

[0458] "Means for recognizing the user's emotions and adjusting the bicycle speed and warning mode based on the emotional state" refers to a system that uses facial expression and voice recognition technology to evaluate the user's emotional state and automatically adjusts speed control and warning methods according to that state.

[0459] This invention improves the safety and comfort of cyclists by combining an emotion recognition engine with a bicycle navigation system that acquires location information, captures camera footage, detects accident hotspots, detects obstacles in real time, issues audio alerts, automatically slows down, and adjusts speed and warning modes based on the user's emotional state.

[0460] The system consists of the following main components:

[0461] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[0462] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[0463] User: An individual who uses a bicycle.

[0464] Emotion engine: A software module for recognizing user emotions and analyzing data.

[0465] Specific operation of the system

[0466] User: Before riding a bicycle, launch the navigation app on the device. Tap the navigation app icon and press the "Start" button to start the GPS.

[0467] Device: When the app is launched, it will enable the GPS function and start acquiring the current location. It will then send that location information to the server.

[0468] Server: Based on the received location information, the server compares it with a database to obtain information on nearby accident-prone areas, and sends the obtained information back to the device.

[0469] Terminal: Based on the returned information, if the user is approaching a nearby accident-prone area, a voice message will be sent saying, "You are approaching an accident-prone area, please be careful."

[0470] Device: Uses the smartphone camera to capture video of the area ahead and temporarily stores it in internal memory. The video is analyzed using a generative AI model to detect obstacles such as pedestrians, bicycles, and cars. The device calculates the distance and relative speed of detected obstacles and manages the obstacle data.

[0471] Device: Operates an emotion engine that recognizes emotions from the user's facial expressions and voice. The emotion engine evaluates the user's stress level, anxiety, and fear. If the user indicates a high stress level, the device automatically adjusts the speed of the bicycle and notifies the user through a voice message saying, "Please relax. We will slow down a bit."

[0472] Terminal: If the terminal detects a rapidly approaching obstacle based on video analysis, it instructs the bicycle's motor control unit to automatically decelerate. It selects a specific deceleration mode depending on the type of obstacle and instructs the user to hold the handlebars with both hands. After confirming that the user follows the instruction and holds the handlebars with both hands, the motor control unit gradually reduces the bicycle's speed.

[0473] Specific examples

[0474] Situation: Approaching an intersection while riding a bicycle

[0475] User: Get on the bike and launch the app.

[0476] Device: Obtains current location using GPS and sends it to the server.

[0477] Server: Obtains information on accident-prone areas and sends it to the device.

[0478] Device: Announces "Be careful before approaching the intersection" with a voice message. The camera captures images of the area ahead and analyzes them with generative AI. Video analysis detects sudden approach of pedestrians.

[0479] Terminal: Sends a creep mode command to the motor control device and instructs the user to hold the steering wheel with both hands. Once it is confirmed that the user is holding the steering wheel, the creep mode is activated and the vehicle gradually decelerates.

[0480] On the device: The emotion engine analyzes the user's emotions and, if it detects high stress levels, instructs them to slow down further. A voice message will say, "Relax, we'll slow down even more."

[0481] Examples of prompt statements

[0482] An example of an input prompt sentence is, "How can a bicycle navigation system use an emotion recognition engine to automatically adjust speed based on the user's emotional state?"

[0483] This system can increase user safety and comfort and significantly reduce the risk of accidents.

[0484] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0485] Step 1:

[0486] User: Before riding a bicycle, launch the navigation app. Tap the navigation app icon and press the "Start" button to start the GPS.

[0487] Input: Launching the app and pressing the "Start" button.

[0488] Specific actions: The navigation app is launched, the GPS function is enabled, and the current location is acquired.

[0489] Output: Current location information.

[0490] Step 2:

[0491] Device: Sends current location information to the server.

[0492] Input: The current location information obtained.

[0493] Specific operation: The location information obtained from the GPS module is sent to the server via mobile data communication.

[0494] Output: Current location information sent to the server.

[0495] Step 3:

[0496] Server: Based on the received location information, the server compares it with a database to obtain information on nearby accident-prone areas.

[0497] Input: Current location information sent from the device.

[0498] What it does: Executes a database query to find information about accident hotspots near the current location.

[0499] Output: Accident hotspot information.

[0500] Step 4:

[0501] Server: Returns the acquired information on accident-prone areas to the terminal.

[0502] Input: Accident black spot information from a database.

[0503] Specific operation: Accident-prone area information is compiled as a text message and sent to the device via mobile data communication.

[0504] Output: Accident hotspot information sent to the device.

[0505] Step 5:

[0506] Terminal: Based on the information received by the terminal, it issues a voice warning to the user, for example, "You are approaching an accident-prone area, so please be careful."

[0507] Input: Accident hotspot information received from the server.

[0508] Specific operation: Converts text information into speech and transmits it to the user through a speaker.

[0509] Output: Audio alert.

[0510] Step 6:

[0511] Device: The device uses a smartphone camera to capture video of the area ahead and temporarily stores it in its internal memory. The device then uses a generative AI model to analyze the video and detect obstacles such as pedestrians, bicycles, and cars.

[0512] Input: Front view captured by a smartphone camera.

[0513] Specific operations: Video capture, saving to internal memory, real-time video analysis and obstacle identification using generative AI.

[0514] Output: Information about detected obstacles (type, distance, relative speed).

[0515] Step 7:

[0516] Terminal: Based on the detected obstacle information, it issues an audio warning to the user.

[0517] Input: Obstacle information analyzed by the generation AI.

[0518] Specific behavior: Generates an appropriate warning message depending on the type of obstacle and notifies the user via audio.

[0519] Output: Audio warning message.

[0520] Step 8:

[0521] Device: Operates an emotion engine that recognizes emotions from the user's facial expressions and voice. The emotion engine evaluates the user's stress level, anxiety, and fear.

[0522] Input: User's facial expression and voice data.

[0523] Specific operation: Analyze and evaluate the user's emotional state using facial expression and voice recognition software.

[0524] Output: User's emotional state data (stress level, anxiety, fear, etc.).

[0525] Step 9:

[0526] Device: If the user exhibits high stress levels, the device automatically adjusts the speed of the bike and provides a voice prompt saying, "Relax, we're going to slow down a bit."

[0527] Input: User emotional state data.

[0528] Specific operation: Instructs the motor control device to adjust the speed, and at the same time generates a voice message and notifies the user through the speaker.

[0529] Output: Adjusted bike speed, voice notification.

[0530] Step 10:

[0531] Terminal: If a rapidly approaching obstacle is detected based on video analysis, the device instructs the bicycle's motor control device to automatically slow down and instructs the user to keep both hands on the handlebars.

[0532] Input: Approaching obstacle based on generative AI analysis.

[0533] Specific actions: Motor control instructions and a voice command to "Hold the steering wheel with both hands."

[0534] Output: Reduced bike speed, voice instructions.

[0535] Step 11:

[0536] User: Follow the instructions and hold the steering wheel with both hands.

[0537] Input: Voice commands.

[0538] Specific action: The user holds the steering wheel with both hands.

[0539] Output: Both hands on the handlebars.

[0540] Step 12:

[0541] Terminal: After confirming that the rider is still holding the handlebars, the motor control unit gradually reduces the speed of the bicycle.

[0542] Input: Confirmation that the user has both hands on the steering wheel.

[0543] Specific operation: The speed is gradually reduced based on instructions from the motor control device.

[0544] Power: Reduced bike speed.

[0545] (Application example 2)

[0546] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0547] In bicycle navigation systems, detecting traffic accident hotspots and automatically slowing down when encountering obstacles are important accident prevention measures, but many systems do not take the user's emotional state into account. As a result, they may not respond appropriately when the user feels strong stress or fear, which could increase the risk of an accident. Furthermore, current bicycle navigation systems do not incorporate real-time emotion recognition and driving style adjustment, making it difficult to ensure sufficient safety and comfort.

[0548] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring location information, means for acquiring video of the road ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time and detecting obstacles, means for recognizing the user's emotions using an emotion recognition engine and adjusting the driving style according to the stress level, means for issuing an audio alert when an accident-prone area or an obstacle is detected, and means for automatically slowing down when approaching an obstacle. This enables appropriate action to be taken in real time when the user is feeling high stress or fear, thereby reducing the risk of an accident and improving the safety and comfort of the user.

[0549] "Means for obtaining location information" refers to devices or software that obtain location information using GPS functions or other location measurement technologies to identify a user's current location.

[0550] "Means for obtaining forward-looking images using a camera" refers to devices or software that use a camera installed in front of a bicycle or autonomous vehicle to obtain video data in real time.

[0551] "Means for detecting accident-prone areas" refers to devices or software that use acquired location information and external databases to detect areas where traffic accidents frequently occur.

[0552] "Means for analyzing video in real time and detecting obstacles" refers to devices or software that use generative AI models or deep learning technology to analyze acquired video data in real time and detect pedestrians, vehicles, and other obstacles.

[0553] "Means for recognizing a user's emotions using an emotion recognition engine" refers to devices or software that use deep learning models or algorithms to analyze data on a user's facial expressions and voice to recognize their emotional state.

[0554] "Means for adjusting driving style according to stress level" refers to devices or software for adjusting the speed or changing the driving mode of a bicycle or autonomous vehicle based on the stress level recognized by the emotion recognition engine.

[0555] The "means for issuing an audio alert" refers to a device or software that issues an audio alert to alert the user based on detected accident-prone areas or obstacle information.

[0556] "Means for automatically slowing down around obstacles" means motor controls and software for automatically reducing vehicle speed based on the location and distance of a detected obstacle.

[0557] MODE FOR CARRYING OUT THE INVENTION

[0558] This invention relates to a navigation system for an autonomous vehicle that recognizes a user's emotional state in real time to optimize the driving style and improve safety and comfort. The system includes a means for acquiring location information, a means for capturing images of the road ahead using a camera, a means for detecting accident-prone areas, a means for analyzing the images in real time to detect obstacles, a means for recognizing the user's emotions using an emotion recognition engine, a means for adjusting the driving style according to the user's stress level, a means for issuing an audio alert, and a means for automatically slowing down the vehicle.

[0559] Hardware and Software Configuration

[0560] Hardware:

[0561] GPS module: Obtains location information.

[0562] Camera: Captures real-time images of the area ahead.

[0563] Motor control unit: Controls the speed of an autonomous vehicle.

[0564] Microphone: Captures the user's voice.

[0565] software:

[0566] Emotion recognition engine: Analyzes emotions from the user's facial expressions and voice.

[0567] Generative AI model: Analyzes images ahead and detects obstacles.

[0568] Traffic information API: Links with external databases to obtain information on areas prone to accidents.

[0569] Program processing

[0570] server

[0571] The server receives the location information and compares it with a database of accident-prone areas. The results are sent back to the device, which then issues an alert about any dangerous areas the user is approaching. The server also analyzes the captured video data with a generative AI model to detect obstacles in real time.

[0572] Terminal

[0573] The device captures images of the road ahead using a camera and temporarily stores them. An emotion recognition engine analyzes the user's facial expressions and voice, monitoring their emotions in real time. Based on their emotional state and the captured images, the device adjusts its driving style (speed and warning level) appropriately.

[0574] User

[0575] The user launches the application and gets into the self-driving vehicle. The system provides voice alerts and adjusts driving according to the user's emotional state, and the user adjusts their driving accordingly.

[0576] Specific examples

[0577] 1. Situation: A user is driving an autonomous vehicle and approaches a congested area.

[0578] 2. Terminal:

[0579] The camera captures images of the road ahead, and a generative AI model analyzes the traffic congestion situation.

[0580] The user's voice is picked up by a microphone and stress levels are measured using an emotion recognition engine.

[0581] 3. Server:

[0582] Based on the acquired location information, traffic congestion information is obtained via a traffic information API.

[0583] 4. Results:

[0584] If the user indicates high stress levels, the vehicle's speed will be automatically reduced and a voice notification will be provided saying, "Relax. We will reduce your speed."

[0585] Prompt Sentence Examples

[0586] "Create an application that uses an emotion recognition model to analyze passenger emotions in real time and adjust the vehicle's driving style based on those emotions. Additionally, use an external traffic information API to obtain traffic congestion information and include guidelines to reduce passenger stress."

[0587] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0588] Step 1:

[0589] The device acquires video of the road ahead using the autonomous vehicle's camera, uses the real-time video data acquired from the camera as input, temporarily stores it in its internal memory, and provides video data for the next processing step as output.

[0590] Step 2:

[0591] The device uses an emotion recognition engine to recognize the user's emotions in real time. It uses audio data acquired from a microphone and facial expression data acquired from a camera as input. The emotion recognition engine analyzes this data to identify the user's emotional state. The output is generated as the user's emotional state data (e.g., stress level, high, medium, low).

[0592] Step 3:

[0593] The device sends the location information it has acquired to the server. It uses the location data from the GPS module as input. It sends the location data to the server and requests information on accident-prone areas. It receives the information on accident-prone areas returned from the server as output.

[0594] Step 4:

[0595] The server compares the location data it receives with a database of accident-prone areas. The location data and the information on accident-prone areas in the database are used as input. The database is compared to extract information on the relevant accident-prone areas and sent to the terminal. The information on the relevant accident-prone areas is provided as output.

[0596] Step 5:

[0597] An audio alert is issued to the user based on the information on accident-prone areas received by the device. The information on accident-prone areas and the previously acquired video data are used as input. An audio alert generation engine is used to warn the user. The audio data that issues the audio alert is generated as output and executed.

[0598] Step 6:

[0599] The video data acquired by the device is analyzed in real time using a generative AI model to detect obstacles. Video data of the front is used as input. The generative AI model is used to recognize objects on the screen and identify the type of obstacle (pedestrian, car, etc.), as well as their relative speed and distance. Information data on the detected obstacle is generated as output and used for the next deceleration process.

[0600] Step 7:

[0601] The device comprehensively assesses the emotional state, obstacle data, and information on accident-prone areas, and sends instructions to the motor controller to adjust the driving style. It also uses the emotional state data, obstacle data, and information on accident-prone areas as inputs. This includes issuing instructions to reduce speed when stress levels are high or obstacles are nearby. It generates deceleration command data that is sent to the motor controller as output.

[0602] Step 8:

[0603] The motor control device receives the deceleration command from the terminal and adjusts the speed of the autonomous vehicle. It uses the deceleration command data as input, adjusts the motor output of the autonomous vehicle according to the deceleration command, and reduces the speed appropriately. It feeds back the actual speed data of the decelerated vehicle as output, and maintains a safe driving style.

[0604] In this way, the steps work in cooperation with each other, thereby making it possible to improve the safety and comfort of the user.

[0605] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0606] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0607] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0608] [Second embodiment]

[0609] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0610] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0611] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0612] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0613] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0614] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0615] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0616] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0617] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0618] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0619] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0620] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0621] ---

[0622] The present invention relates to a bicycle navigation system, and provides a support system for cyclists to ride safely. This system has functions to detect traffic accident hot spots and automatically slow down when encountering obstacles, and can provide specific measures for users to avoid accidents.

[0623] Explanation of program processing

[0624] 1. System Configuration

[0625] The system mainly consists of the following components:

[0626] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[0627] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[0628] User: An individual who uses a bicycle.

[0629] 2. Specific Program Processing

[0630] Location information acquisition and analysis

[0631] User:

[0632] Before getting on your bike, launch the navigation app on your device.

[0633] Device:

[0634] When the system starts up, it enables the GPS function and begins obtaining the current location.

[0635] The acquired current location information is sent to the server.

[0636] server:

[0637] Based on the received location information, the system compares it with a database to obtain information on nearby accident-prone areas.

[0638] The acquired information is returned to the device.

[0639] Device:

[0640] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[0641] Examples:

[0642] - A voice notification will say, "There is an accident-prone area ahead. Please be careful."

[0643] Video analysis and hazard detection

[0644] Device:

[0645] Uses the smartphone camera to capture images of the area ahead.

[0646] The acquired video is analyzed in real time using generative AI.

[0647] Detects the presence of obstacles such as pedestrians, bicycles, and cars.

[0648] Examples:

[0649] - If a pedestrian suddenly approaches ahead, it will be detected through video analysis.

[0650] Auto-slowdown and alert features

[0651] Device:

[0652] If a rapidly approaching obstacle is detected based on video analysis, the system instructs the bicycle's motor control device to automatically slow down.

[0653] Select a specific deceleration mode depending on the type of obstacle.

[0654] Slow down mode for pedestrians.

[0655] Use left-side avoidance mode for other bicycles.

[0656] For automobiles, the alert mode is set by flashing lights.

[0657] User:

[0658] You will be instructed to make sure you keep both hands on the wheel while slowing down.

[0659] Device:

[0660] Only with user confirmation will the motor controller gradually reduce the bike's speed.

[0661] Specific examples

[0662] Situation: Approaching an intersection while riding a bicycle

[0663] User:

[0664] Get on your bike and launch the app.

[0665] Device:

[0666] The current location is obtained using GPS and sent to the server.

[0667] server:

[0668] Information on accident-prone areas is acquired and sent to the terminal.

[0669] Device:

[0670] A voice message will be displayed saying, "Be careful before approaching the intersection."

[0671] A camera captures images of the area ahead and analyzes them using generative AI.

[0672] Video analysis detects sudden approach of pedestrians.

[0673] Device:

[0674] A creep mode command is sent to the motor controller and the user is instructed to keep both hands on the steering wheel.

[0675] Once it is confirmed that you are holding the steering wheel, a creep mode is activated and the vehicle gradually slows down.

[0676] In this way, the system of the present invention helps users to ride bicycles safely and significantly reduces the risk of accidents.

[0677] The processing flow will be explained below.

[0678] ---

[0679] Step 1:

[0680] User: Turns on the smartphone and launches the bicycle navigation app.

[0681] Device: When the app is launched, enable GPS and camera functions.

[0682] Step 2:

[0683] Device: Uses GPS to obtain current location information.

[0684] Device: Sends the acquired current location information to the server.

[0685] Step 3:

[0686] Server: Based on the received location information, the server compares it with a database of accident-prone areas to obtain information on nearby accident-prone locations.

[0687] Server: Returns the accident-prone location information obtained as a result of the matching to the terminal.

[0688] Step 4:

[0689] Terminal: Analyzes the accident-prone location information returned from the server and determines whether the current location is close to an accident-prone location.

[0690] Terminal: When approaching a nearby accident hotspot, the device will issue a voice alert to the user.

[0691] Step 5:

[0692] Device: The smartphone camera captures real-time images of the road ahead.

[0693] Device: The acquired video is temporarily stored in the internal memory, and analysis begins using the generation AI.

[0694] Step 6:

[0695] Device: Analyzes video using generative AI and detects obstacles such as pedestrians, bicycles, and cars.

[0696] Terminal: Calculates the distance and relative speed of detected obstacles.

[0697] Step 7:

[0698] Terminal: Based on the video analysis results, evaluate whether there is an approaching obstacle.

[0699] Terminal: Prepares to take specific action (slow mode, drive left mode, flash lights) if assessed as dangerous.

[0700] Step 8:

[0701] Device: Select slow-walk mode for pedestrians, drive left mode for cyclists, and flashing light alert mode for cars.

[0702] Terminal: Tells the motor controller to slow down or flash lights.

[0703] Step 9:

[0704] Device: A sensor checks whether the user has both hands on the steering wheel.

[0705] Terminal: If the driver is confirmed to be holding the steering wheel, the motor control unit will automatically begin a gradual deceleration process.

[0706] Step 10:

[0707] Terminal: Ensures that the bicycle's speed has been reduced to a safe level for obstacles.

[0708] User: Keep both hands on the steering wheel while slowing down and ensure safe driving.

[0709] This allows the system to automatically take appropriate measures before the user gets into a dangerous situation, supporting safe cycling.

[0710] Example 1

[0711] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0712] Systems designed to help cyclists ride safely face challenges due to their inadequate ability to detect frequent traffic accidents and obstacles. In particular, they lack the ability to obtain real-time location information, provide information about accident-prone areas and obstacles, and automatically adjust speed. They also need a way to avoid unnecessary slowdowns by obtaining user confirmation.

[0713] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0714] In this invention, the server includes means for acquiring location information, means for acquiring video of the area ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time and detecting obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle speed toward an obstacle, and means for executing the speed reduction when the user agrees to the speed reduction. This makes it possible to detect accident-prone areas and obstacles and provide appropriate alerts and deceleration to ensure safe cycling for cyclists.

[0715] "Means of obtaining location information" refers to a function that measures the current location using a smartphone, GPS device, etc. and obtains the data.

[0716] "Means for obtaining images of the front using a camera" refers to a function that collects images of the front of the bicycle in real time using a smartphone, camera device, etc.

[0717] "Means for detecting accident-prone areas" is a function that identifies areas where traffic accidents have occurred frequently in the past based on location information and extracts the necessary data.

[0718] "Means for analyzing and detecting obstacles in real time" refers to a function that instantly analyzes the images captured by the camera and identifies obstacles such as pedestrians, bicycles, and automobiles.

[0719] "Means for issuing audio alerts" is a function that issues an audio warning to the user when an accident-prone area or obstacle is detected.

[0720] "Means for automatically reducing the speed of the bicycle" is a function that automatically controls the speed of the bicycle in response to detected obstacles, ensuring safety.

[0721] The "means for slowing down when the user agrees to slow down" is a function that notifies the user of the need to slow down and reduces the speed only if the user complies with the instruction.

[0722] "Generative AI" is an artificial intelligence that performs real-time image analysis based on video data acquired by a camera to determine the type and location of obstacles.

[0723] This invention provides a bicycle navigation system to support cyclists in riding safely. In particular, it has the function of detecting traffic accident hotspots and automatically slowing down the vehicle when encountering obstacles, aiming to significantly reduce the risk of accidents. This system consists of three main elements: a server, a terminal, and a user.

[0724] System Components

[0725] server:

[0726] The server maintains a database of areas prone to traffic accidents and provides location information to the terminal. The server receives the location information and returns the information on areas prone to traffic accidents to the terminal.

[0727] Device:

[0728] The terminal is a mobile device such as a smartphone, and uses GPS and camera functions. When the system is started, the terminal begins acquiring its current location and acquires location information and images of the area ahead in real time. The acquired images are then analyzed using a generative AI model to detect obstacles and issue instructions for automatic deceleration.

[0729] User:

[0730] The user is an individual who rides a bicycle. Before riding, the user launches the navigation app and follows the alerts and instructions from the device.

[0731] Hardware and software used

[0732] GPS Module:

[0733] It is built into the device and acquires the current location.

[0734] camera:

[0735] It is built into the terminal and captures images of what is in front of it.

[0736] Generative AI models:

[0737] It runs on the device, analyzes images captured by the camera in real time, and detects obstacles.

[0738] Motor Controller:

[0739] A device for automatically adjusting the speed of a bicycle.

[0740] Specific Examples of the Invention

[0741] Situation: Approaching an intersection while riding a bicycle

[0742] User:

[0743] The user gets on the bicycle and starts the navigation app on the device. The user drives the bicycle following the instructions of the app.

[0744] Device:

[0745] When the navigation app is launched, it uses the GPS module to obtain the current location and sends that information to the server. Based on the information on accident-prone areas returned from the server, it notifies the user by voice, "There is an accident-prone area ahead. Please be careful." It also uses a camera to obtain images of the area ahead and analyzes them in real time using a generative AI model. Based on the analysis results, if the device detects a pedestrian approaching quickly, it instructs the motor control device to enter slow-down mode.

[0746] User:

[0747] When instructed to slow down, the user ensures they keep both hands on the steering wheel, then follows further instructions from the device to drive safely.

[0748] In this way, the system of the present invention combines real-time location information with video analytics to provide specific alerts and slow-down instructions to help cyclists avoid accidents, thereby increasing cyclist safety and significantly reducing the risk of accidents.

[0749] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0750] Step 1:

[0751] The user launches a navigation app. The input is the user tapping the navigation app on their smartphone. The output is the navigation app launching.

[0752] Specific operation:

[0753] The user taps the navigation app icon on the smartphone home screen.

[0754] Step 2:

[0755] The device uses the GPS module to obtain its current location. The input is satellite data received by the device's GPS module. The output is the current location data.

[0756] Specific operation:

[0757] The smartphone obtains the current location information from satellites and sends that information to the location management module within the app.

[0758] Step 3:

[0759] The current location information acquired by the device is sent to the server. The input is the current location data. The output is the location information data sent to the server.

[0760] Specific operation:

[0761] The GPS information is sent to the server using the terminal's network module.

[0762] Step 4:

[0763] Based on the location information received by the server, information on accident-prone areas is retrieved from the database. The input is the current location data sent from the device. The output is information on accident-prone areas.

[0764] Specific operation:

[0765] The server queries the database and retrieves the results.

[0766] Step 5:

[0767] The server returns the information about accident-prone areas it has acquired to the terminal. The input is the information about accident-prone areas acquired by the server. The output is the information about accident-prone areas that is sent to the terminal.

[0768] Specific operation:

[0769] The server obtains information about areas prone to accidents from the database and transmits it to the terminal via the network.

[0770] Step 6:

[0771] The terminal issues a voice alert to the user based on the returned information on accident-prone areas. The input is the accident-prone area information sent from the server. The output is the issuance of a voice alert.

[0772] Specific operation:

[0773] The device will notify you with a voice message saying, "There is an accident-prone area ahead. Please be careful."

[0774] Step 7:

[0775] The device uses the smartphone camera to capture images of the front. The input is the camera image data. The output is the captured front image data.

[0776] Specific operation:

[0777] The camera continuously captures images of the area ahead and stores the data in the device's memory.

[0778] Step 8:

[0779] The image captured by the device is input into a generative AI model and analyzed in real time. The input is the image data from the camera. The output is the analyzed obstacle data.

[0780] Specific operation:

[0781] The device inputs the video data into a generative AI model, which then analyzes the data to identify the type and location of obstacles.

[0782] Step 9:

[0783] The device detects obstacles based on the analysis results and instructs the bicycle's motor control device to slow down. The input is the analysis result from the generative AI model. The output is a command to the motor control device to slow down.

[0784] Specific operation:

[0785] If a pedestrian is detected, the device will issue an alert saying "Pedestrian detected, bicycle speed set to slow mode" and send a command to the motor control device to slow down.

[0786] Step 10:

[0787] Verify that the user keeps both hands on the steering wheel when instructed to slow down. The input is an audio alert. The output is the user gripping the steering wheel.

[0788] Specific operation:

[0789] The user follows the instruction "Please hold the steering wheel with both hands" and performs an operation to confirm that he or she is holding the steering wheel.

[0790] Step 11:

[0791] The terminal gradually slows down after the user confirms. The input is the user's confirmation. The output is a decrease in the bicycle's speed.

[0792] Specific operation:

[0793] The device detects the user's confirmation and begins to safely slow down, gradually reducing the bicycle's speed.

[0794] The bicycle navigation system of the present invention is implemented through the above steps. The specific operations and data inputs and outputs at each step ensure that the system functions accurately and safely.

[0795] (Application example 1)

[0796] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0797] There is a need to reduce the risk of traffic accidents while riding a bicycle and ensure safety. However, current bicycle navigation systems and automated driving assistance systems do not fully fulfill their functions of detecting accident-prone areas in real time or automatically detecting and slowing down obstacles. Another problem is the lack of user awareness in situations where the risk of an accident is high. In these circumstances, it is necessary to develop a reliable accident avoidance and deceleration system.

[0798] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0799] In this invention, the server includes means for acquiring location information, means for acquiring video of the road ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time to detect obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle when approaching an obstacle, means for analyzing the camera video with a generative AI model to detect objects, means for confirming that the handlebars of the bicycle are being held, and means for controlling the bicycle's slow-down mode. This makes it possible to detect traffic accident risks while riding a bicycle in real time and to appropriately slow down the bicycle.

[0800] "Location information" is data that indicates a specific location or coordinates using technology such as GPS.

[0801] A "camera" is a device that receives light and records and displays images or videos.

[0802] An "accident hotspot" is a location in a particular area or section of road where traffic accidents occur frequently.

[0803] A "generative AI model" is a computer program that has been trained using machine learning algorithms and is used to automate specific tasks.

[0804] An "obstacle" is any object that impedes the progress of a bicycle or vehicle, such as a pedestrian, a car, or another bicycle.

[0805] "Audio alert" is a function that transmits a warning message by voice from an audio output device.

[0806] "Automatic deceleration" is a function that automatically reduces the speed of a bicycle or vehicle when the system detects an obstacle.

[0807] A "server" is a computer system that provides services to other devices and systems over a network.

[0808] "Crawl mode" is a driving mode in which a vehicle or bicycle is controlled to a speed significantly slower than normal.

[0809] "Video analysis" is a technology in which a computer processes images and videos captured by a camera to extract and recognize specific information.

[0810] "Confirm that you are holding the handlebars" is a function that uses sensors to detect how the user is holding the handlebars of a bicycle or vehicle.

[0811] This invention relates to a driving assistance system for bicycles or autonomous vehicles. The system acquires location information, captures images of the road ahead with a camera, and analyzes this information in real time to detect accident-prone areas and obstacles, and issues audio alerts and automatically slows down the vehicle. A specific implementation method for this system will be described below.

[0812] 1. System Configuration

[0813] The system includes the following main components:

[0814] server

[0815] Devices (smartphones and in-car cameras)

[0816] User (bicyclist or vehicle driver)

[0817] 2. System Operation

[0818] Location information acquisition and analysis

[0819] User: Before getting on a bike or vehicle, launch the navigation app on your device.

[0820] Device:

[0821] Enable the GPS function to obtain your current location.

[0822] The acquired location information is sent to the server.

[0823] server:

[0824] The received location information is compared with a database to obtain information on nearby accident-prone areas.

[0825] The acquired information is returned to the device.

[0826] Device:

[0827] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[0828] Example: A voice message will be sent saying, "There is an accident-prone area ahead. Please be careful."

[0829] Video analysis and hazard detection

[0830] Device:

[0831] Use an onboard camera to capture images of the vehicle ahead.

[0832] The captured video is analyzed in real time using a generative AI model.

[0833] Detects the presence of obstacles such as pedestrians, bicycles, and vehicles.

[0834] Example: If a pedestrian suddenly approaches ahead, this is detected through video analysis.

[0835] Example prompt: "Analyze the video in front of the vehicle in real time to detect pedestrians and other obstacles."

[0836] Auto-slowdown and alert features

[0837] Device:

[0838] If the system detects a rapidly approaching obstacle based on video analysis, it will instruct the motor control device of the bicycle or vehicle to automatically slow down.

[0839] Select a specific deceleration mode depending on the type of obstacle.

[0840] Slow down mode for pedestrians.

[0841] Use left-side avoidance mode for other bicycles.

[0842] For automobiles, the alert mode is set by flashing lights.

[0843] User:

[0844] You will be instructed to keep both hands on the steering wheel while slowing down.

[0845] Device:

[0846] Only when the motor controller determines that the user is still holding the handlebars will it gradually reduce the speed of the bicycle or vehicle.

[0847] Hardware and Software

[0848] Hardware used:

[0849] GPS Modules

[0850] In-car camera

[0851] Software used:

[0852] OpenCV (camera image analysis)

[0853] Generative AI model (obstacle detection)

[0854] Specific examples

[0855] Situation: Vehicle approaches a busy intersection

[0856] 1. The user launches the app and enables the GPS function.

[0857] 2. The device sends its current location to the server and obtains data on accident-prone areas.

[0858] 3. The server sends the acquired data back to the device, and the device issues a voice notification saying, "You are approaching an accident-prone area. Please be careful."

[0859] 4. The onboard camera captures images of the road ahead and performs real-time analysis using a generative AI model.

[0860] 5. If the device detects an obstacle such as a pedestrian or another vehicle, it will automatically switch to creep mode and safely slow down.

[0861] In this way, this system helps users to ride bicycles and vehicles safely, significantly reducing the risk of accidents.

[0862] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0863] Step 1:

[0864] User: Before getting on a bike or vehicle, launch the navigation app on your device.

[0865] Input: None

[0866] Output: App launch

[0867] What happens: A user taps to launch an app on their smartphone or vehicle's infotainment system.

[0868] Step 2:

[0869] Device: Enable GPS to obtain your current location.

[0870] Input: None

[0871] Output: Current location information (latitude and longitude)

[0872] Specific operation: The device obtains current location data using the built-in GPS module.

[0873] Step 3:

[0874] Device: Sends the acquired location information to the server.

[0875] Input: Current location information (output of step 2)

[0876] Output: Sending completion message or data to the server

[0877] What it does: Your device sends its current location data to a server via an internet connection.

[0878] Step 4:

[0879] Server: The received location information is compared with a database to obtain information on nearby accident-prone areas.

[0880] Input: Current location

[0881] Output: Accident hotspot information

[0882] Specific operation: The server accesses the database and queries for data on accident-prone areas that match the current location information.

[0883] Step 5:

[0884] Server: Returns the acquired information on accident-prone areas to the terminal.

[0885] Input: Accident hotspot information

[0886] Output: Accident-prone areas information (response data sent to the device)

[0887] Specific operations: The server constructs the search results and sends the data back to the device.

[0888] Step 6:

[0889] Terminal: Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[0890] Input: Accident hotspot information

[0891] Output: Audio alert

[0892] Specific operation: The device uses the built-in speaker to notify the user by voice, "There is an accident-prone area ahead. Please be careful."

[0893] Step 7:

[0894] Terminal: Uses the onboard camera to capture images of the area ahead.

[0895] Input: None

[0896] Output: Video data (live feed)

[0897] Specific operation: The camera captures images of the area in front of the vehicle and acquires real-time video data.

[0898] Step 8:

[0899] Terminal: The captured video is analyzed in real time using a generative AI model.

[0900] Input: Video data

[0901] Output: Obstacle detection results (pedestrians, bicycles, vehicles, etc.)

[0902] Specific operation: The acquired video data is input into a generative AI model, which then detects obstacles such as pedestrians and vehicles.

[0903] Step 9:

[0904] Terminal: Based on video analysis, if a rapidly approaching obstacle is detected, the device instructs the motor control device of the bicycle or vehicle to automatically slow down.

[0905] Input: Obstacle detection result

[0906] Output: Deceleration instruction

[0907] Specific operation: Sends a command to the motor control device to reduce speed depending on the danger level of the obstacle.

[0908] Step 10:

[0909] Device: Check that the user is holding the wheel, and only then will creep mode be activated.

[0910] Input: User's hand position data (steering wheel sensor)

[0911] Output: Crawl mode execution command

[0912] Specific operation: A sensor installed on the handle detects the position of the user's hands and, based on that information, instructs the motor control device to enter slow-motion mode.

[0913] Sample prompt: "Analyze the video in front of the vehicle in real time to detect pedestrians and other obstacles."

[0914] Through these steps, a system that reduces the risk of accidents while riding a bicycle or vehicle and supports safe driving will be efficiently realized.

[0915] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0916] ---

[0917] This invention combines an emotion recognition engine with a bicycle navigation system to improve the safety and comfort of cyclists. In addition to detecting areas prone to traffic accidents and automatically slowing down when encountering obstacles, this system can recognize the user's emotions in real time and take appropriate action based on that data.

[0918] Explanation of program processing

[0919] 1. System Configuration

[0920] The system mainly consists of the following components:

[0921] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[0922] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[0923] User: An individual who uses a bicycle.

[0924] Emotion engine: A software module for recognizing user emotions and analyzing data.

[0925] 2. Specific Program Processing

[0926] Location information acquisition and analysis

[0927] User:

[0928] Before getting on your bike, launch the navigation app on your device.

[0929] Device:

[0930] When the system starts up, it enables the GPS function and begins obtaining the current location.

[0931] The acquired current location information is sent to the server.

[0932] server:

[0933] Based on the received location information, the system compares it with a database to obtain information on nearby accident-prone areas.

[0934] The acquired information is returned to the device.

[0935] Device:

[0936] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[0937] Video analysis and hazard detection

[0938] Device:

[0939] Uses the smartphone camera to capture images of the area ahead.

[0940] The acquired video is temporarily stored in internal memory, and analysis begins using the generating AI.

[0941] Device:

[0942] Generative AI analyzes the image and detects obstacles such as pedestrians, bicycles, and cars.

[0943] Calculates the distance and relative speed of detected obstacles.

[0944] Emotion Recognition and Its Applications

[0945] Device:

[0946] An emotion engine is activated that recognizes emotions from the user's facial expressions, voice, etc.

[0947] An emotion engine assesses the user's stress level, anxiety, and fear.

[0948] Device:

[0949] If the user exhibits high stress levels, the bike speed is automatically adjusted.

[0950] If the user is feeling anxious or scared, the warning alert will be intensified.

[0951] Examples:

[0952] Situation 1: If the user is feeling stressed, the system will detect this through its emotion engine and automatically slow down the speed.

[0953] A voice message will say, "Relax, we'll slow down a bit."

[0954] Situation 2: If the user is feeling fear, the system will issue a detailed warning alert.

[0955] A voice message will be displayed saying, "There is an imminent danger ahead, please slow down immediately."

[0956] Auto-slowdown and alert features

[0957] Device:

[0958] If a rapidly approaching obstacle is detected based on video analysis, the system instructs the bicycle's motor control device to automatically slow down.

[0959] Select a specific deceleration mode depending on the type of obstacle.

[0960] Slow down mode for pedestrians.

[0961] Left-hand driving mode for other bicycles.

[0962] For automobiles, the alert mode is set by flashing lights.

[0963] User:

[0964] You will be instructed to make sure you keep both hands on the wheel while slowing down.

[0965] Device:

[0966] Only with user confirmation will the motor controller gradually reduce the bike's speed.

[0967] Specific examples

[0968] Situation: Approaching an intersection while riding a bicycle

[0969] User:

[0970] Get on your bike and launch the app.

[0971] Device:

[0972] The current location is obtained using GPS and sent to the server.

[0973] server:

[0974] Information on accident-prone areas is acquired and sent to the terminal.

[0975] Device:

[0976] A voice message will be displayed saying, "Be careful before approaching the intersection."

[0977] A camera captures images of the area ahead and analyzes them using generative AI.

[0978] Video analysis detects sudden approach of pedestrians.

[0979] Device:

[0980] A creep mode command is sent to the motor controller and the user is instructed to keep both hands on the steering wheel.

[0981] Once it is confirmed that you are holding the steering wheel, a creep mode is activated and the vehicle gradually slows down.

[0982] Device:

[0983] The emotion engine analyzes the user's emotions and if it detects high stress levels, it will instruct them to slow down further.

[0984] A voice will inform you, "Relax, we'll slow down further."

[0985] In this way, the system of the present invention increases the safety and comfort of the user and significantly reduces the risk of accidents.

[0986] The processing flow will be explained below.

[0987] ---

[0988] Step 1:

[0989] User: Turns on the smartphone and launches the bicycle navigation app.

[0990] Device: When the app is launched, enable GPS and camera functions.

[0991] Step 2:

[0992] Device: Uses GPS to obtain current location information.

[0993] Device: Sends the acquired current location information to the server.

[0994] Step 3:

[0995] Server: Based on the received location information, the server compares it with a database of accident-prone areas to obtain information on nearby accident-prone locations.

[0996] Server: Returns the accident-prone location information obtained as a result of the matching to the terminal.

[0997] Step 4:

[0998] Terminal: Analyzes the accident-prone location information returned from the server and determines whether the current location is close to an accident-prone location.

[0999] Terminal: When approaching a nearby accident hotspot, the device will issue a voice alert to the user.

[1000] Step 5:

[1001] Device: The smartphone camera captures real-time images of the road ahead.

[1002] Device: The acquired video is temporarily stored in the internal memory, and analysis begins using the generation AI.

[1003] Step 6:

[1004] Device: Analyzes video using generative AI and detects obstacles such as pedestrians, bicycles, and cars.

[1005] Terminal: Calculates the distance and relative speed of detected obstacles.

[1006] Step 7:

[1007] Terminal: Based on the video analysis results, evaluate whether there is an approaching obstacle.

[1008] Terminal: Prepares to take specific action (slow mode, drive left mode, flash lights) if assessed as dangerous.

[1009] Step 8:

[1010] Device: Select slow-walk mode for pedestrians, drive left mode for cyclists, and flashing light alert mode for cars.

[1011] Terminal: Tells the motor controller to slow down or flash lights.

[1012] Step 9:

[1013] Device: Runs the emotion engine, analyzes the user's facial expressions and voice, and recognizes their emotional state in real time.

[1014] Device: When the user's stress level, anxiety, or fear is detected, the device analyzes the information and determines the necessary response.

[1015] Step 10:

[1016] Device: If it detects high stress levels in the user, it will issue instructions to automatically slow down the bike.

[1017] Device: A voice prompt tells the user to "Relax, we'll slow down a bit."

[1018] Step 11:

[1019] Device: If the emotion engine detects that the user is anxious or scared, it will increase the intensity of the warning alert.

[1020] Device: A voice message will be displayed saying "There is an imminent danger ahead, please slow down immediately."

[1021] Step 12:

[1022] Terminal: The motor control unit follows the instructions and gradually reduces the speed of the bicycle.

[1023] User: Keep both hands on the steering wheel while slowing down and ensure safe driving.

[1024] In this way, the system monitors the user's emotional state in real time and takes appropriate action as needed, supporting safe cycling.

[1025] Example 2

[1026] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1027] While conventional bicycle navigation systems provide basic risk avoidance based on location information and obstacle detection, they are unable to take the user's emotional state into account and therefore are unable to fully enhance the user's safety and comfort. In particular, when the user is experiencing high levels of stress, impatience, or fear, it is difficult for the system to respond appropriately, potentially increasing the risk of an accident.

[1028] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1029] In this invention, the server includes means for acquiring location information, means for acquiring video of the area ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time to detect obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle when approaching an obstacle, and means for recognizing the user's emotions and adjusting the bicycle speed and warning mode based on the user's emotional state. This allows for appropriate responses according to the user's emotional state, improving the user's safety and comfort.

[1030] A "means for obtaining location information" is a device or system that has the ability to determine current location using GPS or other location information services.

[1031] "Means for obtaining forward images using a camera" refers to a device or system that has the function of capturing and recording images of the area in front of the bicycle using a smartphone or dedicated camera.

[1032] The "means for detecting accident-prone areas" is a system that has the function of identifying areas where traffic accidents are likely to occur by comparing location information with a pre-set database.

[1033] "Means for analyzing captured video in real time and detecting obstacles" refers to a system that uses video processing technology and generative AI models to recognize obstacles such as pedestrians, bicycles, and cars within the video.

[1034] The "means for issuing audio alerts" refers to a system that has the function of issuing audio messages to alert the user based on detected accident hotspots and obstacles.

[1035] "Means for automatically slowing down the bicycle speed" refers to a system that operates the motor control device to slow down the bicycle speed depending on the type and distance of an obstacle detected.

[1036] "Means for recognizing the user's emotions and adjusting the bicycle speed and warning mode based on the emotional state" refers to a system that uses facial expression and voice recognition technology to evaluate the user's emotional state and automatically adjusts speed control and warning methods according to that state.

[1037] This invention improves the safety and comfort of cyclists by combining an emotion recognition engine with a bicycle navigation system that acquires location information, captures camera footage, detects accident hotspots, detects obstacles in real time, issues audio alerts, automatically slows down, and adjusts speed and warning modes based on the user's emotional state.

[1038] The system consists of the following main components:

[1039] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[1040] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[1041] User: An individual who uses a bicycle.

[1042] Emotion engine: A software module for recognizing user emotions and analyzing data.

[1043] Specific operation of the system

[1044] User: Before riding a bicycle, launch the navigation app on the device. Tap the navigation app icon and press the "Start" button to start the GPS.

[1045] Device: When the app is launched, it will enable the GPS function and start acquiring the current location. It will then send that location information to the server.

[1046] Server: Based on the received location information, the server compares it with a database to obtain information on nearby accident-prone areas, and sends the obtained information back to the device.

[1047] Terminal: Based on the returned information, if the user is approaching a nearby accident-prone area, a voice message will be sent saying, "You are approaching an accident-prone area, please be careful."

[1048] Device: Uses the smartphone camera to capture video of the area ahead and temporarily stores it in internal memory. The video is analyzed using a generative AI model to detect obstacles such as pedestrians, bicycles, and cars. The device calculates the distance and relative speed of detected obstacles and manages the obstacle data.

[1049] Device: Operates an emotion engine that recognizes emotions from the user's facial expressions and voice. The emotion engine evaluates the user's stress level, anxiety, and fear. If the user indicates a high stress level, the device automatically adjusts the speed of the bicycle and notifies the user through a voice message saying, "Please relax. We will slow down a bit."

[1050] Terminal: If the terminal detects a rapidly approaching obstacle based on video analysis, it instructs the bicycle's motor control unit to automatically decelerate. It selects a specific deceleration mode depending on the type of obstacle and instructs the user to hold the handlebars with both hands. After confirming that the user follows the instruction and holds the handlebars with both hands, the motor control unit gradually reduces the bicycle's speed.

[1051] Specific examples

[1052] Situation: Approaching an intersection while riding a bicycle

[1053] User: Get on the bike and launch the app.

[1054] Device: Obtains current location using GPS and sends it to the server.

[1055] Server: Obtains information on accident-prone areas and sends it to the device.

[1056] Device: Announces "Be careful before approaching the intersection" with a voice message. The camera captures images of the area ahead and analyzes them with generative AI. Video analysis detects sudden approach of pedestrians.

[1057] Terminal: Sends a creep mode command to the motor control device and instructs the user to hold the steering wheel with both hands. Once it is confirmed that the user is holding the steering wheel, the creep mode is activated and the vehicle gradually decelerates.

[1058] On the device: The emotion engine analyzes the user's emotions and, if it detects high stress levels, instructs them to slow down further. A voice message will say, "Relax, we'll slow down even more."

[1059] Examples of prompt statements

[1060] An example of an input prompt sentence is, "How can a bicycle navigation system use an emotion recognition engine to automatically adjust speed based on the user's emotional state?"

[1061] This system can increase user safety and comfort and significantly reduce the risk of accidents.

[1062] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1063] Step 1:

[1064] User: Before riding a bicycle, launch the navigation app. Tap the navigation app icon and press the "Start" button to start the GPS.

[1065] Input: Launching the app and pressing the "Start" button.

[1066] Specific actions: The navigation app is launched, the GPS function is enabled, and the current location is acquired.

[1067] Output: Current location information.

[1068] Step 2:

[1069] Device: Sends current location information to the server.

[1070] Input: The current location information obtained.

[1071] Specific operation: The location information obtained from the GPS module is sent to the server via mobile data communication.

[1072] Output: Current location information sent to the server.

[1073] Step 3:

[1074] Server: Based on the received location information, the server compares it with a database to obtain information on nearby accident-prone areas.

[1075] Input: Current location information sent from the device.

[1076] What it does: Executes a database query to find information about accident hotspots near the current location.

[1077] Output: Accident hotspot information.

[1078] Step 4:

[1079] Server: Returns the acquired information on accident-prone areas to the terminal.

[1080] Input: Accident black spot information from a database.

[1081] Specific operation: Accident-prone area information is compiled as a text message and sent to the device via mobile data communication.

[1082] Output: Accident hotspot information sent to the device.

[1083] Step 5:

[1084] Terminal: Based on the information received by the terminal, it issues a voice warning to the user, for example, "You are approaching an accident-prone area, so please be careful."

[1085] Input: Accident hotspot information received from the server.

[1086] Specific operation: Converts text information into speech and transmits it to the user through a speaker.

[1087] Output: Audio alert.

[1088] Step 6:

[1089] Device: The device uses a smartphone camera to capture video of the area ahead and temporarily stores it in its internal memory. The device then uses a generative AI model to analyze the video and detect obstacles such as pedestrians, bicycles, and cars.

[1090] Input: Front view captured by a smartphone camera.

[1091] Specific operations: Video capture, saving to internal memory, real-time video analysis and obstacle identification using generative AI.

[1092] Output: Information about detected obstacles (type, distance, relative speed).

[1093] Step 7:

[1094] Terminal: Based on the detected obstacle information, it issues an audio warning to the user.

[1095] Input: Obstacle information analyzed by the generation AI.

[1096] Specific behavior: Generates an appropriate warning message depending on the type of obstacle and notifies the user via audio.

[1097] Output: Audio warning message.

[1098] Step 8:

[1099] Device: Operates an emotion engine that recognizes emotions from the user's facial expressions and voice. The emotion engine evaluates the user's stress level, anxiety, and fear.

[1100] Input: User's facial expression and voice data.

[1101] Specific operation: Analyze and evaluate the user's emotional state using facial expression and voice recognition software.

[1102] Output: User's emotional state data (stress level, anxiety, fear, etc.).

[1103] Step 9:

[1104] Device: If the user exhibits high stress levels, the device automatically adjusts the speed of the bike and provides a voice prompt saying, "Relax, we're going to slow down a bit."

[1105] Input: User emotional state data.

[1106] Specific operation: Instructs the motor control device to adjust the speed, and at the same time generates a voice message and notifies the user through the speaker.

[1107] Output: Adjusted bike speed, voice notification.

[1108] Step 10:

[1109] Terminal: If a rapidly approaching obstacle is detected based on video analysis, the device instructs the bicycle's motor control device to automatically slow down and instructs the user to keep both hands on the handlebars.

[1110] Input: Approaching obstacle based on generative AI analysis.

[1111] Specific actions: Motor control instructions and a voice command to "Hold the steering wheel with both hands."

[1112] Output: Reduced bike speed, voice instructions.

[1113] Step 11:

[1114] User: Follow the instructions and hold the steering wheel with both hands.

[1115] Input: Voice commands.

[1116] Specific action: The user holds the steering wheel with both hands.

[1117] Output: Both hands on the handlebars.

[1118] Step 12:

[1119] Terminal: After confirming that the rider is still holding the handlebars, the motor control unit gradually reduces the speed of the bicycle.

[1120] Input: Confirmation that the user has both hands on the steering wheel.

[1121] Specific operation: The speed is gradually reduced based on instructions from the motor control device.

[1122] Power: Reduced bike speed.

[1123] (Application example 2)

[1124] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1125] In bicycle navigation systems, detecting traffic accident hotspots and automatically slowing down when encountering obstacles are important accident prevention measures, but many systems do not take the user's emotional state into account. As a result, they may not respond appropriately when the user feels strong stress or fear, which could increase the risk of an accident. Furthermore, current bicycle navigation systems do not incorporate real-time emotion recognition and driving style adjustment, making it difficult to ensure sufficient safety and comfort.

[1126] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring location information, means for acquiring video of the road ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time and detecting obstacles, means for recognizing the user's emotions using an emotion recognition engine and adjusting the driving style according to the stress level, means for issuing an audio alert when an accident-prone area or an obstacle is detected, and means for automatically slowing down when approaching an obstacle. This enables appropriate action to be taken in real time when the user is feeling high stress or fear, thereby reducing the risk of an accident and improving the safety and comfort of the user.

[1127] "Means for obtaining location information" refers to devices or software that obtain location information using GPS functions or other location measurement technologies to identify a user's current location.

[1128] "Means for obtaining forward-looking images using a camera" refers to devices or software that use a camera installed in front of a bicycle or autonomous vehicle to obtain video data in real time.

[1129] "Means for detecting accident-prone areas" refers to devices or software that use acquired location information and external databases to detect areas where traffic accidents frequently occur.

[1130] "Means for analyzing video in real time and detecting obstacles" refers to devices or software that use generative AI models or deep learning technology to analyze acquired video data in real time and detect pedestrians, vehicles, and other obstacles.

[1131] "Means for recognizing a user's emotions using an emotion recognition engine" refers to devices or software that use deep learning models or algorithms to analyze data on a user's facial expressions and voice to recognize their emotional state.

[1132] "Means for adjusting driving style according to stress level" refers to devices or software for adjusting the speed or changing the driving mode of a bicycle or autonomous vehicle based on the stress level recognized by the emotion recognition engine.

[1133] The "means for issuing an audio alert" refers to a device or software that issues an audio alert to alert the user based on detected accident-prone areas or obstacle information.

[1134] "Means for automatically slowing down around obstacles" means motor controls and software for automatically reducing vehicle speed based on the location and distance of a detected obstacle.

[1135] MODE FOR CARRYING OUT THE INVENTION

[1136] This invention relates to a navigation system for an autonomous vehicle that recognizes a user's emotional state in real time to optimize the driving style and improve safety and comfort. The system includes a means for acquiring location information, a means for capturing images of the road ahead using a camera, a means for detecting accident-prone areas, a means for analyzing the images in real time to detect obstacles, a means for recognizing the user's emotions using an emotion recognition engine, a means for adjusting the driving style according to the user's stress level, a means for issuing an audio alert, and a means for automatically slowing down the vehicle.

[1137] Hardware and Software Configuration

[1138] Hardware:

[1139] GPS module: Obtains location information.

[1140] Camera: Captures real-time images of the area ahead.

[1141] Motor control unit: Controls the speed of an autonomous vehicle.

[1142] Microphone: Captures the user's voice.

[1143] software:

[1144] Emotion recognition engine: Analyzes emotions from the user's facial expressions and voice.

[1145] Generative AI model: Analyzes images ahead and detects obstacles.

[1146] Traffic information API: Links with external databases to obtain information on areas prone to accidents.

[1147] Program processing

[1148] server

[1149] The server receives the location information and compares it with a database of accident-prone areas. The results are sent back to the device, which then issues an alert about any dangerous areas the user is approaching. The server also analyzes the captured video data with a generative AI model to detect obstacles in real time.

[1150] Terminal

[1151] The device captures images of the road ahead using a camera and temporarily stores them. An emotion recognition engine analyzes the user's facial expressions and voice, monitoring their emotions in real time. Based on their emotional state and the captured images, the device adjusts its driving style (speed and warning level) appropriately.

[1152] User

[1153] The user launches the application and gets into the self-driving vehicle. The system provides voice alerts and adjusts driving according to the user's emotional state, and the user adjusts their driving accordingly.

[1154] Specific examples

[1155] 1. Situation: A user is driving an autonomous vehicle and approaches a congested area.

[1156] 2. Terminal:

[1157] The camera captures images of the road ahead, and a generative AI model analyzes the traffic congestion situation.

[1158] The user's voice is picked up by a microphone and stress levels are measured using an emotion recognition engine.

[1159] 3. Server:

[1160] Based on the acquired location information, traffic congestion information is obtained via a traffic information API.

[1161] 4. Results:

[1162] If the user indicates high stress levels, the vehicle's speed will be automatically reduced and a voice notification will be provided saying, "Relax. We will reduce your speed."

[1163] Prompt Sentence Examples

[1164] "Create an application that uses an emotion recognition model to analyze passenger emotions in real time and adjust the vehicle's driving style based on those emotions. Additionally, use an external traffic information API to obtain traffic congestion information and include guidelines to reduce passenger stress."

[1165] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1166] Step 1:

[1167] The device acquires video of the road ahead using the autonomous vehicle's camera, uses the real-time video data acquired from the camera as input, temporarily stores it in its internal memory, and provides video data for the next processing step as output.

[1168] Step 2:

[1169] The device uses an emotion recognition engine to recognize the user's emotions in real time. It uses audio data acquired from a microphone and facial expression data acquired from a camera as input. The emotion recognition engine analyzes this data to identify the user's emotional state. The output is generated as the user's emotional state data (e.g., stress level, high, medium, low).

[1170] Step 3:

[1171] The device sends the location information it has acquired to the server. It uses the location data from the GPS module as input. It sends the location data to the server and requests information on accident-prone areas. It receives the information on accident-prone areas returned from the server as output.

[1172] Step 4:

[1173] The server compares the location data it receives with a database of accident-prone areas. The location data and the information on accident-prone areas in the database are used as input. The database is compared to extract information on the relevant accident-prone areas and sent to the terminal. The information on the relevant accident-prone areas is provided as output.

[1174] Step 5:

[1175] An audio alert is issued to the user based on the information on accident-prone areas received by the device. The information on accident-prone areas and the previously acquired video data are used as input. An audio alert generation engine is used to warn the user. The audio data that issues the audio alert is generated as output and executed.

[1176] Step 6:

[1177] The video data acquired by the device is analyzed in real time using a generative AI model to detect obstacles. Video data of the front is used as input. The generative AI model is used to recognize objects on the screen and identify the type of obstacle (pedestrian, car, etc.), as well as their relative speed and distance. Information data on the detected obstacle is generated as output and used for the next deceleration process.

[1178] Step 7:

[1179] The device comprehensively assesses the emotional state, obstacle data, and information on accident-prone areas, and sends instructions to the motor controller to adjust the driving style. It also uses the emotional state data, obstacle data, and information on accident-prone areas as inputs. This includes issuing instructions to reduce speed when stress levels are high or obstacles are nearby. It generates deceleration command data that is sent to the motor controller as output.

[1180] Step 8:

[1181] The motor control device receives the deceleration command from the terminal and adjusts the speed of the autonomous vehicle. It uses the deceleration command data as input, adjusts the motor output of the autonomous vehicle according to the deceleration command, and reduces the speed appropriately. It feeds back the actual speed data of the decelerated vehicle as output, and maintains a safe driving style.

[1182] In this way, the steps work in cooperation with each other, thereby making it possible to improve the safety and comfort of the user.

[1183] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1184] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1185] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1186] [Third embodiment]

[1187] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1188] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1189] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1190] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1191] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1192] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1193] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1194] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1195] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1196] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1197] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1198] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1199] ---

[1200] The present invention relates to a bicycle navigation system, and provides a support system for cyclists to ride safely. This system has functions to detect traffic accident hot spots and automatically slow down when encountering obstacles, and can provide specific measures for users to avoid accidents.

[1201] Explanation of program processing

[1202] 1. System Configuration

[1203] The system mainly consists of the following components:

[1204] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[1205] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[1206] User: An individual who uses a bicycle.

[1207] 2. Specific Program Processing

[1208] Location information acquisition and analysis

[1209] User:

[1210] Before getting on your bike, launch the navigation app on your device.

[1211] Device:

[1212] When the system starts up, it enables the GPS function and begins obtaining the current location.

[1213] The acquired current location information is sent to the server.

[1214] server:

[1215] Based on the received location information, the system compares it with a database to obtain information on nearby accident-prone areas.

[1216] The acquired information is returned to the device.

[1217] Device:

[1218] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[1219] Examples:

[1220] - A voice notification will say, "There is an accident-prone area ahead. Please be careful."

[1221] Video analysis and hazard detection

[1222] Device:

[1223] Uses the smartphone camera to capture images of the area ahead.

[1224] The acquired video is analyzed in real time using generative AI.

[1225] Detects the presence of obstacles such as pedestrians, bicycles, and cars.

[1226] Examples:

[1227] - If a pedestrian suddenly approaches ahead, it will be detected through video analysis.

[1228] Auto-slowdown and alert features

[1229] Device:

[1230] If a rapidly approaching obstacle is detected based on video analysis, the system instructs the bicycle's motor control device to automatically slow down.

[1231] Select a specific deceleration mode depending on the type of obstacle.

[1232] Slow down mode for pedestrians.

[1233] Use left-side avoidance mode for other bicycles.

[1234] For automobiles, the alert mode is set by flashing lights.

[1235] User:

[1236] You will be instructed to make sure you keep both hands on the wheel while slowing down.

[1237] Device:

[1238] Only with user confirmation will the motor controller gradually reduce the bike's speed.

[1239] Specific examples

[1240] Situation: Approaching an intersection while riding a bicycle

[1241] User:

[1242] Get on your bike and launch the app.

[1243] Device:

[1244] The current location is obtained using GPS and sent to the server.

[1245] server:

[1246] Information on accident-prone areas is acquired and sent to the terminal.

[1247] Device:

[1248] A voice message will be displayed saying, "Be careful before approaching the intersection."

[1249] A camera captures images of the area ahead and analyzes them using generative AI.

[1250] Video analysis detects sudden approach of pedestrians.

[1251] Device:

[1252] A creep mode command is sent to the motor controller and the user is instructed to keep both hands on the steering wheel.

[1253] Once it is confirmed that you are holding the steering wheel, a creep mode is activated and the vehicle gradually slows down.

[1254] In this way, the system of the present invention helps users to ride bicycles safely and significantly reduces the risk of accidents.

[1255] The processing flow will be explained below.

[1256] ---

[1257] Step 1:

[1258] User: Turns on the smartphone and launches the bicycle navigation app.

[1259] Device: When the app is launched, enable GPS and camera functions.

[1260] Step 2:

[1261] Device: Uses GPS to obtain current location information.

[1262] Device: Sends the acquired current location information to the server.

[1263] Step 3:

[1264] Server: Based on the received location information, the server compares it with a database of accident-prone areas to obtain information on nearby accident-prone locations.

[1265] Server: Returns the accident-prone location information obtained as a result of the matching to the terminal.

[1266] Step 4:

[1267] Terminal: Analyzes the accident-prone location information returned from the server and determines whether the current location is close to an accident-prone location.

[1268] Terminal: When approaching a nearby accident hotspot, the device will issue a voice alert to the user.

[1269] Step 5:

[1270] Device: The smartphone camera captures real-time images of the road ahead.

[1271] Device: The acquired video is temporarily stored in the internal memory, and analysis begins using the generation AI.

[1272] Step 6:

[1273] Device: Analyzes video using generative AI and detects obstacles such as pedestrians, bicycles, and cars.

[1274] Terminal: Calculates the distance and relative speed of detected obstacles.

[1275] Step 7:

[1276] Terminal: Based on the video analysis results, evaluate whether there is an approaching obstacle.

[1277] Terminal: Prepares to take specific action (slow mode, drive left mode, flash lights) if assessed as dangerous.

[1278] Step 8:

[1279] Device: Select slow-walk mode for pedestrians, drive left mode for cyclists, and flashing light alert mode for cars.

[1280] Terminal: Tells the motor controller to slow down or flash lights.

[1281] Step 9:

[1282] Device: A sensor checks whether the user has both hands on the steering wheel.

[1283] Terminal: If the driver is confirmed to be holding the steering wheel, the motor control unit will automatically begin a gradual deceleration process.

[1284] Step 10:

[1285] Terminal: Ensures that the bicycle's speed has been reduced to a safe level for obstacles.

[1286] User: Keep both hands on the steering wheel while slowing down and ensure safe driving.

[1287] This allows the system to automatically take appropriate measures before the user gets into a dangerous situation, supporting safe cycling.

[1288] Example 1

[1289] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1290] Systems designed to help cyclists ride safely face challenges due to their inadequate ability to detect frequent traffic accidents and obstacles. In particular, they lack the ability to obtain real-time location information, provide information about accident-prone areas and obstacles, and automatically adjust speed. They also need a way to avoid unnecessary slowdowns by obtaining user confirmation.

[1291] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1292] In this invention, the server includes means for acquiring location information, means for acquiring video of the area ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time and detecting obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle speed toward an obstacle, and means for executing the speed reduction when the user agrees to the speed reduction. This makes it possible to detect accident-prone areas and obstacles and provide appropriate alerts and deceleration to ensure safe cycling for cyclists.

[1293] "Means of obtaining location information" refers to a function that measures the current location using a smartphone, GPS device, etc. and obtains the data.

[1294] "Means for obtaining images of the front using a camera" refers to a function that collects images of the front of the bicycle in real time using a smartphone, camera device, etc.

[1295] "Means for detecting accident-prone areas" is a function that identifies areas where traffic accidents have occurred frequently in the past based on location information and extracts the necessary data.

[1296] "Means for analyzing and detecting obstacles in real time" refers to a function that instantly analyzes the images captured by the camera and identifies obstacles such as pedestrians, bicycles, and automobiles.

[1297] "Means for issuing audio alerts" is a function that issues an audio warning to the user when an accident-prone area or obstacle is detected.

[1298] "Means for automatically reducing the speed of the bicycle" is a function that automatically controls the speed of the bicycle in response to detected obstacles, ensuring safety.

[1299] The "means for slowing down when the user agrees to slow down" is a function that notifies the user of the need to slow down and reduces the speed only if the user complies with the instruction.

[1300] "Generative AI" is an artificial intelligence that performs real-time image analysis based on video data acquired by a camera to determine the type and location of obstacles.

[1301] This invention provides a bicycle navigation system to support cyclists in riding safely. In particular, it has the function of detecting traffic accident hotspots and automatically slowing down the vehicle when encountering obstacles, aiming to significantly reduce the risk of accidents. This system consists of three main elements: a server, a terminal, and a user.

[1302] System Components

[1303] server:

[1304] The server maintains a database of areas prone to traffic accidents and provides location information to the terminal. The server receives the location information and returns the information on areas prone to traffic accidents to the terminal.

[1305] Device:

[1306] The terminal is a mobile device such as a smartphone, and uses GPS and camera functions. When the system is started, the terminal begins acquiring its current location and acquires location information and images of the area ahead in real time. The acquired images are then analyzed using a generative AI model to detect obstacles and issue instructions for automatic deceleration.

[1307] User:

[1308] The user is an individual who rides a bicycle. Before riding, the user launches the navigation app and follows the alerts and instructions from the device.

[1309] Hardware and software used

[1310] GPS Module:

[1311] It is built into the device and acquires the current location.

[1312] camera:

[1313] It is built into the terminal and captures images of what is in front of it.

[1314] Generative AI models:

[1315] It runs on the device, analyzes images captured by the camera in real time, and detects obstacles.

[1316] Motor Controller:

[1317] A device for automatically adjusting the speed of a bicycle.

[1318] Specific Examples of the Invention

[1319] Situation: Approaching an intersection while riding a bicycle

[1320] User:

[1321] The user gets on the bicycle and starts the navigation app on the device. The user drives the bicycle following the instructions of the app.

[1322] Device:

[1323] When the navigation app is launched, it uses the GPS module to obtain the current location and sends that information to the server. Based on the information on accident-prone areas returned from the server, it notifies the user by voice, "There is an accident-prone area ahead. Please be careful." It also uses a camera to obtain images of the area ahead and analyzes them in real time using a generative AI model. Based on the analysis results, if the device detects a pedestrian approaching quickly, it instructs the motor control device to enter slow-down mode.

[1324] User:

[1325] When instructed to slow down, the user ensures they keep both hands on the steering wheel, then follows further instructions from the device to drive safely.

[1326] In this way, the system of the present invention combines real-time location information with video analytics to provide specific alerts and slow-down instructions to help cyclists avoid accidents, thereby increasing cyclist safety and significantly reducing the risk of accidents.

[1327] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1328] Step 1:

[1329] The user launches a navigation app. The input is the user tapping the navigation app on their smartphone. The output is the navigation app launching.

[1330] Specific operation:

[1331] The user taps the navigation app icon on the smartphone home screen.

[1332] Step 2:

[1333] The device uses the GPS module to obtain its current location. The input is satellite data received by the device's GPS module. The output is the current location data.

[1334] Specific operation:

[1335] The smartphone obtains the current location information from satellites and sends that information to the location management module within the app.

[1336] Step 3:

[1337] The current location information acquired by the device is sent to the server. The input is the current location data. The output is the location information data sent to the server.

[1338] Specific operation:

[1339] The GPS information is sent to the server using the terminal's network module.

[1340] Step 4:

[1341] Based on the location information received by the server, information on accident-prone areas is retrieved from the database. The input is the current location data sent from the device. The output is information on accident-prone areas.

[1342] Specific operation:

[1343] The server queries the database and retrieves the results.

[1344] Step 5:

[1345] The server returns the information about accident-prone areas it has acquired to the terminal. The input is the information about accident-prone areas acquired by the server. The output is the information about accident-prone areas that is sent to the terminal.

[1346] Specific operation:

[1347] The server obtains information about areas prone to accidents from the database and transmits it to the terminal via the network.

[1348] Step 6:

[1349] The terminal issues a voice alert to the user based on the returned information on accident-prone areas. The input is the accident-prone area information sent from the server. The output is the issuance of a voice alert.

[1350] Specific operation:

[1351] The device will notify you with a voice message saying, "There is an accident-prone area ahead. Please be careful."

[1352] Step 7:

[1353] The device uses the smartphone camera to capture images of the front. The input is the camera image data. The output is the captured front image data.

[1354] Specific operation:

[1355] The camera continuously captures images of the area ahead and stores the data in the device's memory.

[1356] Step 8:

[1357] The image captured by the device is input into a generative AI model and analyzed in real time. The input is the image data from the camera. The output is the analyzed obstacle data.

[1358] Specific operation:

[1359] The device inputs the video data into a generative AI model, which then analyzes the data to identify the type and location of obstacles.

[1360] Step 9:

[1361] The device detects obstacles based on the analysis results and instructs the bicycle's motor control device to slow down. The input is the analysis result from the generative AI model. The output is a command to the motor control device to slow down.

[1362] Specific operation:

[1363] If a pedestrian is detected, the device will issue an alert saying "Pedestrian detected, bicycle speed set to slow mode" and send a command to the motor control device to slow down.

[1364] Step 10:

[1365] Verify that the user keeps both hands on the steering wheel when instructed to slow down. The input is an audio alert. The output is the user gripping the steering wheel.

[1366] Specific operation:

[1367] The user follows the instruction "Please hold the steering wheel with both hands" and performs an operation to confirm that he or she is holding the steering wheel.

[1368] Step 11:

[1369] The terminal gradually slows down after the user confirms. The input is the user's confirmation. The output is a decrease in the bicycle's speed.

[1370] Specific operation:

[1371] The device detects the user's confirmation and begins to safely slow down, gradually reducing the bicycle's speed.

[1372] The bicycle navigation system of the present invention is implemented through the above steps. The specific operations and data inputs and outputs at each step ensure that the system functions accurately and safely.

[1373] (Application example 1)

[1374] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1375] There is a need to reduce the risk of traffic accidents while riding a bicycle and ensure safety. However, current bicycle navigation systems and automated driving assistance systems do not fully fulfill their functions of detecting accident-prone areas in real time or automatically detecting and slowing down obstacles. Another problem is the lack of user awareness in situations where the risk of an accident is high. In these circumstances, it is necessary to develop a reliable accident avoidance and deceleration system.

[1376] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1377] In this invention, the server includes means for acquiring location information, means for acquiring video of the road ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time to detect obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle when approaching an obstacle, means for analyzing the camera video with a generative AI model to detect objects, means for confirming that the handlebars of the bicycle are being held, and means for controlling the bicycle's slow-down mode. This makes it possible to detect traffic accident risks while riding a bicycle in real time and to appropriately slow down the bicycle.

[1378] "Location information" is data that indicates a specific location or coordinates using technology such as GPS.

[1379] A "camera" is a device that receives light and records and displays images or videos.

[1380] An "accident hotspot" is a location in a particular area or section of road where traffic accidents occur frequently.

[1381] A "generative AI model" is a computer program that has been trained using machine learning algorithms and is used to automate specific tasks.

[1382] An "obstacle" is any object that impedes the progress of a bicycle or vehicle, such as a pedestrian, a car, or another bicycle.

[1383] "Audio alert" is a function that transmits a warning message by voice from an audio output device.

[1384] "Automatic deceleration" is a function that automatically reduces the speed of a bicycle or vehicle when the system detects an obstacle.

[1385] A "server" is a computer system that provides services to other devices and systems over a network.

[1386] "Crawl mode" is a driving mode in which a vehicle or bicycle is controlled to a speed significantly slower than normal.

[1387] "Video analysis" is a technology in which a computer processes images and videos captured by a camera to extract and recognize specific information.

[1388] "Confirm that you are holding the handlebars" is a function that uses sensors to detect how the user is holding the handlebars of a bicycle or vehicle.

[1389] This invention relates to a driving assistance system for bicycles or autonomous vehicles. The system acquires location information, captures images of the road ahead with a camera, and analyzes this information in real time to detect accident-prone areas and obstacles, and issues audio alerts and automatically slows down the vehicle. A specific implementation method for this system will be described below.

[1390] 1. System Configuration

[1391] The system includes the following main components:

[1392] server

[1393] Devices (smartphones and in-car cameras)

[1394] User (bicyclist or vehicle driver)

[1395] 2. System Operation

[1396] Location information acquisition and analysis

[1397] User: Before getting on a bike or vehicle, launch the navigation app on your device.

[1398] Device:

[1399] Enable the GPS function to obtain your current location.

[1400] The acquired location information is sent to the server.

[1401] server:

[1402] The received location information is compared with a database to obtain information on nearby accident-prone areas.

[1403] The acquired information is returned to the device.

[1404] Device:

[1405] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[1406] Example: A voice message will be sent saying, "There is an accident-prone area ahead. Please be careful."

[1407] Video analysis and hazard detection

[1408] Device:

[1409] Use an onboard camera to capture images of the vehicle ahead.

[1410] The captured video is analyzed in real time using a generative AI model.

[1411] Detects the presence of obstacles such as pedestrians, bicycles, and vehicles.

[1412] Example: If a pedestrian suddenly approaches ahead, this is detected through video analysis.

[1413] Example prompt: "Analyze the video in front of the vehicle in real time to detect pedestrians and other obstacles."

[1414] Auto-slowdown and alert features

[1415] Device:

[1416] If the system detects a rapidly approaching obstacle based on video analysis, it will instruct the motor control device of the bicycle or vehicle to automatically slow down.

[1417] Select a specific deceleration mode depending on the type of obstacle.

[1418] Slow down mode for pedestrians.

[1419] Use left-side avoidance mode for other bicycles.

[1420] For automobiles, the alert mode is set by flashing lights.

[1421] User:

[1422] You will be instructed to keep both hands on the steering wheel while slowing down.

[1423] Device:

[1424] Only when the motor controller determines that the user is still holding the handlebars will it gradually reduce the speed of the bicycle or vehicle.

[1425] Hardware and Software

[1426] Hardware used:

[1427] GPS Modules

[1428] In-car camera

[1429] Software used:

[1430] OpenCV (camera image analysis)

[1431] Generative AI model (obstacle detection)

[1432] Specific examples

[1433] Situation: Vehicle approaches a busy intersection

[1434] 1. The user launches the app and enables the GPS function.

[1435] 2. The device sends its current location to the server and obtains data on accident-prone areas.

[1436] 3. The server sends the acquired data back to the device, and the device issues a voice notification saying, "You are approaching an accident-prone area. Please be careful."

[1437] 4. The onboard camera captures images of the road ahead and performs real-time analysis using a generative AI model.

[1438] 5. If the device detects an obstacle such as a pedestrian or another vehicle, it will automatically switch to creep mode and safely slow down.

[1439] In this way, this system helps users to ride bicycles and vehicles safely, significantly reducing the risk of accidents.

[1440] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1441] Step 1:

[1442] User: Before getting on a bike or vehicle, launch the navigation app on your device.

[1443] Input: None

[1444] Output: App launch

[1445] What happens: A user taps to launch an app on their smartphone or vehicle's infotainment system.

[1446] Step 2:

[1447] Device: Enable GPS to obtain your current location.

[1448] Input: None

[1449] Output: Current location information (latitude and longitude)

[1450] Specific operation: The device obtains current location data using the built-in GPS module.

[1451] Step 3:

[1452] Device: Sends the acquired location information to the server.

[1453] Input: Current location information (output of step 2)

[1454] Output: Sending completion message or data to the server

[1455] What it does: Your device sends its current location data to a server via an internet connection.

[1456] Step 4:

[1457] Server: The received location information is compared with a database to obtain information on nearby accident-prone areas.

[1458] Input: Current location

[1459] Output: Accident hotspot information

[1460] Specific operation: The server accesses the database and queries for data on accident-prone areas that match the current location information.

[1461] Step 5:

[1462] Server: Returns the acquired information on accident-prone areas to the terminal.

[1463] Input: Accident hotspot information

[1464] Output: Accident-prone areas information (response data sent to the device)

[1465] Specific operations: The server constructs the search results and sends the data back to the device.

[1466] Step 6:

[1467] Terminal: Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[1468] Input: Accident hotspot information

[1469] Output: Audio alert

[1470] Specific operation: The device uses the built-in speaker to notify the user by voice, "There is an accident-prone area ahead. Please be careful."

[1471] Step 7:

[1472] Terminal: Uses the onboard camera to capture images of the area ahead.

[1473] Input: None

[1474] Output: Video data (live feed)

[1475] Specific operation: The camera captures images of the area in front of the vehicle and acquires real-time video data.

[1476] Step 8:

[1477] Terminal: The captured video is analyzed in real time using a generative AI model.

[1478] Input: Video data

[1479] Output: Obstacle detection results (pedestrians, bicycles, vehicles, etc.)

[1480] Specific operation: The acquired video data is input into a generative AI model, which then detects obstacles such as pedestrians and vehicles.

[1481] Step 9:

[1482] Terminal: Based on video analysis, if a rapidly approaching obstacle is detected, the device instructs the motor control device of the bicycle or vehicle to automatically slow down.

[1483] Input: Obstacle detection result

[1484] Output: Deceleration instruction

[1485] Specific operation: Sends a command to the motor control device to reduce speed depending on the danger level of the obstacle.

[1486] Step 10:

[1487] Device: Check that the user is holding the wheel, and only then will creep mode be activated.

[1488] Input: User's hand position data (steering wheel sensor)

[1489] Output: Crawl mode execution command

[1490] Specific operation: A sensor installed on the handle detects the position of the user's hands and, based on that information, instructs the motor control device to enter slow-motion mode.

[1491] Sample prompt: "Analyze the video in front of the vehicle in real time to detect pedestrians and other obstacles."

[1492] Through these steps, a system that reduces the risk of accidents while riding a bicycle or vehicle and supports safe driving will be efficiently realized.

[1493] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1494] ---

[1495] This invention combines an emotion recognition engine with a bicycle navigation system to improve the safety and comfort of cyclists. In addition to detecting areas prone to traffic accidents and automatically slowing down when encountering obstacles, this system can recognize the user's emotions in real time and take appropriate action based on that data.

[1496] Explanation of program processing

[1497] 1. System Configuration

[1498] The system mainly consists of the following components:

[1499] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[1500] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[1501] User: An individual who uses a bicycle.

[1502] Emotion engine: A software module for recognizing user emotions and analyzing data.

[1503] 2. Specific Program Processing

[1504] Location information acquisition and analysis

[1505] User:

[1506] Before getting on your bike, launch the navigation app on your device.

[1507] Device:

[1508] When the system starts up, it enables the GPS function and begins obtaining the current location.

[1509] The acquired current location information is sent to the server.

[1510] server:

[1511] Based on the received location information, the system compares it with a database to obtain information on nearby accident-prone areas.

[1512] The acquired information is returned to the device.

[1513] Device:

[1514] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[1515] Video analysis and hazard detection

[1516] Device:

[1517] Uses the smartphone camera to capture images of the area ahead.

[1518] The acquired video is temporarily stored in internal memory, and analysis begins using the generating AI.

[1519] Device:

[1520] Generative AI analyzes the image and detects obstacles such as pedestrians, bicycles, and cars.

[1521] Calculates the distance and relative speed of detected obstacles.

[1522] Emotion Recognition and Its Applications

[1523] Device:

[1524] An emotion engine is activated that recognizes emotions from the user's facial expressions, voice, etc.

[1525] An emotion engine assesses the user's stress level, anxiety, and fear.

[1526] Device:

[1527] If the user exhibits high stress levels, the bike speed is automatically adjusted.

[1528] If the user is feeling anxious or scared, the warning alert will be intensified.

[1529] Examples:

[1530] Situation 1: If the user is feeling stressed, the system will detect this through its emotion engine and automatically slow down the speed.

[1531] A voice message will say, "Relax, we'll slow down a bit."

[1532] Situation 2: If the user is feeling fear, the system will issue a detailed warning alert.

[1533] A voice message will be displayed saying, "There is an imminent danger ahead, please slow down immediately."

[1534] Auto-slowdown and alert features

[1535] Device:

[1536] If a rapidly approaching obstacle is detected based on video analysis, the system instructs the bicycle's motor control device to automatically slow down.

[1537] Select a specific deceleration mode depending on the type of obstacle.

[1538] Slow down mode for pedestrians.

[1539] Left-hand driving mode for other bicycles.

[1540] For automobiles, the alert mode is set by flashing lights.

[1541] User:

[1542] You will be instructed to make sure you keep both hands on the wheel while slowing down.

[1543] Device:

[1544] Only with user confirmation will the motor controller gradually reduce the bike's speed.

[1545] Specific examples

[1546] Situation: Approaching an intersection while riding a bicycle

[1547] User:

[1548] Get on your bike and launch the app.

[1549] Device:

[1550] The current location is obtained using GPS and sent to the server.

[1551] server:

[1552] Information on accident-prone areas is acquired and sent to the terminal.

[1553] Device:

[1554] A voice message will be displayed saying, "Be careful before approaching the intersection."

[1555] A camera captures images of the area ahead and analyzes them using generative AI.

[1556] Video analysis detects sudden approach of pedestrians.

[1557] Device:

[1558] A creep mode command is sent to the motor controller and the user is instructed to keep both hands on the steering wheel.

[1559] Once it is confirmed that you are holding the steering wheel, a creep mode is activated and the vehicle gradually slows down.

[1560] Device:

[1561] The emotion engine analyzes the user's emotions and if it detects high stress levels, it will instruct them to slow down further.

[1562] A voice will inform you, "Relax, we'll slow down further."

[1563] In this way, the system of the present invention increases the safety and comfort of the user and significantly reduces the risk of accidents.

[1564] The processing flow will be explained below.

[1565] ---

[1566] Step 1:

[1567] User: Turns on the smartphone and launches the bicycle navigation app.

[1568] Device: When the app is launched, enable GPS and camera functions.

[1569] Step 2:

[1570] Device: Uses GPS to obtain current location information.

[1571] Device: Sends the acquired current location information to the server.

[1572] Step 3:

[1573] Server: Based on the received location information, the server compares it with a database of accident-prone areas to obtain information on nearby accident-prone locations.

[1574] Server: Returns the accident-prone location information obtained as a result of the matching to the terminal.

[1575] Step 4:

[1576] Terminal: Analyzes the accident-prone location information returned from the server and determines whether the current location is close to an accident-prone location.

[1577] Terminal: When approaching a nearby accident hotspot, the device will issue a voice alert to the user.

[1578] Step 5:

[1579] Device: The smartphone camera captures real-time images of the road ahead.

[1580] Device: The acquired video is temporarily stored in the internal memory, and analysis begins using the generation AI.

[1581] Step 6:

[1582] Device: Analyzes video using generative AI and detects obstacles such as pedestrians, bicycles, and cars.

[1583] Terminal: Calculates the distance and relative speed of detected obstacles.

[1584] Step 7:

[1585] Terminal: Based on the video analysis results, evaluate whether there is an approaching obstacle.

[1586] Terminal: Prepares to take specific action (slow mode, drive left mode, flash lights) if assessed as dangerous.

[1587] Step 8:

[1588] Device: Select slow-walk mode for pedestrians, drive left mode for cyclists, and flashing light alert mode for cars.

[1589] Terminal: Tells the motor controller to slow down or flash lights.

[1590] Step 9:

[1591] Device: Runs the emotion engine, analyzes the user's facial expressions and voice, and recognizes their emotional state in real time.

[1592] Device: When the user's stress level, anxiety, or fear is detected, the device analyzes the information and determines the necessary response.

[1593] Step 10:

[1594] Device: If it detects high stress levels in the user, it will issue instructions to automatically slow down the bike.

[1595] Device: A voice prompt tells the user to "Relax, we'll slow down a bit."

[1596] Step 11:

[1597] Device: If the emotion engine detects that the user is anxious or scared, it will increase the intensity of the warning alert.

[1598] Device: A voice message will be displayed saying "There is an imminent danger ahead, please slow down immediately."

[1599] Step 12:

[1600] Terminal: The motor control unit follows the instructions and gradually reduces the speed of the bicycle.

[1601] User: Keep both hands on the steering wheel while slowing down and ensure safe driving.

[1602] In this way, the system monitors the user's emotional state in real time and takes appropriate action as needed, supporting safe cycling.

[1603] Example 2

[1604] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1605] While conventional bicycle navigation systems provide basic risk avoidance based on location information and obstacle detection, they are unable to take the user's emotional state into account and therefore are unable to fully enhance the user's safety and comfort. In particular, when the user is experiencing high levels of stress, impatience, or fear, it is difficult for the system to respond appropriately, potentially increasing the risk of an accident.

[1606] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1607] In this invention, the server includes means for acquiring location information, means for acquiring video of the area ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time to detect obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle when approaching an obstacle, and means for recognizing the user's emotions and adjusting the bicycle speed and warning mode based on the user's emotional state. This allows for appropriate responses according to the user's emotional state, improving the user's safety and comfort.

[1608] A "means for obtaining location information" is a device or system that has the ability to determine current location using GPS or other location information services.

[1609] "Means for obtaining forward images using a camera" refers to a device or system that has the function of capturing and recording images of the area in front of the bicycle using a smartphone or dedicated camera.

[1610] The "means for detecting accident-prone areas" is a system that has the function of identifying areas where traffic accidents are likely to occur by comparing location information with a pre-set database.

[1611] "Means for analyzing captured video in real time and detecting obstacles" refers to a system that uses video processing technology and generative AI models to recognize obstacles such as pedestrians, bicycles, and cars within the video.

[1612] The "means for issuing audio alerts" refers to a system that has the function of issuing audio messages to alert the user based on detected accident hotspots and obstacles.

[1613] "Means for automatically slowing down the bicycle speed" refers to a system that operates the motor control device to slow down the bicycle speed depending on the type and distance of an obstacle detected.

[1614] "Means for recognizing the user's emotions and adjusting the bicycle speed and warning mode based on the emotional state" refers to a system that uses facial expression and voice recognition technology to evaluate the user's emotional state and automatically adjusts speed control and warning methods according to that state.

[1615] This invention improves the safety and comfort of cyclists by combining an emotion recognition engine with a bicycle navigation system that acquires location information, captures camera footage, detects accident hotspots, detects obstacles in real time, issues audio alerts, automatically slows down, and adjusts speed and warning modes based on the user's emotional state.

[1616] The system consists of the following main components:

[1617] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[1618] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[1619] User: An individual who uses a bicycle.

[1620] Emotion engine: A software module for recognizing user emotions and analyzing data.

[1621] Specific operation of the system

[1622] User: Before riding a bicycle, launch the navigation app on the device. Tap the navigation app icon and press the "Start" button to start the GPS.

[1623] Device: When the app is launched, it will enable the GPS function and start acquiring the current location. It will then send that location information to the server.

[1624] Server: Based on the received location information, the server compares it with a database to obtain information on nearby accident-prone areas, and sends the obtained information back to the device.

[1625] Terminal: Based on the returned information, if the user is approaching a nearby accident-prone area, a voice message will be sent saying, "You are approaching an accident-prone area, please be careful."

[1626] Device: Uses the smartphone camera to capture video of the area ahead and temporarily stores it in internal memory. The video is analyzed using a generative AI model to detect obstacles such as pedestrians, bicycles, and cars. The device calculates the distance and relative speed of detected obstacles and manages the obstacle data.

[1627] Device: Operates an emotion engine that recognizes emotions from the user's facial expressions and voice. The emotion engine evaluates the user's stress level, anxiety, and fear. If the user indicates a high stress level, the device automatically adjusts the speed of the bicycle and notifies the user through a voice message saying, "Please relax. We will slow down a bit."

[1628] Terminal: If the terminal detects a rapidly approaching obstacle based on video analysis, it instructs the bicycle's motor control unit to automatically decelerate. It selects a specific deceleration mode depending on the type of obstacle and instructs the user to hold the handlebars with both hands. After confirming that the user follows the instruction and holds the handlebars with both hands, the motor control unit gradually reduces the bicycle's speed.

[1629] Specific examples

[1630] Situation: Approaching an intersection while riding a bicycle

[1631] User: Get on the bike and launch the app.

[1632] Device: Obtains current location using GPS and sends it to the server.

[1633] Server: Obtains information on accident-prone areas and sends it to the device.

[1634] Device: Announces "Be careful before approaching the intersection" with a voice message. The camera captures images of the area ahead and analyzes them with generative AI. Video analysis detects sudden approach of pedestrians.

[1635] Terminal: Sends a creep mode command to the motor control device and instructs the user to hold the steering wheel with both hands. Once it is confirmed that the user is holding the steering wheel, the creep mode is activated and the vehicle gradually decelerates.

[1636] On the device: The emotion engine analyzes the user's emotions and, if it detects high stress levels, instructs them to slow down further. A voice message will say, "Relax, we'll slow down even more."

[1637] Examples of prompt statements

[1638] An example of an input prompt sentence is, "How can a bicycle navigation system use an emotion recognition engine to automatically adjust speed based on the user's emotional state?"

[1639] This system can increase user safety and comfort and significantly reduce the risk of accidents.

[1640] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1641] Step 1:

[1642] User: Before riding a bicycle, launch the navigation app. Tap the navigation app icon and press the "Start" button to start the GPS.

[1643] Input: Launching the app and pressing the "Start" button.

[1644] Specific actions: The navigation app is launched, the GPS function is enabled, and the current location is acquired.

[1645] Output: Current location information.

[1646] Step 2:

[1647] Device: Sends current location information to the server.

[1648] Input: The current location information obtained.

[1649] Specific operation: The location information obtained from the GPS module is sent to the server via mobile data communication.

[1650] Output: Current location information sent to the server.

[1651] Step 3:

[1652] Server: Based on the received location information, the server compares it with a database to obtain information on nearby accident-prone areas.

[1653] Input: Current location information sent from the device.

[1654] What it does: Executes a database query to find information about accident hotspots near the current location.

[1655] Output: Accident hotspot information.

[1656] Step 4:

[1657] Server: Returns the acquired information on accident-prone areas to the terminal.

[1658] Input: Accident black spot information from a database.

[1659] Specific operation: Accident-prone area information is compiled as a text message and sent to the device via mobile data communication.

[1660] Output: Accident hotspot information sent to the device.

[1661] Step 5:

[1662] Terminal: Based on the information received by the terminal, it issues a voice warning to the user, for example, "You are approaching an accident-prone area, so please be careful."

[1663] Input: Accident hotspot information received from the server.

[1664] Specific operation: Converts text information into speech and transmits it to the user through a speaker.

[1665] Output: Audio alert.

[1666] Step 6:

[1667] Device: The device uses a smartphone camera to capture video of the area ahead and temporarily stores it in its internal memory. The device then uses a generative AI model to analyze the video and detect obstacles such as pedestrians, bicycles, and cars.

[1668] Input: Front view captured by a smartphone camera.

[1669] Specific operations: Video capture, saving to internal memory, real-time video analysis and obstacle identification using generative AI.

[1670] Output: Information about detected obstacles (type, distance, relative speed).

[1671] Step 7:

[1672] Terminal: Based on the detected obstacle information, it issues an audio warning to the user.

[1673] Input: Obstacle information analyzed by the generation AI.

[1674] Specific behavior: Generates an appropriate warning message depending on the type of obstacle and notifies the user via audio.

[1675] Output: Audio warning message.

[1676] Step 8:

[1677] Device: Operates an emotion engine that recognizes emotions from the user's facial expressions and voice. The emotion engine evaluates the user's stress level, anxiety, and fear.

[1678] Input: User's facial expression and voice data.

[1679] Specific operation: Analyze and evaluate the user's emotional state using facial expression and voice recognition software.

[1680] Output: User's emotional state data (stress level, anxiety, fear, etc.).

[1681] Step 9:

[1682] Device: If the user exhibits high stress levels, the device automatically adjusts the speed of the bike and provides a voice prompt saying, "Relax, we're going to slow down a bit."

[1683] Input: User emotional state data.

[1684] Specific operation: Instructs the motor control device to adjust the speed, and at the same time generates a voice message and notifies the user through the speaker.

[1685] Output: Adjusted bike speed, voice notification.

[1686] Step 10:

[1687] Terminal: If a rapidly approaching obstacle is detected based on video analysis, the device instructs the bicycle's motor control device to automatically slow down and instructs the user to keep both hands on the handlebars.

[1688] Input: Approaching obstacle based on generative AI analysis.

[1689] Specific actions: Motor control instructions and a voice command to "Hold the steering wheel with both hands."

[1690] Output: Reduced bike speed, voice instructions.

[1691] Step 11:

[1692] User: Follow the instructions and hold the steering wheel with both hands.

[1693] Input: Voice commands.

[1694] Specific action: The user holds the steering wheel with both hands.

[1695] Output: Both hands on the handlebars.

[1696] Step 12:

[1697] Terminal: After confirming that the rider is still holding the handlebars, the motor control unit gradually reduces the speed of the bicycle.

[1698] Input: Confirmation that the user has both hands on the steering wheel.

[1699] Specific operation: The speed is gradually reduced based on instructions from the motor control device.

[1700] Power: Reduced bike speed.

[1701] (Application example 2)

[1702] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1703] In bicycle navigation systems, detecting traffic accident hotspots and automatically slowing down when encountering obstacles are important accident prevention measures, but many systems do not take the user's emotional state into account. As a result, they may not respond appropriately when the user feels strong stress or fear, which could increase the risk of an accident. Furthermore, current bicycle navigation systems do not incorporate real-time emotion recognition and driving style adjustment, making it difficult to ensure sufficient safety and comfort.

[1704] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring location information, means for acquiring video of the road ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time and detecting obstacles, means for recognizing the user's emotions using an emotion recognition engine and adjusting the driving style according to the stress level, means for issuing an audio alert when an accident-prone area or an obstacle is detected, and means for automatically slowing down when approaching an obstacle. This enables appropriate action to be taken in real time when the user is feeling high stress or fear, thereby reducing the risk of an accident and improving the safety and comfort of the user.

[1705] "Means for obtaining location information" refers to devices or software that obtain location information using GPS functions or other location measurement technologies to identify a user's current location.

[1706] "Means for obtaining forward-looking images using a camera" refers to devices or software that use a camera installed in front of a bicycle or autonomous vehicle to obtain video data in real time.

[1707] "Means for detecting accident-prone areas" refers to devices or software that use acquired location information and external databases to detect areas where traffic accidents frequently occur.

[1708] "Means for analyzing video in real time and detecting obstacles" refers to devices or software that use generative AI models or deep learning technology to analyze acquired video data in real time and detect pedestrians, vehicles, and other obstacles.

[1709] "Means for recognizing a user's emotions using an emotion recognition engine" refers to devices or software that use deep learning models or algorithms to analyze data on a user's facial expressions and voice to recognize their emotional state.

[1710] "Means for adjusting driving style according to stress level" refers to devices or software for adjusting the speed or changing the driving mode of a bicycle or autonomous vehicle based on the stress level recognized by the emotion recognition engine.

[1711] The "means for issuing an audio alert" refers to a device or software that issues an audio alert to alert the user based on detected accident-prone areas or obstacle information.

[1712] "Means for automatically slowing down around obstacles" means motor controls and software for automatically reducing vehicle speed based on the location and distance of a detected obstacle.

[1713] MODE FOR CARRYING OUT THE INVENTION

[1714] This invention relates to a navigation system for an autonomous vehicle that recognizes a user's emotional state in real time to optimize the driving style and improve safety and comfort. The system includes a means for acquiring location information, a means for capturing images of the road ahead using a camera, a means for detecting accident-prone areas, a means for analyzing the images in real time to detect obstacles, a means for recognizing the user's emotions using an emotion recognition engine, a means for adjusting the driving style according to the user's stress level, a means for issuing an audio alert, and a means for automatically slowing down the vehicle.

[1715] Hardware and Software Configuration

[1716] Hardware:

[1717] GPS module: Obtains location information.

[1718] Camera: Captures real-time images of the area ahead.

[1719] Motor control unit: Controls the speed of an autonomous vehicle.

[1720] Microphone: Captures the user's voice.

[1721] software:

[1722] Emotion recognition engine: Analyzes emotions from the user's facial expressions and voice.

[1723] Generative AI model: Analyzes images ahead and detects obstacles.

[1724] Traffic information API: Links with external databases to obtain information on areas prone to accidents.

[1725] Program processing

[1726] server

[1727] The server receives the location information and compares it with a database of accident-prone areas. The results are sent back to the device, which then issues an alert about any dangerous areas the user is approaching. The server also analyzes the captured video data with a generative AI model to detect obstacles in real time.

[1728] Terminal

[1729] The device captures images of the road ahead using a camera and temporarily stores them. An emotion recognition engine analyzes the user's facial expressions and voice, monitoring their emotions in real time. Based on their emotional state and the captured images, the device adjusts its driving style (speed and warning level) appropriately.

[1730] User

[1731] The user launches the application and gets into the self-driving vehicle. The system provides voice alerts and adjusts driving according to the user's emotional state, and the user adjusts their driving accordingly.

[1732] Specific examples

[1733] 1. Situation: A user is driving an autonomous vehicle and approaches a congested area.

[1734] 2. Terminal:

[1735] The camera captures images of the road ahead, and a generative AI model analyzes the traffic congestion situation.

[1736] The user's voice is picked up by a microphone and stress levels are measured using an emotion recognition engine.

[1737] 3. Server:

[1738] Based on the acquired location information, traffic congestion information is obtained via a traffic information API.

[1739] 4. Results:

[1740] If the user indicates high stress levels, the vehicle's speed will be automatically reduced and a voice notification will be provided saying, "Relax. We will reduce your speed."

[1741] Prompt Sentence Examples

[1742] "Create an application that uses an emotion recognition model to analyze passenger emotions in real time and adjust the vehicle's driving style based on those emotions. Additionally, use an external traffic information API to obtain traffic congestion information and include guidelines to reduce passenger stress."

[1743] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1744] Step 1:

[1745] The device acquires video of the road ahead using the autonomous vehicle's camera, uses the real-time video data acquired from the camera as input, temporarily stores it in its internal memory, and provides video data for the next processing step as output.

[1746] Step 2:

[1747] The device uses an emotion recognition engine to recognize the user's emotions in real time. It uses audio data acquired from a microphone and facial expression data acquired from a camera as input. The emotion recognition engine analyzes this data to identify the user's emotional state. The output is generated as the user's emotional state data (e.g., stress level, high, medium, low).

[1748] Step 3:

[1749] The device sends the location information it has acquired to the server. It uses the location data from the GPS module as input. It sends the location data to the server and requests information on accident-prone areas. It receives the information on accident-prone areas returned from the server as output.

[1750] Step 4:

[1751] The server compares the location data it receives with a database of accident-prone areas. The location data and the information on accident-prone areas in the database are used as input. The database is compared to extract information on the relevant accident-prone areas and sent to the terminal. The information on the relevant accident-prone areas is provided as output.

[1752] Step 5:

[1753] An audio alert is issued to the user based on the information on accident-prone areas received by the device. The information on accident-prone areas and the previously acquired video data are used as input. An audio alert generation engine is used to warn the user. The audio data that issues the audio alert is generated as output and executed.

[1754] Step 6:

[1755] The video data acquired by the device is analyzed in real time using a generative AI model to detect obstacles. Video data of the front is used as input. The generative AI model is used to recognize objects on the screen and identify the type of obstacle (pedestrian, car, etc.), as well as their relative speed and distance. Information data on the detected obstacle is generated as output and used for the next deceleration process.

[1756] Step 7:

[1757] The device comprehensively assesses the emotional state, obstacle data, and information on accident-prone areas, and sends instructions to the motor controller to adjust the driving style. It also uses the emotional state data, obstacle data, and information on accident-prone areas as inputs. This includes issuing instructions to reduce speed when stress levels are high or obstacles are nearby. It generates deceleration command data that is sent to the motor controller as output.

[1758] Step 8:

[1759] The motor control device receives the deceleration command from the terminal and adjusts the speed of the autonomous vehicle. It uses the deceleration command data as input, adjusts the motor output of the autonomous vehicle according to the deceleration command, and reduces the speed appropriately. It feeds back the actual speed data of the decelerated vehicle as output, and maintains a safe driving style.

[1760] In this way, the steps work in cooperation with each other, thereby making it possible to improve the safety and comfort of the user.

[1761] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1762] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1763] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1764] [Fourth embodiment]

[1765] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1766] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1767] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1768] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1769] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1770] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1771] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1772] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1773] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1774] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1775] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1776] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1777] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1778] ---

[1779] The present invention relates to a bicycle navigation system, and provides a support system for cyclists to ride safely. This system has functions to detect traffic accident hot spots and automatically slow down when encountering obstacles, and can provide specific measures for users to avoid accidents.

[1780] Explanation of program processing

[1781] 1. System Configuration

[1782] The system mainly consists of the following components:

[1783] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[1784] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[1785] User: An individual who uses a bicycle.

[1786] 2. Specific Program Processing

[1787] Location information acquisition and analysis

[1788] User:

[1789] Before getting on your bike, launch the navigation app on your device.

[1790] Device:

[1791] When the system starts up, it enables the GPS function and begins obtaining the current location.

[1792] The acquired current location information is sent to the server.

[1793] server:

[1794] Based on the received location information, the system compares it with a database to obtain information on nearby accident-prone areas.

[1795] The acquired information is returned to the device.

[1796] Device:

[1797] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[1798] Examples:

[1799] - A voice notification will say, "There is an accident-prone area ahead. Please be careful."

[1800] Video analysis and hazard detection

[1801] Device:

[1802] Uses the smartphone camera to capture images of the area ahead.

[1803] The acquired video is analyzed in real time using generative AI.

[1804] Detects the presence of obstacles such as pedestrians, bicycles, and cars.

[1805] Examples:

[1806] - If a pedestrian suddenly approaches ahead, it will be detected through video analysis.

[1807] Auto-slowdown and alert features

[1808] Device:

[1809] If a rapidly approaching obstacle is detected based on video analysis, the system instructs the bicycle's motor control device to automatically slow down.

[1810] Select a specific deceleration mode depending on the type of obstacle.

[1811] Slow down mode for pedestrians.

[1812] Use left-side avoidance mode for other bicycles.

[1813] For automobiles, the alert mode is set by flashing lights.

[1814] User:

[1815] You will be instructed to make sure you keep both hands on the wheel while slowing down.

[1816] Device:

[1817] Only with user confirmation will the motor controller gradually reduce the bike's speed.

[1818] Specific examples

[1819] Situation: Approaching an intersection while riding a bicycle

[1820] User:

[1821] Get on your bike and launch the app.

[1822] Device:

[1823] The current location is obtained using GPS and sent to the server.

[1824] server:

[1825] Information on accident-prone areas is acquired and sent to the terminal.

[1826] Device:

[1827] A voice message will be displayed saying, "Be careful before approaching the intersection."

[1828] A camera captures images of the area ahead and analyzes them using generative AI.

[1829] Video analysis detects sudden approach of pedestrians.

[1830] Device:

[1831] A creep mode command is sent to the motor controller and the user is instructed to keep both hands on the steering wheel.

[1832] Once it is confirmed that you are holding the steering wheel, a creep mode is activated and the vehicle gradually slows down.

[1833] In this way, the system of the present invention helps users to ride bicycles safely and significantly reduces the risk of accidents.

[1834] The processing flow will be explained below.

[1835] ---

[1836] Step 1:

[1837] User: Turns on the smartphone and launches the bicycle navigation app.

[1838] Device: When the app is launched, enable GPS and camera functions.

[1839] Step 2:

[1840] Device: Uses GPS to obtain current location information.

[1841] Device: Sends the acquired current location information to the server.

[1842] Step 3:

[1843] Server: Based on the received location information, the server compares it with a database of accident-prone areas to obtain information on nearby accident-prone locations.

[1844] Server: Returns the accident-prone location information obtained as a result of the matching to the terminal.

[1845] Step 4:

[1846] Terminal: Analyzes the accident-prone location information returned from the server and determines whether the current location is close to an accident-prone location.

[1847] Terminal: When approaching a nearby accident hotspot, the device will issue a voice alert to the user.

[1848] Step 5:

[1849] Device: The smartphone camera captures real-time images of the road ahead.

[1850] Device: The acquired video is temporarily stored in the internal memory, and analysis begins using the generation AI.

[1851] Step 6:

[1852] Device: Analyzes video using generative AI and detects obstacles such as pedestrians, bicycles, and cars.

[1853] Terminal: Calculates the distance and relative speed of detected obstacles.

[1854] Step 7:

[1855] Terminal: Based on the video analysis results, evaluate whether there is an approaching obstacle.

[1856] Terminal: Prepares to take specific action (slow mode, drive left mode, flash lights) if assessed as dangerous.

[1857] Step 8:

[1858] Device: Select slow-walk mode for pedestrians, drive left mode for cyclists, and flashing light alert mode for cars.

[1859] Terminal: Tells the motor controller to slow down or flash lights.

[1860] Step 9:

[1861] Device: A sensor checks whether the user has both hands on the steering wheel.

[1862] Terminal: If the driver is confirmed to be holding the steering wheel, the motor control unit will automatically begin a gradual deceleration process.

[1863] Step 10:

[1864] Terminal: Ensures that the bicycle's speed has been reduced to a safe level for obstacles.

[1865] User: Keep both hands on the steering wheel while slowing down and ensure safe driving.

[1866] This allows the system to automatically take appropriate measures before the user gets into a dangerous situation, supporting safe cycling.

[1867] Example 1

[1868] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1869] Systems designed to help cyclists ride safely face challenges due to their inadequate ability to detect frequent traffic accidents and obstacles. In particular, they lack the ability to obtain real-time location information, provide information about accident-prone areas and obstacles, and automatically adjust speed. They also need a way to avoid unnecessary slowdowns by obtaining user confirmation.

[1870] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1871] In this invention, the server includes means for acquiring location information, means for acquiring video of the area ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time and detecting obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle speed toward an obstacle, and means for executing the speed reduction when the user agrees to the speed reduction. This makes it possible to detect accident-prone areas and obstacles and provide appropriate alerts and deceleration to ensure safe cycling for cyclists.

[1872] "Means of obtaining location information" refers to a function that measures the current location using a smartphone, GPS device, etc. and obtains the data.

[1873] "Means for obtaining images of the front using a camera" refers to a function that collects images of the front of the bicycle in real time using a smartphone, camera device, etc.

[1874] "Means for detecting accident-prone areas" is a function that identifies areas where traffic accidents have occurred frequently in the past based on location information and extracts the necessary data.

[1875] "Means for analyzing and detecting obstacles in real time" refers to a function that instantly analyzes the images captured by the camera and identifies obstacles such as pedestrians, bicycles, and automobiles.

[1876] "Means for issuing audio alerts" is a function that issues an audio warning to the user when an accident-prone area or obstacle is detected.

[1877] "Means for automatically reducing the speed of the bicycle" is a function that automatically controls the speed of the bicycle in response to detected obstacles, ensuring safety.

[1878] The "means for slowing down when the user agrees to slow down" is a function that notifies the user of the need to slow down and reduces the speed only if the user complies with the instruction.

[1879] "Generative AI" is an artificial intelligence that performs real-time image analysis based on video data acquired by a camera to determine the type and location of obstacles.

[1880] This invention provides a bicycle navigation system to support cyclists in riding safely. In particular, it has the function of detecting traffic accident hotspots and automatically slowing down the vehicle when encountering obstacles, aiming to significantly reduce the risk of accidents. This system consists of three main elements: a server, a terminal, and a user.

[1881] System Components

[1882] server:

[1883] The server maintains a database of areas prone to traffic accidents and provides location information to the terminal. The server receives the location information and returns the information on areas prone to traffic accidents to the terminal.

[1884] Device:

[1885] The terminal is a mobile device such as a smartphone, and uses GPS and camera functions. When the system is started, the terminal begins acquiring its current location and acquires location information and images of the area ahead in real time. The acquired images are then analyzed using a generative AI model to detect obstacles and issue instructions for automatic deceleration.

[1886] User:

[1887] The user is an individual who rides a bicycle. Before riding, the user launches the navigation app and follows the alerts and instructions from the device.

[1888] Hardware and software used

[1889] GPS Module:

[1890] It is built into the device and acquires the current location.

[1891] camera:

[1892] It is built into the terminal and captures images of what is in front of it.

[1893] Generative AI models:

[1894] It runs on the device, analyzes images captured by the camera in real time, and detects obstacles.

[1895] Motor Controller:

[1896] A device for automatically adjusting the speed of a bicycle.

[1897] Specific Examples of the Invention

[1898] Situation: Approaching an intersection while riding a bicycle

[1899] User:

[1900] The user gets on the bicycle and starts the navigation app on the device. The user drives the bicycle following the instructions of the app.

[1901] Device:

[1902] When the navigation app is launched, it uses the GPS module to obtain the current location and sends that information to the server. Based on the information on accident-prone areas returned from the server, it notifies the user by voice, "There is an accident-prone area ahead. Please be careful." It also uses a camera to obtain images of the area ahead and analyzes them in real time using a generative AI model. Based on the analysis results, if the device detects a pedestrian approaching quickly, it instructs the motor control device to enter slow-down mode.

[1903] User:

[1904] When instructed to slow down, the user ensures they keep both hands on the steering wheel, then follows further instructions from the device to drive safely.

[1905] In this way, the system of the present invention combines real-time location information with video analytics to provide specific alerts and slow-down instructions to help cyclists avoid accidents, thereby increasing cyclist safety and significantly reducing the risk of accidents.

[1906] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1907] Step 1:

[1908] The user launches a navigation app. The input is the user tapping the navigation app on their smartphone. The output is the navigation app launching.

[1909] Specific operation:

[1910] The user taps the navigation app icon on the smartphone home screen.

[1911] Step 2:

[1912] The device uses the GPS module to obtain its current location. The input is satellite data received by the device's GPS module. The output is the current location data.

[1913] Specific operation:

[1914] The smartphone obtains the current location information from satellites and sends that information to the location management module within the app.

[1915] Step 3:

[1916] The current location information acquired by the device is sent to the server. The input is the current location data. The output is the location information data sent to the server.

[1917] Specific operation:

[1918] The GPS information is sent to the server using the terminal's network module.

[1919] Step 4:

[1920] Based on the location information received by the server, information on accident-prone areas is retrieved from the database. The input is the current location data sent from the device. The output is information on accident-prone areas.

[1921] Specific operation:

[1922] The server queries the database and retrieves the results.

[1923] Step 5:

[1924] The server returns the information about accident-prone areas it has acquired to the terminal. The input is the information about accident-prone areas acquired by the server. The output is the information about accident-prone areas that is sent to the terminal.

[1925] Specific operation:

[1926] The server obtains information about areas prone to accidents from the database and transmits it to the terminal via the network.

[1927] Step 6:

[1928] The terminal issues a voice alert to the user based on the returned information on accident-prone areas. The input is the accident-prone area information sent from the server. The output is the issuance of a voice alert.

[1929] Specific operation:

[1930] The device will notify you with a voice message saying, "There is an accident-prone area ahead. Please be careful."

[1931] Step 7:

[1932] The device uses the smartphone camera to capture images of the front. The input is the camera image data. The output is the captured front image data.

[1933] Specific operation:

[1934] The camera continuously captures images of the area ahead and stores the data in the device's memory.

[1935] Step 8:

[1936] The image captured by the device is input into a generative AI model and analyzed in real time. The input is the image data from the camera. The output is the analyzed obstacle data.

[1937] Specific operation:

[1938] The device inputs the video data into a generative AI model, which then analyzes the data to identify the type and location of obstacles.

[1939] Step 9:

[1940] The device detects obstacles based on the analysis results and instructs the bicycle's motor control device to slow down. The input is the analysis result from the generative AI model. The output is a command to the motor control device to slow down.

[1941] Specific operation:

[1942] If a pedestrian is detected, the device will issue an alert saying "Pedestrian detected, bicycle speed set to slow mode" and send a command to the motor control device to slow down.

[1943] Step 10:

[1944] Verify that the user keeps both hands on the steering wheel when instructed to slow down. The input is an audio alert. The output is the user gripping the steering wheel.

[1945] Specific operation:

[1946] The user follows the instruction "Please hold the steering wheel with both hands" and performs an operation to confirm that he or she is holding the steering wheel.

[1947] Step 11:

[1948] The terminal gradually slows down after the user confirms. The input is the user's confirmation. The output is a decrease in the bicycle's speed.

[1949] Specific operation:

[1950] The device detects the user's confirmation and begins to safely slow down, gradually reducing the bicycle's speed.

[1951] The bicycle navigation system of the present invention is implemented through the above steps. The specific operations and data inputs and outputs at each step ensure that the system functions accurately and safely.

[1952] (Application example 1)

[1953] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1954] There is a need to reduce the risk of traffic accidents while riding a bicycle and ensure safety. However, current bicycle navigation systems and automated driving assistance systems do not fully fulfill their functions of detecting accident-prone areas in real time or automatically detecting and slowing down obstacles. Another problem is the lack of user awareness in situations where the risk of an accident is high. In these circumstances, it is necessary to develop a reliable accident avoidance and deceleration system.

[1955] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1956] In this invention, the server includes means for acquiring location information, means for acquiring video of the road ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time to detect obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle when approaching an obstacle, means for analyzing the camera video with a generative AI model to detect objects, means for confirming that the handlebars of the bicycle are being held, and means for controlling the bicycle's slow-down mode. This makes it possible to detect traffic accident risks while riding a bicycle in real time and to appropriately slow down the bicycle.

[1957] "Location information" is data that indicates a specific location or coordinates using technology such as GPS.

[1958] A "camera" is a device that receives light and records and displays images or videos.

[1959] An "accident hotspot" is a location in a particular area or section of road where traffic accidents occur frequently.

[1960] A "generative AI model" is a computer program that has been trained using machine learning algorithms and is used to automate specific tasks.

[1961] An "obstacle" is any object that impedes the progress of a bicycle or vehicle, such as a pedestrian, a car, or another bicycle.

[1962] "Audio alert" is a function that transmits a warning message by voice from an audio output device.

[1963] "Automatic deceleration" is a function that automatically reduces the speed of a bicycle or vehicle when the system detects an obstacle.

[1964] A "server" is a computer system that provides services to other devices and systems over a network.

[1965] "Crawl mode" is a driving mode in which a vehicle or bicycle is controlled to a speed significantly slower than normal.

[1966] "Video analysis" is a technology in which a computer processes images and videos captured by a camera to extract and recognize specific information.

[1967] "Confirm that you are holding the handlebars" is a function that uses sensors to detect how the user is holding the handlebars of a bicycle or vehicle.

[1968] This invention relates to a driving assistance system for bicycles or autonomous vehicles. The system acquires location information, captures images of the road ahead with a camera, and analyzes this information in real time to detect accident-prone areas and obstacles, and issues audio alerts and automatically slows down the vehicle. A specific implementation method for this system will be described below.

[1969] 1. System Configuration

[1970] The system includes the following main components:

[1971] server

[1972] Devices (smartphones and in-car cameras)

[1973] User (bicyclist or vehicle driver)

[1974] 2. System Operation

[1975] Location information acquisition and analysis

[1976] User: Before getting on a bike or vehicle, launch the navigation app on your device.

[1977] Device:

[1978] Enable the GPS function to obtain your current location.

[1979] The acquired location information is sent to the server.

[1980] server:

[1981] The received location information is compared with a database to obtain information on nearby accident-prone areas.

[1982] The acquired information is returned to the device.

[1983] Device:

[1984] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[1985] Example: A voice message will be sent saying, "There is an accident-prone area ahead. Please be careful."

[1986] Video analysis and hazard detection

[1987] Device:

[1988] Use an onboard camera to capture images of the vehicle ahead.

[1989] The captured video is analyzed in real time using a generative AI model.

[1990] Detects the presence of obstacles such as pedestrians, bicycles, and vehicles.

[1991] Example: If a pedestrian suddenly approaches ahead, this is detected through video analysis.

[1992] Example prompt: "Analyze the video in front of the vehicle in real time to detect pedestrians and other obstacles."

[1993] Auto-slowdown and alert features

[1994] Device:

[1995] If the system detects a rapidly approaching obstacle based on video analysis, it will instruct the motor control device of the bicycle or vehicle to automatically slow down.

[1996] Select a specific deceleration mode depending on the type of obstacle.

[1997] Slow down mode for pedestrians.

[1998] Use left-side avoidance mode for other bicycles.

[1999] For automobiles, the alert mode is set by flashing lights.

[2000] User:

[2001] You will be instructed to keep both hands on the steering wheel while slowing down.

[2002] Device:

[2003] Only when the motor controller determines that the user is still holding the handlebars will it gradually reduce the speed of the bicycle or vehicle.

[2004] Hardware and Software

[2005] Hardware used:

[2006] GPS Modules

[2007] In-car camera

[2008] Software used:

[2009] OpenCV (camera image analysis)

[2010] Generative AI model (obstacle detection)

[2011] Specific examples

[2012] Situation: Vehicle approaches a busy intersection

[2013] 1. The user launches the app and enables the GPS function.

[2014] 2. The device sends its current location to the server and obtains data on accident-prone areas.

[2015] 3. The server sends the acquired data back to the device, and the device issues a voice notification saying, "You are approaching an accident-prone area. Please be careful."

[2016] 4. The onboard camera captures images of the road ahead and performs real-time analysis using a generative AI model.

[2017] 5. If the device detects an obstacle such as a pedestrian or another vehicle, it will automatically switch to creep mode and safely slow down.

[2018] In this way, this system helps users to ride bicycles and vehicles safely, significantly reducing the risk of accidents.

[2019] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2020] Step 1:

[2021] User: Before getting on a bike or vehicle, launch the navigation app on your device.

[2022] Input: None

[2023] Output: App launch

[2024] What happens: A user taps to launch an app on their smartphone or vehicle's infotainment system.

[2025] Step 2:

[2026] Device: Enable GPS to obtain your current location.

[2027] Input: None

[2028] Output: Current location information (latitude and longitude)

[2029] Specific operation: The device obtains current location data using the built-in GPS module.

[2030] Step 3:

[2031] Device: Sends the acquired location information to the server.

[2032] Input: Current location information (output of step 2)

[2033] Output: Sending completion message or data to the server

[2034] What it does: Your device sends its current location data to a server via an internet connection.

[2035] Step 4:

[2036] Server: The received location information is compared with a database to obtain information on nearby accident-prone areas.

[2037] Input: Current location

[2038] Output: Accident hotspot information

[2039] Specific operation: The server accesses the database and queries for data on accident-prone areas that match the current location information.

[2040] Step 5:

[2041] Server: Returns the acquired information on accident-prone areas to the terminal.

[2042] Input: Accident hotspot information

[2043] Output: Accident-prone areas information (response data sent to the device)

[2044] Specific operations: The server constructs the search results and sends the data back to the device.

[2045] Step 6:

[2046] Terminal: Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[2047] Input: Accident hotspot information

[2048] Output: Audio alert

[2049] Specific operation: The device uses the built-in speaker to notify the user by voice, "There is an accident-prone area ahead. Please be careful."

[2050] Step 7:

[2051] Terminal: Uses the onboard camera to capture images of the area ahead.

[2052] Input: None

[2053] Output: Video data (live feed)

[2054] Specific operation: The camera captures images of the area in front of the vehicle and acquires real-time video data.

[2055] Step 8:

[2056] Terminal: The captured video is analyzed in real time using a generative AI model.

[2057] Input: Video data

[2058] Output: Obstacle detection results (pedestrians, bicycles, vehicles, etc.)

[2059] Specific operation: The acquired video data is input into a generative AI model, which then detects obstacles such as pedestrians and vehicles.

[2060] Step 9:

[2061] Terminal: Based on video analysis, if a rapidly approaching obstacle is detected, the device instructs the motor control device of the bicycle or vehicle to automatically slow down.

[2062] Input: Obstacle detection result

[2063] Output: Deceleration instruction

[2064] Specific operation: Sends a command to the motor control device to reduce speed depending on the danger level of the obstacle.

[2065] Step 10:

[2066] Device: Check that the user is holding the wheel, and only then will creep mode be activated.

[2067] Input: User's hand position data (steering wheel sensor)

[2068] Output: Crawl mode execution command

[2069] Specific operation: A sensor installed on the handle detects the position of the user's hands and, based on that information, instructs the motor control device to enter slow-motion mode.

[2070] Sample prompt: "Analyze the video in front of the vehicle in real time to detect pedestrians and other obstacles."

[2071] Through these steps, a system that reduces the risk of accidents while riding a bicycle or vehicle and supports safe driving will be efficiently realized.

[2072] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2073] ---

[2074] This invention combines an emotion recognition engine with a bicycle navigation system to improve the safety and comfort of cyclists. In addition to detecting areas prone to traffic accidents and automatically slowing down when encountering obstacles, this system can recognize the user's emotions in real time and take appropriate action based on that data.

[2075] Explanation of program processing

[2076] 1. System Configuration

[2077] The system mainly consists of the following components:

[2078] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[2079] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[2080] User: An individual who uses a bicycle.

[2081] Emotion engine: A software module for recognizing user emotions and analyzing data.

[2082] 2. Specific Program Processing

[2083] Location information acquisition and analysis

[2084] User:

[2085] Before getting on your bike, launch the navigation app on your device.

[2086] Device:

[2087] When the system starts up, it enables the GPS function and begins obtaining the current location.

[2088] The acquired current location information is sent to the server.

[2089] server:

[2090] Based on the received location information, the system compares it with a database to obtain information on nearby accident-prone areas.

[2091] The acquired information is returned to the device.

[2092] Device:

[2093] Based on the returned information, if the user is approaching a nearby accident-prone area, an audio alert will be issued.

[2094] Video analysis and hazard detection

[2095] Device:

[2096] Uses the smartphone camera to capture images of the area ahead.

[2097] The acquired video is temporarily stored in internal memory, and analysis begins using the generating AI.

[2098] Device:

[2099] Generative AI analyzes the image and detects obstacles such as pedestrians, bicycles, and cars.

[2100] Calculates the distance and relative speed of detected obstacles.

[2101] Emotion Recognition and Its Applications

[2102] Device:

[2103] An emotion engine is activated that recognizes emotions from the user's facial expressions, voice, etc.

[2104] An emotion engine assesses the user's stress level, anxiety, and fear.

[2105] Device:

[2106] If the user exhibits high stress levels, the bike speed is automatically adjusted.

[2107] If the user is feeling anxious or scared, the warning alert will be intensified.

[2108] Examples:

[2109] Situation 1: If the user is feeling stressed, the system will detect this through its emotion engine and automatically slow down the speed.

[2110] A voice message will say, "Relax, we'll slow down a bit."

[2111] Situation 2: If the user is feeling fear, the system will issue a detailed warning alert.

[2112] A voice message will be displayed saying, "There is an imminent danger ahead, please slow down immediately."

[2113] Auto-slowdown and alert features

[2114] Device:

[2115] If a rapidly approaching obstacle is detected based on video analysis, the system instructs the bicycle's motor control device to automatically slow down.

[2116] Select a specific deceleration mode depending on the type of obstacle.

[2117] Slow down mode for pedestrians.

[2118] Left-hand driving mode for other bicycles.

[2119] For automobiles, the alert mode is set by flashing lights.

[2120] User:

[2121] You will be instructed to make sure you keep both hands on the wheel while slowing down.

[2122] Device:

[2123] Only with user confirmation will the motor controller gradually reduce the bike's speed.

[2124] Specific examples

[2125] Situation: Approaching an intersection while riding a bicycle

[2126] User:

[2127] Get on your bike and launch the app.

[2128] Device:

[2129] The current location is obtained using GPS and sent to the server.

[2130] server:

[2131] Information on accident-prone areas is acquired and sent to the terminal.

[2132] Device:

[2133] A voice message will be displayed saying, "Be careful before approaching the intersection."

[2134] A camera captures images of the area ahead and analyzes them using generative AI.

[2135] Video analysis detects sudden approach of pedestrians.

[2136] Device:

[2137] A creep mode command is sent to the motor controller and the user is instructed to keep both hands on the steering wheel.

[2138] Once it is confirmed that you are holding the steering wheel, a creep mode is activated and the vehicle gradually slows down.

[2139] Device:

[2140] The emotion engine analyzes the user's emotions and if it detects high stress levels, it will instruct them to slow down further.

[2141] A voice will inform you, "Relax, we'll slow down further."

[2142] In this way, the system of the present invention increases the safety and comfort of the user and significantly reduces the risk of accidents.

[2143] The processing flow will be explained below.

[2144] ---

[2145] Step 1:

[2146] User: Turns on the smartphone and launches the bicycle navigation app.

[2147] Device: When the app is launched, enable GPS and camera functions.

[2148] Step 2:

[2149] Device: Uses GPS to obtain current location information.

[2150] Device: Sends the acquired current location information to the server.

[2151] Step 3:

[2152] Server: Based on the received location information, the server compares it with a database of accident-prone areas to obtain information on nearby accident-prone locations.

[2153] Server: Returns the accident-prone location information obtained as a result of the matching to the terminal.

[2154] Step 4:

[2155] Terminal: Analyzes the accident-prone location information returned from the server and determines whether the current location is close to an accident-prone location.

[2156] Terminal: When approaching a nearby accident hotspot, the device will issue a voice alert to the user.

[2157] Step 5:

[2158] Device: The smartphone camera captures real-time images of the road ahead.

[2159] Device: The acquired video is temporarily stored in the internal memory, and analysis begins using the generation AI.

[2160] Step 6:

[2161] Device: Analyzes video using generative AI and detects obstacles such as pedestrians, bicycles, and cars.

[2162] Terminal: Calculates the distance and relative speed of detected obstacles.

[2163] Step 7:

[2164] Terminal: Based on the video analysis results, evaluate whether there is an approaching obstacle.

[2165] Terminal: Prepares to take specific action (slow mode, drive left mode, flash lights) if assessed as dangerous.

[2166] Step 8:

[2167] Device: Select slow-walk mode for pedestrians, drive left mode for cyclists, and flashing light alert mode for cars.

[2168] Terminal: Tells the motor controller to slow down or flash lights.

[2169] Step 9:

[2170] Device: Runs the emotion engine, analyzes the user's facial expressions and voice, and recognizes their emotional state in real time.

[2171] Device: When the user's stress level, anxiety, or fear is detected, the device analyzes the information and determines the necessary response.

[2172] Step 10:

[2173] Device: If it detects high stress levels in the user, it will issue instructions to automatically slow down the bike.

[2174] Device: A voice prompt tells the user to "Relax, we'll slow down a bit."

[2175] Step 11:

[2176] Device: If the emotion engine detects that the user is anxious or scared, it will increase the intensity of the warning alert.

[2177] Device: A voice message will be displayed saying "There is an imminent danger ahead, please slow down immediately."

[2178] Step 12:

[2179] Terminal: The motor control unit follows the instructions and gradually reduces the speed of the bicycle.

[2180] User: Keep both hands on the steering wheel while slowing down and ensure safe driving.

[2181] In this way, the system monitors the user's emotional state in real time and takes appropriate action as needed, supporting safe cycling.

[2182] Example 2

[2183] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2184] While conventional bicycle navigation systems provide basic risk avoidance based on location information and obstacle detection, they are unable to take the user's emotional state into account and therefore are unable to fully enhance the user's safety and comfort. In particular, when the user is experiencing high levels of stress, impatience, or fear, it is difficult for the system to respond appropriately, potentially increasing the risk of an accident.

[2185] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2186] In this invention, the server includes means for acquiring location information, means for acquiring video of the area ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time to detect obstacles, means for issuing an audio alert when an accident-prone area or obstacle is detected, means for automatically slowing the bicycle when approaching an obstacle, and means for recognizing the user's emotions and adjusting the bicycle speed and warning mode based on the user's emotional state. This allows for appropriate responses according to the user's emotional state, improving the user's safety and comfort.

[2187] A "means for obtaining location information" is a device or system that has the ability to determine current location using GPS or other location information services.

[2188] "Means for obtaining forward images using a camera" refers to a device or system that has the function of capturing and recording images of the area in front of the bicycle using a smartphone or dedicated camera.

[2189] The "means for detecting accident-prone areas" is a system that has the function of identifying areas where traffic accidents are likely to occur by comparing location information with a pre-set database.

[2190] "Means for analyzing captured video in real time and detecting obstacles" refers to a system that uses video processing technology and generative AI models to recognize obstacles such as pedestrians, bicycles, and cars within the video.

[2191] The "means for issuing audio alerts" refers to a system that has the function of issuing audio messages to alert the user based on detected accident hotspots and obstacles.

[2192] "Means for automatically slowing down the bicycle speed" refers to a system that operates the motor control device to slow down the bicycle speed depending on the type and distance of an obstacle detected.

[2193] "Means for recognizing the user's emotions and adjusting the bicycle speed and warning mode based on the emotional state" refers to a system that uses facial expression and voice recognition technology to evaluate the user's emotional state and automatically adjusts speed control and warning methods according to that state.

[2194] This invention improves the safety and comfort of cyclists by combining an emotion recognition engine with a bicycle navigation system that acquires location information, captures camera footage, detects accident hotspots, detects obstacles in real time, issues audio alerts, automatically slows down, and adjusts speed and warning modes based on the user's emotional state.

[2195] The system consists of the following main components:

[2196] Server: Maintains a database of areas prone to traffic accidents and provides location information to devices.

[2197] Device: A mobile device such as a smartphone, using the GPS and camera functions.

[2198] User: An individual who uses a bicycle.

[2199] Emotion engine: A software module for recognizing user emotions and analyzing data.

[2200] Specific operation of the system

[2201] User: Before riding a bicycle, launch the navigation app on the device. Tap the navigation app icon and press the "Start" button to start the GPS.

[2202] Device: When the app is launched, it will enable the GPS function and start acquiring the current location. It will then send that location information to the server.

[2203] Server: Based on the received location information, the server compares it with a database to obtain information on nearby accident-prone areas, and sends the obtained information back to the device.

[2204] Terminal: Based on the returned information, if the user is approaching a nearby accident-prone area, a voice message will be sent saying, "You are approaching an accident-prone area, please be careful."

[2205] Device: Uses the smartphone camera to capture video of the area ahead and temporarily stores it in internal memory. The video is analyzed using a generative AI model to detect obstacles such as pedestrians, bicycles, and cars. The device calculates the distance and relative speed of detected obstacles and manages the obstacle data.

[2206] Device: Operates an emotion engine that recognizes emotions from the user's facial expressions and voice. The emotion engine evaluates the user's stress level, anxiety, and fear. If the user indicates a high stress level, the device automatically adjusts the speed of the bicycle and notifies the user through a voice message saying, "Please relax. We will slow down a bit."

[2207] Terminal: If the terminal detects a rapidly approaching obstacle based on video analysis, it instructs the bicycle's motor control unit to automatically decelerate. It selects a specific deceleration mode depending on the type of obstacle and instructs the user to hold the handlebars with both hands. After confirming that the user follows the instruction and holds the handlebars with both hands, the motor control unit gradually reduces the bicycle's speed.

[2208] Specific examples

[2209] Situation: Approaching an intersection while riding a bicycle

[2210] User: Get on the bike and launch the app.

[2211] Device: Obtains current location using GPS and sends it to the server.

[2212] Server: Obtains information on accident-prone areas and sends it to the device.

[2213] Device: Announces "Be careful before approaching the intersection" with a voice message. The camera captures images of the area ahead and analyzes them with generative AI. Video analysis detects sudden approach of pedestrians.

[2214] Terminal: Sends a creep mode command to the motor control device and instructs the user to hold the steering wheel with both hands. Once it is confirmed that the user is holding the steering wheel, the creep mode is activated and the vehicle gradually decelerates.

[2215] On the device: The emotion engine analyzes the user's emotions and, if it detects high stress levels, instructs them to slow down further. A voice message will say, "Relax, we'll slow down even more."

[2216] Examples of prompt statements

[2217] An example of an input prompt sentence is, "How can a bicycle navigation system use an emotion recognition engine to automatically adjust speed based on the user's emotional state?"

[2218] This system can increase user safety and comfort and significantly reduce the risk of accidents.

[2219] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2220] Step 1:

[2221] User: Before riding a bicycle, launch the navigation app. Tap the navigation app icon and press the "Start" button to start the GPS.

[2222] Input: Launching the app and pressing the "Start" button.

[2223] Specific actions: The navigation app is launched, the GPS function is enabled, and the current location is acquired.

[2224] Output: Current location information.

[2225] Step 2:

[2226] Device: Sends current location information to the server.

[2227] Input: The current location information obtained.

[2228] Specific operation: The location information obtained from the GPS module is sent to the server via mobile data communication.

[2229] Output: Current location information sent to the server.

[2230] Step 3:

[2231] Server: Based on the received location information, the server compares it with a database to obtain information on nearby accident-prone areas.

[2232] Input: Current location information sent from the device.

[2233] What it does: Executes a database query to find information about accident hotspots near the current location.

[2234] Output: Accident hotspot information.

[2235] Step 4:

[2236] Server: Returns the acquired information on accident-prone areas to the terminal.

[2237] Input: Accident black spot information from a database.

[2238] Specific operation: Accident-prone area information is compiled as a text message and sent to the device via mobile data communication.

[2239] Output: Accident hotspot information sent to the device.

[2240] Step 5:

[2241] Terminal: Based on the information received by the terminal, it issues a voice warning to the user, for example, "You are approaching an accident-prone area, so please be careful."

[2242] Input: Accident hotspot information received from the server.

[2243] Specific operation: Converts text information into speech and transmits it to the user through a speaker.

[2244] Output: Audio alert.

[2245] Step 6:

[2246] Device: The device uses a smartphone camera to capture video of the area ahead and temporarily stores it in its internal memory. The device then uses a generative AI model to analyze the video and detect obstacles such as pedestrians, bicycles, and cars.

[2247] Input: Front view captured by a smartphone camera.

[2248] Specific operations: Video capture, saving to internal memory, real-time video analysis and obstacle identification using generative AI.

[2249] Output: Information about detected obstacles (type, distance, relative speed).

[2250] Step 7:

[2251] Terminal: Based on the detected obstacle information, it issues an audio warning to the user.

[2252] Input: Obstacle information analyzed by the generation AI.

[2253] Specific behavior: Generates an appropriate warning message depending on the type of obstacle and notifies the user via audio.

[2254] Output: Audio warning message.

[2255] Step 8:

[2256] Device: Operates an emotion engine that recognizes emotions from the user's facial expressions and voice. The emotion engine evaluates the user's stress level, anxiety, and fear.

[2257] Input: User's facial expression and voice data.

[2258] Specific operation: Analyze and evaluate the user's emotional state using facial expression and voice recognition software.

[2259] Output: User's emotional state data (stress level, anxiety, fear, etc.).

[2260] Step 9:

[2261] Device: If the user exhibits high stress levels, the device automatically adjusts the speed of the bike and provides a voice prompt saying, "Relax, we're going to slow down a bit."

[2262] Input: User emotional state data.

[2263] Specific operation: Instructs the motor control device to adjust the speed, and at the same time generates a voice message and notifies the user through the speaker.

[2264] Output: Adjusted bike speed, voice notification.

[2265] Step 10:

[2266] Terminal: If a rapidly approaching obstacle is detected based on video analysis, the device instructs the bicycle's motor control device to automatically slow down and instructs the user to keep both hands on the handlebars.

[2267] Input: Approaching obstacle based on generative AI analysis.

[2268] Specific actions: Motor control instructions and a voice command to "Hold the steering wheel with both hands."

[2269] Output: Reduced bike speed, voice instructions.

[2270] Step 11:

[2271] User: Follow the instructions and hold the steering wheel with both hands.

[2272] Input: Voice commands.

[2273] Specific action: The user holds the steering wheel with both hands.

[2274] Output: Both hands on the handlebars.

[2275] Step 12:

[2276] Terminal: After confirming that the rider is still holding the handlebars, the motor control unit gradually reduces the speed of the bicycle.

[2277] Input: Confirmation that the user has both hands on the steering wheel.

[2278] Specific operation: The speed is gradually reduced based on instructions from the motor control device.

[2279] Power: Reduced bike speed.

[2280] (Application example 2)

[2281] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2282] In bicycle navigation systems, detecting traffic accident hotspots and automatically slowing down when encountering obstacles are important accident prevention measures, but many systems do not take the user's emotional state into account. As a result, they may not respond appropriately when the user feels strong stress or fear, which could increase the risk of an accident. Furthermore, current bicycle navigation systems do not incorporate real-time emotion recognition and driving style adjustment, making it difficult to ensure sufficient safety and comfort.

[2283] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring location information, means for acquiring video of the road ahead using a camera, means for detecting accident-prone areas using the acquired location information, means for analyzing the acquired video in real time and detecting obstacles, means for recognizing the user's emotions using an emotion recognition engine and adjusting the driving style according to the stress level, means for issuing an audio alert when an accident-prone area or an obstacle is detected, and means for automatically slowing down when approaching an obstacle. This enables appropriate action to be taken in real time when the user is feeling high stress or fear, thereby reducing the risk of an accident and improving the safety and comfort of the user.

[2284] "Means for obtaining location information" refers to devices or software that obtain location information using GPS functions or other location measurement technologies to identify a user's current location.

[2285] "Means for obtaining forward-looking images using a camera" refers to devices or software that use a camera installed in front of a bicycle or autonomous vehicle to obtain video data in real time.

[2286] "Means for detecting accident-prone areas" refers to devices or software that use acquired location information and external databases to detect areas where traffic accidents frequently occur.

[2287] "Means for analyzing video in real time and detecting obstacles" refers to devices or software that use generative AI models or deep learning technology to analyze acquired video data in real time and detect pedestrians, vehicles, and other obstacles.

[2288] "Means for recognizing a user's emotions using an emotion recognition engine" refers to devices or software that use deep learning models or algorithms to analyze data on a user's facial expressions and voice to recognize their emotional state.

[2289] "Means for adjusting driving style according to stress level" refers to devices or software for adjusting the speed or changing the driving mode of a bicycle or autonomous vehicle based on the stress level recognized by the emotion recognition engine.

[2290] The "means for issuing an audio alert" refers to a device or software that issues an audio alert to alert the user based on detected accident-prone areas or obstacle information.

[2291] "Means for automatically slowing down around obstacles" means motor controls and software for automatically reducing vehicle speed based on the location and distance of a detected obstacle.

[2292] MODE FOR CARRYING OUT THE INVENTION

[2293] This invention relates to a navigation system for an autonomous vehicle that recognizes a user's emotional state in real time to optimize the driving style and improve safety and comfort. The system includes a means for acquiring location information, a means for capturing images of the road ahead using a camera, a means for detecting accident-prone areas, a means for analyzing the images in real time to detect obstacles, a means for recognizing the user's emotions using an emotion recognition engine, a means for adjusting the driving style according to the user's stress level, a means for issuing an audio alert, and a means for automatically slowing down the vehicle.

[2294] Hardware and Software Configuration

[2295] Hardware:

[2296] GPS module: Obtains location information.

[2297] Camera: Captures real-time images of the area ahead.

[2298] Motor control unit: Controls the speed of an autonomous vehicle.

[2299] Microphone: Captures the user's voice.

[2300] software:

[2301] Emotion recognition engine: Analyzes emotions from the user's facial expressions and voice.

[2302] Generative AI model: Analyzes images ahead and detects obstacles.

[2303] Traffic information API: Links with external databases to obtain information on areas prone to accidents.

[2304] Program processing

[2305] server

[2306] The server receives the location information and compares it with a database of accident-prone areas. The results are sent back to the device, which then issues an alert about any dangerous areas the user is approaching. The server also analyzes the captured video data with a generative AI model to detect obstacles in real time.

[2307] Terminal

[2308] The device captures images of the road ahead using a camera and temporarily stores them. An emotion recognition engine analyzes the user's facial expressions and voice, monitoring their emotions in real time. Based on their emotional state and the captured images, the device adjusts its driving style (speed and warning level) appropriately.

[2309] User

[2310] The user launches the application and gets into the self-driving vehicle. The system provides voice alerts and adjusts driving according to the user's emotional state, and the user adjusts their driving accordingly.

[2311] Specific examples

[2312] 1. Situation: A user is driving an autonomous vehicle and approaches a congested area.

[2313] 2. Terminal:

[2314] The camera captures images of the road ahead, and a generative AI model analyzes the traffic congestion situation.

[2315] The user's voice is picked up by a microphone and stress levels are measured using an emotion recognition engine.

[2316] 3. Server:

[2317] Based on the acquired location information, traffic congestion information is obtained via a traffic information API.

[2318] 4. Results:

[2319] If the user indicates high stress levels, the vehicle's speed will be automatically reduced and a voice notification will be provided saying, "Relax. We will reduce your speed."

[2320] Prompt Sentence Examples

[2321] "Create an application that uses an emotion recognition model to analyze passenger emotions in real time and adjust the vehicle's driving style based on those emotions. Additionally, use an external traffic information API to obtain traffic congestion information and include guidelines to reduce passenger stress."

[2322] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2323] Step 1:

[2324] The device acquires video of the road ahead using the autonomous vehicle's camera, uses the real-time video data acquired from the camera as input, temporarily stores it in its internal memory, and provides video data for the next processing step as output.

[2325] Step 2:

[2326] The device uses an emotion recognition engine to recognize the user's emotions in real time. It uses audio data acquired from a microphone and facial expression data acquired from a camera as input. The emotion recognition engine analyzes this data to identify the user's emotional state. The output is generated as the user's emotional state data (e.g., stress level, high, medium, low).

[2327] Step 3:

[2328] The device sends the location information it has acquired to the server. It uses the location data from the GPS module as input. It sends the location data to the server and requests information on accident-prone areas. It receives the information on accident-prone areas returned from the server as output.

[2329] Step 4:

[2330] The server compares the location data it receives with a database of accident-prone areas. The location data and the information on accident-prone areas in the database are used as input. The database is compared to extract information on the relevant accident-prone areas and sent to the terminal. The information on the relevant accident-prone areas is provided as output.

[2331] Step 5:

[2332] An audio alert is issued to the user based on the information on accident-prone areas received by the device. The information on accident-prone areas and the previously acquired video data are used as input. An audio alert generation engine is used to warn the user. The audio data that issues the audio alert is generated as output and executed.

[2333] Step 6:

[2334] The video data acquired by the device is analyzed in real time using a generative AI model to detect obstacles. Video data of the front is used as input. The generative AI model is used to recognize objects on the screen and identify the type of obstacle (pedestrian, car, etc.), as well as their relative speed and distance. Information data on the detected obstacle is generated as output and used for the next deceleration process.

[2335] Step 7:

[2336] The device comprehensively assesses the emotional state, obstacle data, and information on accident-prone areas, and sends instructions to the motor controller to adjust the driving style. It also uses the emotional state data, obstacle data, and information on accident-prone areas as inputs. This includes issuing instructions to reduce speed when stress levels are high or obstacles are nearby. It generates deceleration command data that is sent to the motor controller as output.

[2337] Step 8:

[2338] The motor control device receives the deceleration command from the terminal and adjusts the speed of the autonomous vehicle. It uses the deceleration command data as input, adjusts the motor output of the autonomous vehicle according to the deceleration command, and reduces the speed appropriately. It feeds back the actual speed data of the decelerated vehicle as output, and maintains a safe driving style.

[2339] In this way, the steps work in cooperation with each other, thereby making it possible to improve the safety and comfort of the user.

[2340] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2341] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2342] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2343] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2344] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2345] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2346] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2347] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2348] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2349] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2350] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2351] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2352] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2353] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2354] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2355] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2356] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2357] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2358] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2359] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2360] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2361] The following is further disclosed regarding the above embodiment.

[2362] (Claim 1)

[2363] A means for acquiring location information;

[2364] A means for acquiring a forward image by a camera;

[2365] A means for detecting accident-prone areas using the acquired location information;

[2366] A means of analyzing the acquired video in real time and detecting obstacles;

[2367] means for issuing an audio alert when an accident hotspot or obstacle is detected;

[2368] means for automatically slowing the bicycle down in response to an obstacle;

[2369] A system including:

[2370] (Claim 2)

[2371] 2. The system according to claim 1, further comprising means for transmitting the acquired location information to a server, and for the server to return information on accident-prone areas.

[2372] (Claim 3)

[2373] 10. The system of claim 1, further comprising means for analyzing the acquired video using generative AI to calculate the relative speed and distance of the object.

[2374] (Claim 4)

[2375] 10. The system of claim 1, further comprising means for automatically slowing down for obstacles using a slow down mode for pedestrians, a keep left mode for cyclists, and a flashing light alert mode for automobiles.

[2376] (Claim 5)

[2377] 10. The system of claim 1, further comprising means for slowing the speed of the bicycle only upon confirmation that the user has both hands on the handlebars.

[2378] "Example 1"

[2379] (Claim 1)

[2380] A means for acquiring location information;

[2381] A means for acquiring a forward image by a camera;

[2382] A means for detecting accident-prone areas using the acquired location information;

[2383] A means of analyzing the acquired video in real time and detecting obstacles;

[2384] means for issuing an audio alert when an accident hotspot or obstacle is detected;

[2385] means for automatically slowing the bicycle down in response to an obstacle;

[2386] means for performing the slowdown when the user agrees to the slowdown;

[2387] A system including:

[2388] (Claim 2)

[2389] 2. The system according to claim 1, further comprising means for transmitting the acquired location information to a server, and for the server to return information on accident-prone areas.

[2390] (Claim 3)

[2391] 10. The system of claim 1, further comprising means for analyzing the acquired video using generative AI to calculate the relative speed and distance of the object.

[2392] "Application Example 1"

[2393] (Claim 1)

[2394] A means for acquiring location information;

[2395] A means for acquiring a forward image by a camera;

[2396] A means for detecting accident-prone areas using the acquired location information;

[2397] A means of analyzing the acquired video in real time and detecting obstacles;

[2398] means for issuing an audio alert when an accident hotspot or obstacle is detected;

[2399] means for automatically slowing the bicycle down in response to an obstacle;

[2400] A means of analyzing camera images with a generative AI model and detecting objects,

[2401] A means of verifying that you are holding the handlebars of the bicycle;

[2402] means for controlling a creep mode of the bicycle;

[2403] A system including:

[2404] (Claim 2)

[2405] 2. The system according to claim 1, further comprising means for transmitting the acquired location information to a server, and for the server to return information on accident-prone areas.

[2406] (Claim 3)

[2407] 10. The system of claim 1, further comprising means for analyzing the captured video with a generative AI model to calculate the relative velocity and distance of the object.

[2408] "Example 2: Combining Emotion Engines"

[2409] (Claim 1)

[2410] A means for acquiring location information;

[2411] A means for acquiring a forward image by a camera;

[2412] A means for detecting accident-prone areas using the acquired location information;

[2413] A means of analyzing the acquired video in real time and detecting obstacles;

[2414] means for issuing an audio alert when an accident hotspot or obstacle is detected;

[2415] means for automatically slowing the bicycle down in response to an obstacle;

[2416] means for recognizing a user's emotions and adjusting the speed or warning mode of the bicycle based on the user's emotional state;

[2417] A system including:

[2418] (Claim 2)

[2419] 2. The system according to claim 1, further comprising means for transmitting the acquired location information to a server, and for the server to return information on accident-prone areas.

[2420] (Claim 3)

[2421] 10. The system of claim 1, further comprising means for analyzing the acquired video using generative AI to calculate the relative speed and distance of the object.

[2422] "Application example 2 when combining emotion engines"

[2423] (Claim 1)

[2424] A means for acquiring location information;

[2425] A means for acquiring a forward image by a camera;

[2426] A means for detecting accident-prone areas using the acquired location information;

[2427] A means of analyzing the acquired video in real time and detecting obstacles;

[2428] A means for recognizing the user's emotions using an emotion recognition engine and adjusting the driving style according to the user's stress level;

[2429] means for issuing an audio alert when an accident hotspot or obstacle is detected;

[2430] means for automatically slowing down in response to obstacles;

[2431] A system including:

[2432] (Claim 2)

[2433] 2. The system according to claim 1, further comprising means for transmitting the acquired location information to a server, and for the server to return information on accident-prone areas.

[2434] (Claim 3)

[2435] 10. The system of claim 1, further comprising means for analyzing the captured video with a generative AI model to calculate the relative velocity and distance of the object. [Explanation of symbols]

[2436] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring location information; A means for acquiring a forward image by a camera; A means for detecting accident-prone areas using the acquired location information; A means of analyzing the acquired video in real time and detecting obstacles; means for issuing an audio alert when an accident hotspot or obstacle is detected; means for automatically slowing the bicycle down in response to an obstacle; A system including:

2. 2. The system according to claim 1, further comprising means for transmitting the acquired location information to a server, and for the server to return information on accident hotspots.

3. The system of claim 1 further comprising means for analyzing the acquired image with a generating AI and calculating the relative speed and distance of the object.

4. 10. The system of claim 1, further comprising means for automatically slowing down for obstacles using a slow down mode for pedestrians, a keep left mode for cyclists, and a flashing light alert mode for automobiles.

5. 10. The system of claim 1, further comprising means for slowing the bicycle down only when the user confirms that both hands are on the handlebars.

Images