Intelligent Acoustic Warning Apparatus for Vehicles and Control Method Thereof

The intelligent acoustic warning device uses neural networks to predict vehicle paths and adjust sounds based on pedestrian behavior and environmental factors, addressing the inadequacies of conventional systems by enhancing safety for vulnerable road users.

KR103043711B1Active Publication Date: 2026-11-06E INTELLIGENCE CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
KR1020250180283
Authority / Receiving Office
KR · KR
Patent Type
Patents
Current Assignee / Owner
Filing Date
2025-11-25
Publication Date
2026-11-06
Estimated Expiration
2045-11-25

AI Technical Summary

Technical Problem

Conventional acoustic warning systems in vehicles do not effectively utilize driving path information and pedestrian behavior to customize warning sounds, leading to inadequate pedestrian awareness, especially for vulnerable road users in low-noise vehicles.

Method used

An intelligent acoustic warning device using pre-trained neural networks to predict vehicle paths, detect surrounding objects, and adjust warning sounds based on protection zones and ambient noise, incorporating pedestrian behavior prediction and vehicle-pedestrian contact risk assessment.

Benefits of technology

Enhances pedestrian safety by providing customized, environment-aware warning sounds that accurately reflect driving conditions and pedestrian behavior, reducing accidents and improving user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 112025132057346-PAT00003_ABST
    Figure 112025132057346-PAT00003_ABST
Patent Text Reader

Abstract

A control method for an intelligent acoustic warning device for a vehicle according to one embodiment includes the steps of: predicting a driving path of a vehicle; detecting surrounding objects of the vehicle; inputting information regarding the predicted driving path and information regarding the detected surrounding objects into a pre-trained neural network model to obtain warning function selection information as the output of the neural network model; and controlling the vehicle to output a warning sound by activating an acoustic warning function according to the obtained warning function selection information.
Need to check novelty before this filing date? Find Prior Art

Description

Technology Field

[0001] The present invention relates to an intelligent acoustic warning device for a vehicle and a method for controlling the device to output a warning sound. Background Technology

[0002] With the recent advancements in autonomous driving technology and Advanced Driver Assistance Systems (ADAS), vehicles have become capable of continuously perceiving their surroundings and evaluating risk factors while driving to perform various warnings or braking controls. Along with these technological progress, the increase in electric vehicles (EVs) and hybrid vehicles has led to a significant reduction in driving noise, raising concerns that pedestrians may find it difficult to audibly perceive the approach of vehicles.

[0003] In conventional technology, it was common practice to output a simple warning sound inside or outside the vehicle based on the vehicle's driving speed, distance from surrounding objects, or the possibility of a collision.

[0004] However, conventional acoustic warning methods use a uniform, single pattern of warning sound regardless of the vehicle's driving path information, so the effectiveness of the acoustic warning has not reached a satisfactory level. Prior art literature

[0005] U.S. Patent Publication US 9,421,909, VEHICLE TO PEDESTRIAN COMMUNICATION SYSTEM AND METHOD The problem to be solved

[0006] According to an embodiment, an intelligent acoustic warning device for a vehicle and a control method thereof are provided, wherein a pre-trained neural network model activates an acoustic warning function by reflecting the vehicle's driving path information.

[0007] The problems to be solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description below. means of solving the problem

[0008] A control method for an intelligent acoustic warning device for a vehicle according to a first aspect includes the steps of: predicting a driving path of a vehicle; detecting surrounding objects of the vehicle; inputting information about the predicted driving path and information about the detected surrounding objects into a pre-trained neural network model to obtain warning function selection information as the output of the neural network model; and controlling the vehicle to output a warning sound by activating an acoustic warning function according to the obtained warning function selection information.

[0009] Here, the step of predicting the driving path of the vehicle involves obtaining road attribute information of the driving path from the electronic map of the navigation program, and the step of obtaining the warning function selection information involves, when data on protection zones for vulnerable road users among the road attribute information is input into the neural network model, the neural network model reflects child protection zones, elderly protection zones, and disabled protection zones as the data on protection zones for vulnerable road users and outputs the warning function selection information customized for each vulnerable road user.

[0010] The method further includes a step of acquiring ambient noise of the vehicle, and the controlling step adjusts the volume of the warning sound by reflecting information about the acquired ambient noise.

[0011] The step of obtaining the above warning function selection information outputs the warning function selection information such that, when at least a portion of the above predicted driving path is included in the child protection zone as the above vulnerable road user protection zone data, a warning sound of a relatively higher frequency than that of the elderly protection zone is output.

[0012] The step of acquiring the above warning function selection information outputs the warning function selection information such that, if at least a portion of the above predicted driving path is included in the elderly protection zone as the above vulnerable road user protection zone data, a warning sound of a relatively lower frequency than that of the child protection zone is output.

[0013] The step of acquiring the above warning function selection information outputs the warning function selection information to illuminate a warning light along with the output of a warning sound when at least a portion of the predicted driving path is included in the disabled protection zone as the above traffic vulnerable protection zone data.

[0014] The above neural network model includes a first neural network model and a second neural network model, and the step of obtaining the warning function selection information includes the step of inputting a pedestrian image as information about the detected surrounding object into the first neural network model to obtain pedestrian behavior prediction information as the output of the first neural network model, and the step of inputting information about the predicted driving path and the pedestrian behavior prediction information into the second neural network model to obtain the risk of contact between the vehicle and the pedestrian as the warning function selection information as the output of the second neural network model, and the controlling step activates the acoustic warning function when the risk of contact is greater than or equal to a preset threshold.

[0015] The first neural network model is trained to minimize the error with the correct label corresponding to the pedestrian's behavior by using a training pedestrian image containing a pedestrian, and is trained to output pedestrian behavior prediction information by extracting visual features of the pedestrian from the pedestrian image and modeling temporal behavior patterns. The second neural network model is trained to minimize the error with the correct label regarding whether there is actual contact between the vehicle and the pedestrian or the risk of contact by using vehicle driving path information including the vehicle's driving speed, steering angle, and planned path, and pedestrian behavior prediction information predicted by the first neural network model as input, and is trained to output the risk of contact between the vehicle and the pedestrian based on the input.

[0016] The learning process of the first neural network model and the second neural network model is performed using a learning dataset that includes multiple video sequences containing pedestrians, vehicle driving path information in various driving environments, and vehicle-pedestrian interaction data. The first neural network model models changes in pedestrian behavior by applying a sequence-based neural network structure to temporally analyze the movement of pedestrians from the learning video sequences, and improves learning performance through data augmentation including simulations of brightness changes, rotation, visual distortion, and speed changes. The second neural network model additionally receives derived feature information representing the relative positional relationship, relative speed, and temporal change amount between the vehicle driving path information and the pedestrian behavior prediction information, and learns by comparing the contact risk or collision probability corresponding to each learning scenario with the correct label.

[0017] The training process of the first neural network model and the second neural network model is configured to pre-train pedestrian behavior prediction information using the first neural network model, and then share the output features of the first neural network model with the training process of the second neural network model, thereby performing a joint training method in which the two neural network models simultaneously reflect pedestrian behavior features and vehicle-pedestrian relationship information; the second neural network model is configured by applying parameter reduction, quantization, or a lightweight computational structure for real-time application within the vehicle; the first neural network model and the second neural network model additionally perform an online training or model calibration process to correct parameters by reflecting the difference between the output values ​​of the two neural network models and actual driving results, based on pedestrian videos collected during vehicle driving, vehicle driving data, and actual warning occurrence history; and the online training or model calibration process is performed on changes in object distribution to maintain or improve the prediction performance of the neural network models according to changes in the driving environment, changes in pedestrian types, or changes in traffic conditions It is performed by including adaptation or weighting methods for recent data.

[0018] As a second aspect, an intelligent acoustic warning device for a vehicle comprises: a memory in which a computer program including at least one instruction is stored; a processor that executes said instruction; and an output unit that outputs a warning sound according to the control of said processor; wherein the processor predicts the driving path of the vehicle and detects surrounding objects of said vehicle, inputs information regarding the predicted driving path and information regarding the detected surrounding objects into a pre-trained neural network model to obtain warning function selection information as the output of said neural network model, and controls said output unit to activate an acoustic warning function according to said warning function selection information so that said warning sound is output.

[0019] According to a third perspective, a computer program stored on a computer-readable recording medium includes instructions for the computer program to cause a processor to perform a control method for the intelligent acoustic warning device for a vehicle.

[0020] According to the fourth aspect, a computer-readable recording medium in which a computer program is stored includes instructions for the computer program to cause a processor to perform a control method for the intelligent acoustic warning device for a vehicle. Effects of the invention

[0021] According to the embodiment, by predicting the vehicle's driving path in advance and precisely analyzing pedestrian behavior through a neural network model based on video information regarding surrounding objects, particularly pedestrians, the potential risk of contact between the vehicle and the pedestrian can be calculated more accurately. Accordingly, unlike conventional acoustic warning methods that rely on single indicators such as distance or speed, customized warning control reflecting actual driving conditions and pedestrian behavior patterns becomes possible.

[0022] Furthermore, by utilizing road attribute information extracted from navigation electronic maps—specifically data on protection zones for vulnerable road users such as child protection zones, elderly protection zones, and disabled protection zones—as input to a neural network model, differentiated warning sounds reflecting the characteristics of each zone can be output. This enables the provision of a warning service more suitable for vulnerable road users who have difficulty auditorily perceiving approaching vehicles, and significantly enhances accident prevention effectiveness within these protection zones.

[0023] In addition, by inputting ambient noise information measured in real-time during vehicle operation into a neural network model and reflecting it in the warning function selection information, the output intensity, frequency, or pattern of the warning sound can be automatically adjusted according to changes in the surrounding environment. Therefore, optimal warning effectiveness can be maintained even in various acoustic environments, such as noisy urban environments or quiet situations at night.

[0024] In particular, by predicting pedestrian behavior through a first neural network model that takes pedestrian video as input, and inputting the predicted pedestrian behavior and vehicle driving path information into a second neural network model to calculate the risk of vehicle-pedestrian contact, it is possible to determine with greater reliability whether to activate the warning function. This has the effect of improving the user experience by detecting actual dangerous situations around the vehicle early and preventing confusion caused by unnecessary warnings or the overuse of warning sounds.

[0025] Ultimately, by outputting warning sounds through the integrated consideration of various environmental information—such as vehicle driving routes, pedestrian behavior, data on protection zones for vulnerable road users, and ambient noise levels—it significantly enhances driving safety and effectively resolves new safety issues arising from the increase in low-noise vehicles. Consequently, it prevents pedestrian accidents and strengthens the protection of vulnerable road users, while simultaneously contributing to the advancement of the technical sophistication of intelligent acoustic warning systems for vehicles. Brief explanation of the drawing

[0026] FIG. 1 is a configuration diagram of a computing device that can be used as an intelligent acoustic warning device for a vehicle according to an embodiment of the present invention. FIG. 2 is a configuration diagram showing the functional classification of computer programs loaded by a processor of a computing device according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating a control method for an intelligent acoustic warning device for a vehicle according to an embodiment of the present invention. Specific details for implementing the invention

[0027] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

[0028] The terms used in this specification will be briefly explained, and the invention will be described in detail.

[0029] The terms used in this invention have been selected based on currently widely used general terms, taking into account their functions within the invention; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms have been arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should be defined not merely by their names, but based on their meanings and the overall content of the invention.

[0030] When a part of a specification is described as 'comprising' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0031] Additionally, the term "part" as used in the specification refers to software or hardware components, such as FPGAs or ASICs, and the "part" performs certain roles. However, the meaning of "part" is not limited to software or hardware. The "part" may be configured to reside in an addressable storage medium or configured to operate one or more processor parts. Thus, by example, the "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts."

[0032] Below, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the invention. Additionally, parts of the drawings that are irrelevant to the description are omitted to clearly explain the invention.

[0033] FIG. 1 is a configuration diagram of a computing device that can be used as an intelligent acoustic warning device for a vehicle according to an embodiment of the present invention, and FIG. 2 is a configuration diagram functionally classifying a computer program loaded by a processor of a computing device according to an embodiment of the present invention.

[0034] Referring to FIGS. 1 and 2, a computing device (100) according to an embodiment includes a memory (110) in which a computer program (111) is stored and a processor (120), and may further include an input unit (130) and / or an output unit (140).

[0035] A computer program (111) stored in the memory (110) of such a computing device (100) includes at least one instruction that can be executed by a processor (120). By executing the instruction of such a computer program (111) by the processor (120), the computer program (111) is loaded so that a control method for an intelligent acoustic warning device for a vehicle according to an embodiment of the present invention can be performed. In addition, various data necessary for the processor (120) to perform a control method for an intelligent acoustic warning device for a vehicle according to an embodiment of the present invention may be stored in the memory (110).

[0036] The processor (120) performs a control method for an intelligent acoustic warning device for a vehicle according to an embodiment by executing instructions of a computer program (111) stored in memory (110) and loading the computer program (111). The functions of the driving path prediction unit (210), surrounding object detection unit (220), first neural network model (230), second neural network model (240), and warning sound control unit (250) based on the computer program (111) thus loaded will be explained again below.

[0037] The input unit (130) receives various data necessary for the processor (120) of the computing device (100) to perform a control method for an intelligent acoustic warning device for a vehicle according to an embodiment, and provides this data to the processor (120). For example, the input unit (130) may receive a captured image of an object surrounding the vehicle and provide it to the processor (120).

[0038] The output unit (140) can provide various data acquired and / or generated as the processor (120) of the computing device (100) performs the control method of the intelligent acoustic warning device for a vehicle according to the embodiment to the outside. For example, the output unit (140) can output a warning sound and / or light a warning light according to the control of the processor (120).

[0039] When a computer program (111) loaded by a processor (120) performs the function of a driving path prediction unit (210), it predicts the driving path of a vehicle. For example, the driving path prediction unit (210) can obtain road attribute information of the driving path from an electronic map of a navigation program.

[0040] When a computer program (111) loaded by a processor (120) performs the function of a surrounding object detection unit (220), it detects surrounding objects of the vehicle. For example, the surrounding object detection unit (220) can detect surrounding objects by performing object detection and object recognition through analysis of a captured image of surrounding objects.

[0041] A computer program (111) loaded by a processor (120) inputs information about a driving path predicted by a driving path prediction unit (210) and information about surrounding objects detected by a surrounding object detection unit (220) into a pre-trained neural network model to obtain warning function selection information as the output of the neural network model. The neural network model includes a first neural network model (230) and a second neural network model (240), and pedestrian image can be input into the first neural network model (230) as information about the detected surrounding objects to obtain pedestrian behavior prediction information as the output of the first neural network model (230), and information about the predicted driving path and pedestrian behavior prediction information can be input into the second neural network model (240) to obtain the risk of contact between a vehicle and a pedestrian as warning function selection information as the output of the second neural network model (240).

[0042] Here, when data of a protection zone for vulnerable road users is input into the second neural network model (240) among the road attribute information, the processor (120) can output warning function selection information customized for each vulnerable road user by reflecting child protection zones, elderly protection zones, and disabled protection zones as data of the protection zone for vulnerable road users. If at least a portion of the predicted driving path in a child protection zone is included as data of the protection zone for vulnerable road users, the warning function selection information can be output to output a warning sound of a relatively higher frequency than in an elderly protection zone. If at least a portion of the predicted driving path in an elderly protection zone is included as data of the protection zone for vulnerable road users, the warning function selection information can be output to output a warning sound of a relatively lower frequency than in a child protection zone. If at least a portion of the predicted driving path in a disabled protection zone is included as data of the protection zone for vulnerable road users, the warning function selection information can be output to light a warning light along with the output of a warning sound.

[0043] When a computer program (111) loaded by a processor (120) performs the function of a warning sound control unit (250), the warning sound control unit (250) controls an output unit (140) to output a warning sound from the vehicle by activating the sound warning function according to warning function selection information obtained through a previously trained neural network model. Additionally, the surrounding object detection unit (220) can acquire ambient noise of the vehicle, and in this case, can adjust the volume of the warning sound by reflecting information about the acquired ambient noise.

[0044] FIG. 3 is a flowchart illustrating a control method for an intelligent acoustic warning device for a vehicle according to an embodiment of the present invention.

[0045] Hereinafter, a method for controlling a computing device (100) as an intelligent acoustic warning device for a vehicle according to an embodiment will be described with reference to FIGS. 1 to 3.

[0046] First, the processor (120) of the computing device (100) can perform a control method for an intelligent acoustic warning device for a vehicle according to an embodiment by executing instructions of a computer program (111) stored in memory (110) and loading the computer program (111). The computer program (111) thus loaded performs the functions of a driving path prediction unit (210), a surrounding object detection unit (220), a first neural network model (230), a second neural network model (240), and a warning sound control unit (250).

[0047] The first neural network model (230) and the second neural network model (240) are in a pre-trained state, and the training process of the first neural network model (230) and the second neural network model (240) is as follows.

[0048] The first neural network model (230) is trained using a training pedestrian image containing a pedestrian, and its parameters are trained to minimize the error with the correct label corresponding to the pedestrian's behavior, and it is trained to output pedestrian behavior prediction information by extracting visual features of the pedestrian from the pedestrian image and modeling the temporal behavior pattern.

[0049] The second neural network model (240) takes as input vehicle driving path information including the vehicle's driving speed, steering angle, and planned path, and pedestrian behavior prediction information predicted by the first neural network model (230), and learns parameters to minimize the error with the correct label regarding whether there is actual contact between the vehicle and the pedestrian or the risk of contact, and learns to output the risk of contact between the vehicle and the pedestrian based on the input.

[0050] Additionally, the learning process of the first neural network model (230) and the second neural network model (240) can be performed using a learning dataset that includes multiple video sequences containing pedestrians, vehicle driving path information in various driving environments, and vehicle-pedestrian interaction data. The first neural network model (230) can improve learning performance by applying a sequence-based neural network structure to model changes in pedestrian behavior in order to temporally analyze the movement of pedestrians from the video sequences for learning, and by using data augmentation that includes simulations of brightness changes, rotation, visual distortion, and speed changes. The second neural network model (240) can be learned by additionally receiving derived feature information representing the relative positional relationship, relative speed, and temporal change amount between the vehicle driving path information and the pedestrian behavior prediction information, and comparing the contact risk or collision probability corresponding to each learning scenario with the correct label.

[0051] In addition, the first neural network model (230) is configured to pre-train pedestrian behavior prediction information and then share the output features of the first neural network model (230) with the learning process of the second neural network model (240), so that the two neural network models (230, 240) are performed in a joint training manner that simultaneously reflects pedestrian behavior features and vehicle-pedestrian relationship information, and the second neural network model (240) is configured by applying a reduced number of parameters, quantization, or a lightweight computational structure for real-time application within the vehicle.

[0052] Based on pedestrian images collected while driving, vehicle driving data, and actual warning occurrence history, an online learning or model calibration process may be additionally performed to correct parameters by reflecting the difference between the output values ​​of the two neural network models (230, 240) and the actual driving results, and the online learning or model calibration process may include adaptation to changes in object distribution or weighting methods for recent data to maintain or improve the prediction performance of the neural network models (230, 240) according to changes in the driving environment, changes in pedestrian types, or changes in traffic conditions.

[0053] When a computer program (111) equipped with the first neural network model (230) and the second neural network model (240) that have undergone the learning process described above is loaded by a processor (120) to perform the function of a driving path prediction unit (210), the driving path of a vehicle is predicted. For example, the driving path prediction unit (210) can obtain road attribute information of the driving path from an electronic map of a navigation program (S310).

[0054] And, when the computer program (111) loaded by the processor (120) performs the function of the surrounding object detection unit (220), it detects surrounding objects of the vehicle. For example, the surrounding object detection unit (220) can detect surrounding objects by performing object detection and object recognition through analysis of a captured image of surrounding objects (S320).

[0055] And, pedestrian image can be input into the first neural network model (230) as information about the detected surrounding object to obtain pedestrian behavior prediction information as the output of the first neural network model (230) (S330), and information about the predicted driving path and pedestrian behavior prediction information can be input into the second neural network model (240) to obtain the risk of contact between the vehicle and the pedestrian as warning function selection information as the output of the second neural network model (240) (S340).

[0056] Here, when data of a protection zone for vulnerable road users is input into the second neural network model (240) among the road attribute information, the processor (120) can output warning function selection information customized for each vulnerable road user by reflecting child protection zones, elderly protection zones, and disabled protection zones as data of the protection zone for vulnerable road users. If at least a portion of the predicted driving path in a child protection zone is included as data of the protection zone for vulnerable road users, the warning function selection information can be output to output a warning sound of a relatively higher frequency than in an elderly protection zone. If at least a portion of the predicted driving path in an elderly protection zone is included as data of the protection zone for vulnerable road users, the warning function selection information can be output to output a warning sound of a relatively lower frequency than in a child protection zone. If at least a portion of the predicted driving path in a disabled protection zone is included as data of the protection zone for vulnerable road users, the warning function selection information can be output to light a warning light along with the output of a warning sound.

[0057] When a computer program (111) loaded by a processor (120) performs the function of a warning sound control unit (250), the warning sound control unit (250) controls an output unit (140) to output a warning sound from the vehicle by activating the sound warning function according to warning function selection information obtained through a previously learned neural network model. Additionally, the surrounding object detection unit (220) can acquire surrounding noise of the vehicle, and in this case, can adjust the volume of the warning sound by reflecting information about the acquired surrounding noise (S350).

[0058] In order to facilitate understanding of the invention, a control method for an intelligent acoustic warning device for vehicles will be described in detail below, taking into account examples.

[0059] The processor (120) of the computing device (100) performs a control method for an intelligent acoustic warning device for a vehicle according to an embodiment by sequentially reading and executing instructions included in a computer program (111) stored in memory (110). When loaded, this computer program (111) is placed in an internal cache or execution area of ​​the processor (120) to control various functional modules, such as a driving path prediction unit (210), a surrounding object detection unit (220), a first neural network model (230), a second neural network model (240), and a warning sound control unit (250). Each module performs an independent function, but can be designed to be linked sequentially or in parallel by the processor (120).

[0060] <신경망 모델(230, 240)의 구성 및 학습 준비 과정>

[0061] Both the first neural network model (230) and the second neural network model (240) are provided in a state where they have been trained in advance using a large amount of training dataset, or are installed in the vehicle after performing additional training during the vehicle manufacturing stage or service stage.

[0062] Prior to the training process, the two models may include a predefined pipeline to perform tasks such as normalization, resizing, channel alignment, brightness correction, and inter-frame time synchronization of input data through well-known data pre-processing procedures. This is intended to respond to changes in the recognition environment, such as varying illumination levels, resolution differences between cameras, and nighttime or adverse weather conditions.

[0063] <제 1 신경망 모델(230)의 상세 학습 과정>

[0064] The first neural network model (230) is trained using training video data that includes pedestrians. The video used here may include various shooting environments such as fixed CCTVs, vehicle front cameras, and mobile cameras, and may be composed of video including various countries, weather conditions, time zones, and crowd density conditions to ensure diversity of training data.

[0065] The training dataset includes ground truth labels representing the behavioral states of pedestrians, and may include various classes such as 'moving forward', 'stopping', 'intending to cross', 'crossing', 'backward', and 'likely to cause sudden behavior'. The model updates parameters in a direction that minimizes the error with these ground truth labels, and generally applies optimization algorithms based on backpropagation and gradient descent (e.g., Adam, RMSProp).

[0066] Additionally, since pedestrian behavior involves temporal continuity, the first neural network model (230) may include conventionally used sequence-based deep learning structures. For example, structures such as CNN-LSTM networks, 3D CNN structures, Temporal Convolution Network (TCN), and Transformer-based behavior recognition models may be applied.

[0067] In the preprocessing stage, the model's generalization performance is enhanced through data augmentation techniques. This can consist of various augmentation methods, such as brightness adjustment, rotation, image warping, random cropping, and the addition of random noise.

[0068] <제 2 신경망 모델(240)의 상세 학습 과정>

[0069] The second neural network model (240) is trained to calculate the risk of contact between a vehicle and a pedestrian by receiving a combination of vehicle driving path information and pedestrian behavior prediction information.

[0070] The input vehicle driving path information may include data such as vehicle speed, acceleration, deceleration, steering angle, steering variation amount, planned travel path (navigation path), lane information, road structure information (curvature, gradient, etc.), and the vehicle's current and expected future position. This data may be obtained from known vehicle sensors (INS, GPS, IMU, vehicle CAN data, etc.), and conventional sensor fusion techniques such as coordinate transformation, time axis alignment, and interpolation may be applied to make it suitable for model input.

[0071] The second neural network model (240) can further improve prediction accuracy by expanding the input to include derived features such as the relative distance between the vehicle and the pedestrian, relative speed, time to collision, lateral approach angle, and pedestrian movement direction.

[0072] The correct answer label may consist of actual collision status extracted from actual driving logs, a risk score, or a risk rating (A / B / C, etc.) based on expert comments.

[0073] The learning process can be performed using general multiple regression (MSE loss) or cross-entropy loss for risk classification.

[0074] <두 모델 간의 공동 학습(joint training) 및 경량화 과정>

[0075] According to an embodiment, the first neural network model (230) pre-trains the behavior of a pedestrian to generate an intermediate representation value (embedding or feature vector) representing the pedestrian behavior characteristics.

[0076] This intermediate feature is shared with the second neural network model (240) so that the two models are jointly optimized, or the second neural network model (240) is configured to more accurately reflect pedestrian behavior patterns through transfer learning.

[0077] Since the second neural network model (240) must be mounted on a vehicle to perform real-time warnings, parameter quantization (8-bit quantization), lightweight CNN structures (MobileNet, EfficientNet-lite, etc.), model pruning, and TensorRT-based optimization for minimizing latency may be applied to reduce execution latency.

[0078] <온라인 학습 및 모델 보정(calibration)>

[0079] By reflecting the latest data accumulated while driving the vehicle (changes in the surrounding environment, changes in the urban / suburban environment, changes in pedestrian types, etc.), the two neural network models (230, 240) can perform online learning or parameter calibration as needed.

[0080] In particular, adaptive learning methods that assign weights to recent data, domain adaptation methods that respond to domain shifts, or temperature scaling methods that readjust values ​​for specific scenarios (e.g., lack of light, rain, snow, etc.) may be applied.

[0081] <운영 단계 - 주행 경로 예측 및 주변 물체 감지>

[0082] The processor (120) obtains electronic map-based road attribute information of the navigation program through the driving path prediction unit (210) and analyzes information such as lane configuration, speed limit, and protection zone for vulnerable road users included in the road attributes (S310).

[0083] The surrounding object detection unit (220) detects surrounding objects using known ADAS sensors such as a camera, radar, and lidar (S320).

[0084] For example, object detection (YOLO, SSD, Faster R-CNN, etc.) and object tracking (Kalman Filter, Deep SORT, etc.) can be performed to calculate the pedestrian's location, direction of movement, speed, etc.

[0085] <위험도 평가 및 맞춤형 경고 제어>

[0086] When a pedestrian video is obtained, the first neural network model (230) receives it as input and predicts the pedestrian's movement state and potential future actions (S330).

[0087] Afterward, the processor (120) inputs the predicted driving path information and pedestrian behavior prediction information into the second neural network model (240) to calculate the vehicle-pedestrian contact risk (S340).

[0088] The warning function selection information can be customized as follows by reflecting data on protection zones for vulnerable road users, which is one of the road attributes.

[0089] Child Protection Zone → Relatively high frequency warning sound

[0090] Elderly Protection Zone → Relatively Low Frequency Warning Sound

[0091] Protection Zone for the Disabled → Warning Sound + Warning Light Illumination

[0092] <경고 음향 출력 제어>

[0093] The warning sound control unit (250) activates the sound warning function based on the warning function selection information received from the second neural network model (240) (S350) and outputs a warning sound by controlling the vehicle external speaker or AVAS device, etc.

[0094] In addition, when ambient noise is detected, the volume of the warning sound can be automatically adjusted by applying a commonly used environment-adaptive volume control algorithm, thereby maintaining warning delivery and pedestrian awareness.

[0095] Meanwhile, a computer program stored on a computer-readable recording medium according to an embodiment includes at least one instruction for a processor of a computing device to perform each step included in the control method of an intelligent acoustic warning device for a vehicle according to the aforementioned embodiments.

[0096] Additionally, according to an embodiment, a computer-readable recording medium storing a computer program includes at least one instruction in the computer program for causing a processor of a computing device to perform each step included in the control method of an intelligent acoustic warning device for a vehicle according to the above-described embodiments.

[0097] As explained above, according to an embodiment of the present invention, by predicting the driving path of a vehicle in advance and precisely analyzing pedestrian behavior through a neural network model based on image information regarding surrounding objects, particularly pedestrians, the potential risk of contact between a vehicle and a pedestrian can be calculated more accurately. Accordingly, unlike conventional acoustic warning methods that rely simply on single indicators such as distance or speed, customized warning control reflecting actual driving conditions and pedestrian behavior patterns becomes possible.

[0098] Furthermore, by utilizing road attribute information extracted from navigation electronic maps—specifically data on protection zones for vulnerable road users such as child protection zones, elderly protection zones, and disabled protection zones—as input to a neural network model, differentiated warning sounds reflecting the characteristics of each zone can be output. This enables the provision of a warning service more suitable for vulnerable road users who have difficulty auditorily perceiving approaching vehicles, and significantly enhances accident prevention effectiveness within these protection zones.

[0099] In addition, by inputting ambient noise information measured in real-time during vehicle operation into a neural network model and reflecting it in the warning function selection information, the output intensity, frequency, or pattern of the warning sound can be automatically adjusted according to changes in the surrounding environment. Therefore, optimal warning effectiveness can be maintained even in various acoustic environments, such as noisy urban environments or quiet situations at night.

[0100] In particular, by predicting pedestrian behavior through a first neural network model that takes pedestrian video as input, and inputting the predicted pedestrian behavior and vehicle driving path information into a second neural network model to calculate the risk of vehicle-pedestrian contact, it is possible to determine with greater reliability whether to activate the warning function. This has the effect of improving the user experience by detecting actual dangerous situations around the vehicle early and preventing confusion caused by unnecessary warnings or the overuse of warning sounds.

[0101] Ultimately, by outputting warning sounds through the integrated consideration of various environmental information—such as vehicle driving routes, pedestrian behavior, data on protection zones for vulnerable road users, and ambient noise levels—it significantly enhances driving safety and effectively resolves new safety issues arising from the increase in low-noise vehicles. Consequently, it prevents pedestrian accidents and strengthens the protection of vulnerable road users, while simultaneously contributing to the advancement of the technical sophistication of intelligent acoustic warning systems for vehicles.

[0102] Combinations of each step of each flowchart attached to the present invention may be performed by computer program instructions. Since these computer program instructions may be loaded into the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, the instructions performed through the processor of the computer or other programmable data processing equipment create means for performing the functions described in each step of the flowchart. Since these computer program instructions may also be stored in a computer-available or computer-readable recording medium that can be directed toward the computer or other programmable data processing equipment to implement the function in a specific manner, the instructions stored in the computer-available or computer-readable recording medium may also produce a manufactured item containing instruction means for performing the function described in each step of the flowchart. Since computer program instructions can be loaded onto a computer or other programmable data processing equipment, instructions that execute a computer or other programmable data processing equipment by performing a series of operation steps on the computer or other programmable data processing equipment to create a process executed by the computer can also provide steps for executing the functions described in each step of the flowchart.

[0103] Additionally, each step may represent a module, segment, or part of code containing one or more executable instructions for executing a specified logical function(s). Also, it should be noted that in some alternative embodiments, the functions mentioned in the steps may occur out of order. For example, two steps described in succession may actually be performed substantially simultaneously, or the steps may sometimes be performed in reverse order according to the corresponding function.

[0104] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are intended to explain, not limit, the technical concept of the present invention, and the scope of the technical concept of the present invention is not limited by such embodiments. The scope of protection of the present invention shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the present invention. Explanation of the symbols

[0105] 100: Computing device 110: Memory 111: Computer program 120: Processor 130: Input section 140: Output section 210: Driving path prediction unit 220: Surrounding object detection unit 230: The First Artificial Neural Network Model 240: The Second Artificial Neural Network Model 250: Warning Sound Control Unit

Claims

Claim 1 A control method for an intelligent acoustic warning device for a vehicle comprises: a step of predicting a driving path of a vehicle; a step of detecting objects surrounding the vehicle; a step of obtaining pedestrian behavior prediction information by inputting a pedestrian image as information regarding the detected surrounding objects into a first neural network model; a step of obtaining warning function selection information including a risk of contact between a vehicle and a pedestrian by inputting information regarding the predicted driving path and the pedestrian behavior prediction information into a second neural network model; and a step of activating an acoustic warning function according to the warning function selection information. The step of predicting the driving path includes a step of obtaining road attribute information from an electronic map of a navigation program. The second neural network model receives data on protection zones for vulnerable road users among the road attribute information and outputs warning function selection information customized for each vulnerable road user by reflecting child protection zones, elderly protection zones, and disabled protection zones. The second neural network model is configured to calculate a risk of contact by additionally receiving derived feature information representing the relative positional relationship, relative speed, and temporal change amount between the vehicle driving path information and the pedestrian behavior prediction information. The output features of the first neural network model are to the second neural network model A control method for an intelligent acoustic warning device for a vehicle, wherein two neural network models are shared and jointly trained to reflect pedestrian behavioral characteristics and vehicle-pedestrian relationship information, wherein the second neural network model is configured by applying a computational structure that reduces the number of parameters, quantizes, or is lightweight for real-time application within the vehicle, wherein the first neural network model and the second neural network model additionally perform an online learning or model correction process to correct parameters by reflecting the difference between the output value and the actual driving result based on data collected while driving the vehicle, and wherein the warning function selection information is configured to include different acoustic characteristics or warning light illumination depending on the type of vulnerable road user protection zone. Claim 2 delete Claim 3 A control method for an intelligent acoustic warning device for a vehicle according to claim 1, further comprising the step of acquiring ambient noise of the vehicle, wherein the activating step adjusts the volume of a warning sound by the acoustic warning function by reflecting information about the acquired ambient noise. Claim 4 A control method for an intelligent acoustic warning device for a vehicle according to claim 1, wherein when the type of the protection zone for vulnerable road users is the child protection zone, the warning function selection information is set to output a warning sound of a relatively higher frequency than that of the protection zone for the elderly. Claim 5 A control method for an intelligent acoustic warning device for a vehicle according to claim 1, wherein when the type of the protection zone for vulnerable road users is the protection zone for the elderly, the warning function selection information is set to output a warning sound of a relatively lower frequency than that of the protection zone for children. Claim 6 A control method for an intelligent acoustic warning device for a vehicle according to claim 1, wherein if the type of the protection zone for vulnerable road users is the protection zone for disabled persons, the warning function selection information is set to illuminate a warning light along with the output of a warning sound. Claim 7 delete Claim 8 A control method for an intelligent acoustic warning device for a vehicle according to claim 1, wherein the first neural network model is trained to output pedestrian behavior prediction information by extracting visual features of a pedestrian from the pedestrian image and modeling temporal behavior patterns, and the second neural network model is trained to output the contact risk level based on vehicle driving path information and pedestrian behavior prediction information. Claim 9 In claim 8, the learning process of the first neural network model and the second neural network model is performed using a learning dataset including a plurality of video sequences containing a pedestrian, vehicle driving path information in various driving environments, and vehicle-pedestrian interaction data; the first neural network model models changes in pedestrian behavior by applying a sequence-based neural network structure to temporally analyze the movement of the pedestrian from the learning video sequences, and improves learning performance through data augmentation including simulations of brightness changes, rotation, visual distortion, and speed changes; and the second neural network model additionally receives derived feature information representing the relative positional relationship, relative speed, and temporal change amount between the vehicle driving path information and the pedestrian behavior prediction information, and learns by comparing the contact risk or collision probability corresponding to each learning scenario with the correct label. Claim 10 In claim 9, the learning process of the first neural network model and the second neural network model is configured such that pedestrian behavior prediction information is pre-trained using the first neural network model, and then the output features of the first neural network model are shared with the learning process of the second neural network model, thereby performing a joint training method in which the first neural network model and the second neural network model simultaneously reflect pedestrian behavior features and vehicle-pedestrian relationship information, and the second neural network model is configured by applying a parameter reduction, quantization, or a lightweight computational structure for real-time application within the vehicle, thereby a control method for an intelligent acoustic warning device for a vehicle. Claim 11 A control method for an intelligent acoustic warning device for a vehicle, wherein the online learning is performed by adapting to changes in object distribution or a weighting method for recent data. Claim 12 Memory storing a computer program including at least one instruction; a processor executing the instruction; and an output unit that outputs a warning sound according to the control of the processor; wherein the processor predicts the driving path of a vehicle, detects objects surrounding the vehicle, inputs a pedestrian image as information regarding the detected surrounding objects into a first neural network model to obtain pedestrian behavior prediction information, inputs information regarding the predicted driving path and the pedestrian behavior prediction information into a second neural network model to obtain warning function selection information including the risk of contact between the vehicle and the pedestrian, activates an audible warning function according to the warning function selection information, obtains road attribute information from an electronic map of a navigation program when predicting the driving path, the second neural network model receives data on protection zones for vulnerable road users among the road attribute information and outputs warning function selection information customized for each vulnerable road user by reflecting child protection zones, elderly protection zones, and disabled protection zones, and the second neural network model is configured to calculate the risk of contact by additionally receiving derived feature information representing the relative positional relationship, relative speed, and temporal change amount between the vehicle driving path information and the pedestrian behavior prediction information, and the output features of the first neural network model are shared with the second neural network model and two An intelligent acoustic warning device for a vehicle, wherein a neural network model is jointly trained to reflect pedestrian behavioral characteristics and vehicle-pedestrian relationship information, the second neural network model is configured by applying a computational structure that reduces the number of parameters, quantizes, or is lightweight for real-time application within the vehicle, the first neural network model and the second neural network model additionally perform an online learning or model correction process to correct parameters by reflecting the difference between the output value and the actual driving result based on data collected while driving the vehicle, and the warning function selection information is configured to include different acoustic characteristics or warning light illumination depending on the type of vulnerable road user protection zone. Claim 13 A computer program stored on a computer-readable recording medium, wherein the computer program comprises instructions for the processor to perform a control method of an intelligent acoustic warning device for a vehicle according to any one of claims 1, 3 to 6, and 8 to 11 by the processor executing at least one instruction. Claim 14 A computer-readable recording medium storing a computer program, wherein the computer program includes instructions for the processor to perform a control method of an intelligent acoustic warning device for a vehicle according to any one of claims 1, 3 to 6, and 8 to 11 by the processor executing at least one instruction.

Citation Information

Patent Citations

  • Apparatus and method for controlling virtual engine sound for a vehicle

    KR1020200145053A

  • Device and method for alarming pedestrian of a vehicle

    KR1020210020291A

  • Vehicle Having a Dangerous Situation Notification Function and Control Method thereof

    KR1020210053385A

  • Collision Avoidance / Mitigation by Machine Learning and Automatic Intervention

    US20240208490A1

  • System and method for mitigating vehicle collision with a bicycle

    US20250273072A1