System and method for providing personalized audio using artificial intelligence-based seat recognition

Abstract

A system and method for providing personalized audio based on seat recognition using artificial intelligence are provided. According to embodiments of the present invention, by recognizing in real time whether a passenger is seated, their face position, face direction, and passenger situation information for each seat through an AI camera installed inside a vehicle, and by actively controlling the rotation direction of a directional speaker corresponding to each seat based on this, it is possible to provide customized sound that precisely corresponds to the passenger's actual listening position. In addition, by combining individual audio signals input from terminals carried by each passenger, as well as audio signals played on the vehicle's AVN system, with seat position information and controlling the output differently for each seat, it is possible to minimize acoustic interference even when multiple passengers simultaneously use different content within the same vehicle.

Description

Technology Field

[0001] Embodiments of the present invention relate to a seat recognition-based personalized audio provision technology using artificial intelligence. Background Technology

[0003] Recently, vehicles have evolved beyond mere means of transportation into spaces that provide diverse information and multimedia services. Audio systems, navigation systems, communication devices, and various electronic devices are integrated within the vehicle interior to provide passengers with music, videos, voice guidance, and various types of information. In particular, as the demand for multiple passengers to consume different content within the vehicle increases, the importance of in-vehicle audio output control technology is growing.

[0004] In this regard, conventional vehicle audio systems have generally utilized methods that output the same sound throughout the entire cabin or adjust volume on a per-seat or per-speaker basis. Additionally, while some technologies have recently been introduced to control vehicle functions by detecting the presence or status of occupants, these are often limited to safety warnings or convenience features.

[0005] However, due to the limited spatial structure and diverse reflection characteristics of the vehicle interior, acoustic interference may occur when multiple sound sources are output simultaneously, potentially adversely affecting the driver's concentration while driving. Furthermore, as the position or posture of occupants can continuously change while the vehicle is in motion, relying solely on a fixed output method has limitations in creating a customized acoustic environment for each user. Prior art literature

[0007] Korean Registered Patent Publication No. 10-2885572 (2025.11.10) The problem to be solved

[0008] The embodiments of the present invention are intended to provide a more advanced audio control technology that can minimize acoustic interference and ensure safety while providing individual acoustic environments to multiple occupants within a vehicle. means of solving the problem

[0010] According to one embodiment, a personalized audio provision system based on seat recognition using artificial intelligence is provided, comprising: one or more AI (Artificial Intelligence) cameras installed inside a vehicle to recognize whether a passenger is seated, face position, face direction, and passenger situation information for each seat; directional speakers installed at positions corresponding to each seat within the vehicle and rotating in the up-down and left-right directions according to the face position and face direction of the passenger recognized by the AI ​​cameras; and a speaker control unit that receives an AVN (Audio Video Navigation) audio signal currently being played from an AVN system within the vehicle and provides it to the directional speakers corresponding to each seat, wherein when an individual audio signal is received via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, the output of the directional speakers corresponding to each seat is individually controlled differently for each seat based on the seat position of the passenger terminal that transmitted the audio signal and the passenger situation information recognized through the AI ​​cameras.

[0011] The AI ​​camera is configured to be installed at different locations within the vehicle to photograph the front and rear seats respectively, and to recognize the face position and face direction of the occupant in each seat by recognizing the body feature points of the occupant in each seat after determining whether the occupant is seated in each seat, and the directional speaker may be configured to rotate in the up-down and left-right directions according to the relative position between the face position of the occupant recognized by the AI ​​camera and the reference position set for each seat, and the rotation direction and rotation angle may be adjusted in real time according to the face direction of the occupant recognized by the AI ​​camera.

[0012] The speaker control unit separates the AVN audio signal received from the AVN system into a navigation audio signal and a non-navigation audio signal, and then provides the navigation audio signal and the non-navigation audio signal to a directional speaker corresponding to the driver's seat of the vehicle, and provides the non-navigation audio signal to a directional speaker corresponding to a seat other than the driver's seat. When the speaker control unit receives an individual audio signal via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, the non-navigation audio signal output from the directional speaker corresponding to the seat of the passenger terminal that transmitted the individual audio signal can be replaced with the individual audio signal.

[0013] The AI ​​camera identifies the passenger for each seat using the facial features of the passenger for each seat, and the speaker control unit can activate an automatic pairing function to enable short-range wireless communication between the directional speaker corresponding to the seat of the identified passenger and the passenger terminal carried by the identified passenger when the information of the identified passenger matches the previously stored user information, and at the same time automatically load audio output profile information previously learned for the identified passenger and apply it to the directional speaker corresponding to the seat of the identified passenger.

[0014] The AI ​​camera analyzes the captured video of each seat based on the artificial intelligence model to identify the gender, age group, speech status, and behavior of the occupant for each seat, and classifies the occupant situation information into one of the following based on the gender, age group, speech status, and behavior of the occupant for each seat: an infant boarding situation, a occupant conversation situation, a occupant sleeping situation, a occupant calling situation, and a occupant disembarking situation. The speaker control unit can individually control the output of the directional speaker corresponding to each seat differently for each seat by considering the occupant situation information.

[0015] The speaker control unit collects external vehicle environment information including two or more of the vehicle's driving speed, whether it has entered a tunnel, the window opening / closing status for each seat, and current weather information from a sensor provided in the vehicle, and combines the occupant situation information and the external vehicle environment to individually control the output of the directional speaker corresponding to each seat differently for each seat.

[0016] The speaker control unit can analyze the coordinate change pattern of the face position of each occupant detected by the AI ​​camera during a set time period to predict the listening position of each occupant and preemptively adjust the rotation direction and rotation angle of the directional speaker according to the listening position.

[0017] The speaker control unit can estimate the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle based on the geometric relationship between the audio output direction and output intensity of the directional speaker corresponding to the passenger seat of the vehicle and the driver's seat of the vehicle, calculate the risk of driving interference based on the driving state of the vehicle and the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle, and control the output of the directional speaker corresponding to the passenger seat according to the risk of driving interference.

[0018] The speaker control unit can dynamically control the output of the directional speaker by transmitting an output control signal corresponding to a specific gesture to the directional speaker corresponding to the seat where the passenger is seated when the passenger takes a predetermined specific gesture.

[0019] According to another embodiment, a method for providing personalized audio based on seat recognition using artificial intelligence is provided, comprising: a step of recognizing whether a passenger is seated, a face position, a face direction, and passenger situation information for each seat in one or more AI (Artificial Intelligence) cameras installed inside a vehicle; a step of rotating a directional speaker installed at a position corresponding to each seat in the vehicle in the up-down and left-right directions according to the face position and face direction of the passenger recognized by the AI ​​camera; a step of receiving an AVN (Audio Video Navigation) audio signal currently being played from an AVN (Audio Video Navigation) system in the vehicle in a speaker control unit and providing it to a directional speaker corresponding to each seat; and a step of, in the speaker control unit, receiving an individual audio signal via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, individually controlling the output of the directional speaker corresponding to each seat differently for each seat based on the seat position of the passenger terminal that transmitted the audio signal and the passenger situation information recognized through the AI ​​camera.

[0020] The AI ​​camera is configured to be installed at different locations within the vehicle to photograph the front and rear seats respectively, and to recognize the face position and face direction of the occupant in each seat by recognizing the body feature points of the occupant in each seat after determining whether the occupant is seated in each seat, and the directional speaker may be configured to rotate in the up-down and left-right directions according to the relative position between the face position of the occupant recognized by the AI ​​camera and the reference position set for each seat, and the rotation direction and rotation angle may be adjusted in real time according to the face direction of the occupant recognized by the AI ​​camera.

[0021] The step of individually controlling the output of a directional speaker corresponding to each of the above seats differently for each of the above seats comprises separating the AVN audio signal received from the AVN system into a navigation audio signal and a non-navigation audio signal, providing the navigation audio signal and the non-navigation audio signal to the directional speaker corresponding to the driver's seat of the vehicle, providing the non-navigation audio signal to the directional speaker corresponding to a seat other than the driver's seat, and, when an individual audio signal is received via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, the non-navigation audio signal output from the directional speaker corresponding to the seat of the passenger terminal that transmitted the individual audio signal can be replaced with the individual audio signal.

[0022] The method for providing personalized audio based on seat recognition using artificial intelligence described above may further include, prior to the step of individually controlling the output of a directional speaker corresponding to each seat differently for each seat, a step of identifying a passenger for each seat using facial features of the passenger for each seat in the AI ​​camera; and, in the speaker control unit, if the information of the identified passenger matches previously stored user information, a step of activating an automatic pairing function to enable short-range wireless communication between the directional speaker corresponding to the seat of the identified passenger and a passenger terminal carried by the identified passenger, while simultaneously automatically loading audio output profile information previously learned for the identified passenger and applying it to the directional speaker corresponding to the seat of the identified passenger.

[0023] The step of recognizing the passenger's boarding status, face position, face direction, and passenger situation information for each seat above involves analyzing a captured video of each seat based on the artificial intelligence model to identify the gender, age group, speech status, and behavior of the passenger for each seat, and classifying the passenger situation information into one of an infant boarding situation, a passenger conversation situation, a passenger sleeping situation, a passenger phone call situation, and a passenger disembarking situation based on the gender, age group, speech status, and behavior of the passenger for each seat.

[0024] The method for providing personalized audio based on seat recognition using artificial intelligence described above may further include: a step in which the speaker control unit collects a vehicle external environment including two or more of the vehicle's driving speed, whether it has entered a tunnel, the window opening / closing status for each seat, and current weather information from a sensor provided in the vehicle; and a step in which the speaker control unit combines the occupant situation information and the vehicle external environment to individually control the output of a directional speaker corresponding to each seat differently for each seat.

[0025] The method for providing personalized audio based on seat recognition using artificial intelligence described above may further include: a step of predicting the listening position of each occupant by analyzing the coordinate change pattern of the face position of each occupant detected by the AI ​​camera for a set period of time in the speaker control unit; and a step of preemptively adjusting the rotation direction and rotation angle of the directional speaker according to the listening position in the speaker control unit.

[0026] The method for providing personalized audio based on seat recognition using artificial intelligence described above may further include: a step of estimating the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle based on the geometric relationship between the audio output direction and output intensity of a directional speaker corresponding to the passenger seat of the vehicle and the driver's seat of the vehicle, in the speaker control unit; a step of calculating a driving interference risk based on the driving state of the vehicle and the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle, in the speaker control unit; and a step of controlling the output of the directional speaker corresponding to the passenger seat according to the driving interference risk, in the speaker control unit.

[0027] The method for providing personalized audio based on seat recognition using artificial intelligence described above may further include, after the step of individually controlling the output of a directional speaker corresponding to each seat differently for each seat, a step of dynamically controlling the output of the directional speaker by transmitting an output control signal corresponding to a specific gesture to the directional speaker corresponding to the seat occupied by the passenger when the passenger takes a predetermined specific gesture in the speaker control unit. Effects of the invention

[0029] According to embodiments of the present invention, by recognizing in real time whether a passenger is seated, their face position, face direction, and passenger situation information for each seat through an AI camera installed inside a vehicle, and actively controlling the rotation direction of a directional speaker corresponding to each seat based on this, it is possible to provide customized sound that precisely corresponds to the passenger's actual listening position. Furthermore, by combining individual audio signals input from terminals carried by each passenger, as well as audio signals played by the vehicle's AVN system, with seat position information to control output differently for each seat, it is possible to minimize acoustic interference even when multiple passengers simultaneously use different content within the same vehicle. In particular, by reflecting the passenger situation information recognized by the AI ​​camera in the output control, it is possible to implement a personalized audio environment that adaptively responds to the passenger's state, going beyond simple seat separation, thereby simultaneously enhancing user convenience and immersion inside the vehicle.

[0030] In addition, according to embodiments of the present invention, after identifying a passenger for each seat, if the information of the identified passenger matches the previously stored user information, an automatic pairing function is activated to enable short-range wireless communication between the directional speaker corresponding to the seat of the identified passenger and the passenger terminal carried by the identified passenger, and at the same time, audio output profile information previously learned for the identified passenger is automatically loaded and applied to the directional speaker corresponding to the seat of the identified passenger, thereby providing customized personalized audio for each seat that reflects the passenger's usual preferences.

[0031] In addition, according to embodiments of the present invention, when it is expected that the audio output from the directional speaker will not be clearly audible to the occupant based on vehicle situation information determined by a sensor provided in the vehicle, the volume of the directional speaker is automatically increased, but the volume of the directional speaker is reduced or blocked in consideration of the occupant situation information, thereby enabling the output of audio that comprehensively considers the occupant situation information and the vehicle situation information.

[0032] In addition, according to embodiments of the present invention, beyond simply separating audio by seat, the influence of audio from other seats transmitted to the driver's seat is quantitatively estimated, and the output of audio from other seats is dynamically controlled based on this, thereby enabling independent audio usage by multiple passengers while protecting the driver's concentration on driving. Brief explanation of the drawing

[0034] FIG. 1 is a block diagram showing the detailed configuration of a personalized audio providing system according to an embodiment of the present invention. FIG. 2 is an example of the installation of an AI camera and a directional speaker according to an embodiment of the present invention. FIG. 3 is an example showing a directional speaker according to an embodiment of the present invention. FIG. 4 is a diagram illustrating the process of an AVN audio signal being input to a directional speaker for each seat according to an embodiment of the present invention. FIG. 5 is an example of an audio signal input to a directional speaker for each seat according to an embodiment of the present invention. FIG. 6 is an example of the operation of a personalized audio providing system according to an embodiment of the present invention. FIG. 7 is a drawing for explaining the process of controlling the output of a directional speaker for each seat according to passenger situation information recognized by an AI camera according to an embodiment of the present invention. FIG. 8 is a drawing illustrating the process of controlling the output of directional speakers for each seat according to passenger situation information and vehicle situation information according to an embodiment of the present invention. FIG. 9 is a drawing showing an example in which a personalized audio providing system according to an embodiment of the present invention is applied to a bus. FIG. 10 is a flowchart illustrating a method for providing personalized audio according to an embodiment of the present invention. FIG. 11 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments. Specific details for implementing the invention

[0035] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. The following detailed description is provided to facilitate a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, this is merely illustrative and the present invention is not limited thereto.

[0036] In describing the embodiments of the present invention, detailed descriptions of known technologies related to the present invention are omitted if it is determined that such detailed descriptions may unnecessarily obscure the essence of the present invention. Furthermore, the terms described below are defined in consideration of their functions within the present invention, and these may vary depending on the intentions or practices of the user or operator. Therefore, such definitions should be based on the content throughout this specification. Terms used in the detailed description are intended merely to describe the embodiments of the present invention and should not be limiting in any way. Unless explicitly stated otherwise, expressions in the singular form include the meaning of the plural form. In this description, expressions such as "include" or "comprise" are intended to refer to certain characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof, and should not be interpreted to exclude the existence or possibility of one or more other characteristics, numbers, steps, actions, elements, parts thereof, or combinations thereof other than those described.

[0038] FIG. 1 is a block diagram showing the detailed configuration of a personalized audio providing system (100) according to an embodiment of the present invention. As shown in FIG. 1, the personalized audio providing system (100) according to an embodiment of the present invention is provided in a vehicle (not shown) to provide personalized audio to passengers in each seat within the vehicle, and includes an AI camera (110), a directional speaker (120), and a speaker control unit (130). At this time, the vehicle is sufficient to be a means of transportation having two or more seats, and the type of vehicle and the number of seats are not particularly limited.

[0039] One or more AI cameras (110) are installed inside the vehicle to recognize whether a passenger is seated in each seat, the position of the face, the direction of the face, and the situational information of the passenger. The AI ​​cameras (110) may be installed at different locations within the vehicle to photograph the front and rear seats, respectively. At this time, the AI ​​cameras (110) may be equipped with an artificial intelligence model. The artificial intelligence model may include models capable of object detection, face recognition, pose estimation, eye tracking, and behavior recognition, or a combination thereof. In these embodiments, the type of artificial intelligence model embedded in the AI ​​camera or the type of algorithm provided in the artificial intelligence model is not particularly limited.

[0040] The AI ​​camera (110) can photograph each seat and analyze the captured image of each seat based on an artificial intelligence model to recognize whether a passenger is seated in each seat, the position of the face, the direction of the face, and the situational information of the passenger. In these embodiments, the position of the face refers to the two-dimensional or three-dimensional spatial coordinates of the passenger's face region or face center point detected based on image information. Additionally, the direction of the face refers to the orientation defined according to the posture of the passenger's face or head, and means a direction that can be expressed as a direction vector of the face center axis. Specifically, the AI ​​camera (110) may be configured to be installed at different locations within the vehicle to photograph the front and rear seats of the vehicle, respectively, and to recognize the position of the passenger's face and the direction of the face of each passenger by recognizing the physical feature points of the passenger in each seat after determining whether a passenger is seated in each seat. For example, the AI ​​camera (110) can recognize body feature points for the face, shoulders, eyes, nose, ears, etc. of each passenger in each seat, and estimate the face position and face direction of each passenger in each seat based on the recognized body feature points. In this process, the AI ​​camera (110) can utilize object detection algorithms, face recognition algorithms, pose estimation algorithms, eye tracking algorithms, and behavior recognition algorithms included in the artificial intelligence model.

[0041] Additionally, the AI ​​camera (110) analyzes the captured video of each seat based on an artificial intelligence model to identify the gender, age group, speech status, and behavior of the passenger in each seat, and can recognize passenger situation information based on the gender, age group, speech status, and behavior of the passenger in each seat. Specifically, the AI ​​camera (110) can identify the gender, age group, speech status, and behavior of the passenger in each seat by utilizing object detection algorithms, face recognition algorithms, posture estimation algorithms, eye tracking algorithms, and behavior recognition algorithms included in the artificial intelligence model, and can recognize passenger situation information by comparing the gender, age group, speech status, and behavior of the passenger in each seat with previously learned training data. At this time, the AI ​​camera (110) can classify the passenger situation information into one of the following based on the gender, age group, speech status, and behavior of the passenger in each seat: a situation where an infant boards, a situation where the passenger converses, a situation where the passenger sleeps, a situation where the passenger makes a phone call, and a situation where the passenger disembarks.

[0042] As an example, the AI ​​camera (110) can determine the passenger situation information as a situation where an infant is on board based on at least one of the passenger's face area size, body proportions, and whether a car seat is present in the seat.

[0043] As another example, the AI ​​camera (110) can determine the passenger situation information as a passenger conversation situation based on whether there are multiple passengers in the vehicle and whether the passengers speak.

[0044] As another example, the AI ​​camera (110) detects the eye area of ​​the passenger and can determine the passenger situation information as a passenger sleeping situation if the time the passenger keeps their eyes closed lasts longer than a first threshold or the blinking cycle of the eyes for a set time exceeds a second threshold.

[0045] As another example, the AI ​​camera (110) can determine the passenger situation information as a passenger call situation based on at least one of whether the passenger speaks, the position of the passenger's hand, the passenger's gesture, and the position of the passenger terminal.

[0046] As another example, the AI ​​camera (110) can determine the passenger situation information as a passenger disembarking situation based on at least one of whether the vehicle door adjacent to the seat is open or closed and the passenger's movement.

[0047] The directional speaker (120) is installed at a position corresponding to each seat in the vehicle and rotates in the up-down and left-right directions according to the position and direction of the passenger's face recognized by the AI ​​camera (110). In the embodiments, the directional speaker (120) is a speaker configured to concentrate and radiate audio to a specific direction or a specific spatial area, unlike a general omnidirectional speaker, and is structured to limit or control the directional angle of the audio output. The directional speaker (120) can concentrate and radiate audio to a specific direction or a specific spatial area by using, for example, at least one of a physical direction control method, an audio beamforming method, or a super-directional (parametric) audio generation method. To this end, the directional speaker (120) may be equipped with an audio output module, a directional angle control module, a driving unit (including a rotary motor), etc.

[0048] As described above, the directional speaker (120) can be installed at a location corresponding to each seat in the vehicle. As an example, the directional speaker (120) can be installed on at least one of the ceiling of each seat in the vehicle, the front top of each seat in the vehicle, the door trim, and the headrest, and can be rotated to face the passenger seated in the designated seat. That is, the directional speaker (120) is provided for each seat and can be rotated up and down and left and right according to the face position and face direction of the passenger recognized by the AI ​​camera (110).

[0049] To this end, the directional speaker (120) may be equipped with an LED pointer (not shown) directed toward the front of the directional speaker (120). This LED pointer can be used as a visual reference to set the output direction of the directional speaker (120). The directional speaker (120) can illuminate the LED to a reference position set for each seat (e.g., the center of the headrest) through the LED pointer, and the rotation direction and rotation angle of the directional speaker (120) when the LED reaches the reference position can be stored as a zero point (i.e., a reference point). Subsequently, the rotation control of the directional speaker (120) can be performed in a relative angle manner based on the zero point, thereby correcting the installation error or mounting deviation of the directional speaker (120).

[0050] Subsequently, the AI ​​camera (110) can recognize the face position and face direction of each occupant for each seat, calculate the relative position between the recognized occupant's face position and the reference position for each seat, for example, the center of the headrest, in a predefined vehicle reference coordinate system or seat reference coordinate system, and then transmit the relative position and the occupant's face direction to the directional speaker (120). In this case, the directional speaker (120) can be rotated up and down and left and right according to the previously calculated relative position so that the audio output from the directional speaker (120) is directed toward the occupant's face position (for example, the exact center of the face). Additionally, the directional speaker (120) can adjust the rotation direction and rotation angle in real time according to the occupant's face direction recognized by the AI ​​camera (110). That is, the directional speaker (120) can set the rotation direction and rotation angle based on the initially set zero point and then adjust the rotation direction and rotation angle in real time according to the face position and face direction of the passenger recognized by the AI ​​camera (110), and accordingly, even if the passenger changes their posture or moves, the audio output can be accurately directed toward the passenger's face.

[0051] The speaker control unit (130) dynamically controls the output of the directional speaker (120) corresponding to each seat based on passenger situation information recognized by the AI ​​camera (110). In the embodiments, the meaning of controlling the output of the directional speaker (120) is used in a broad sense that includes all of the following: controlling the output volume of the directional speaker (120), controlling the output on / off of the directional speaker (120), pausing the output of the directional speaker (120), sharing audio output from the directional speaker (120), and switching the mode of the directional speaker (120). Furthermore, it is sufficient for the speaker control unit (130) to be configured to control each directional speaker (120) individually, and its configuration form is not particularly limited, such as being provided with multiple units for each seat or as a single configuration.

[0052] At this time, the speaker control unit (130) can control the output of the directional speaker corresponding to each seat by linking with the AVN (Audio Video Navigation) system equipped in the vehicle. Specifically, the speaker control unit (130) receives the AVN audio signal currently being played from the AVN system in the vehicle and provides it to the directional speaker corresponding to each seat. However, when receiving an individual audio signal via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, the output of the directional speaker corresponding to each seat can be individually controlled differently for each seat based on the seat location of the passenger terminal that transmitted the audio signal and passenger situation information recognized through the AI ​​camera. The output control method in this speaker control unit (130) will be described in more detail later with reference to the drawings below.

[0053] FIG. 2 is a drawing showing an example of installation of an AI camera (110) and a directional speaker (120) according to one embodiment of the present invention.

[0054] As illustrated in FIG. 2, the AI ​​camera (110) can be installed at different locations within the vehicle to photograph the front seats and rear seats, respectively. As an example, the AI ​​camera (110) can be installed at the center ceiling of the front seats and the center ceiling of the rear seats, respectively. That is, as the AI ​​camera (110) is installed at the first row seats and the second row seats of the vehicle, the blind spots in the shooting range can be minimized. Additionally, the AI ​​camera (110) can be equipped with a wide-angle lens so that it can photograph and monitor two passengers per AI camera (110). However, the installation location of the AI ​​camera (110) is not limited to this, and the AI ​​camera (110) can be installed at various locations such as the vehicle's overhead console, rearview mirror, headrest, etc.

[0055] Additionally, the directional speaker (120) may be installed at a location corresponding to each seat in the vehicle. As an example, the directional speaker (120) may be installed on at least one of the ceiling of each seat in the vehicle, the front top of each seat in the vehicle, the door trim, and the headrest. At this time, the directional speaker (120) may be configured to rotate in the up-down and left-right directions according to the relative position between the passenger's face position recognized by the AI ​​camera (110) and the reference position set for each seat, and the rotation direction and rotation angle may be adjusted in real time according to the direction of the passenger's face recognized by the AI ​​camera (110).

[0056] FIG. 3 is an example showing a directional speaker (120) according to one embodiment of the present invention.

[0057] Referring to FIG. 3, the directional speaker (120) includes a fixed plate (310), a rotating support (320), a speaker body (330), and a driving part (340).

[0058] The fixing plate (310) is fixed to at least one of the ceiling of each seat in the vehicle, the front upper part of each seat in the vehicle, the door trim, and the headrest. The fixing plate (310) can be fixed to at least one of the ceiling of each seat in the vehicle, the front upper part of each seat in the vehicle, the door trim, and the headrest through various methods such as bolt fastening, bracket coupling, and integral molding, and can support the speaker body (330).

[0059] The rotational support member (320) is provided between the fixed plate (310) and the speaker body (330) to support the speaker body (330) so that the speaker body (330) can rotate within a set angle range. The rotational support member (320) may include at least one of a hinge structure, a ball joint structure, and a bearing structure so that the speaker body (330) can rotate in the up-down and left-right directions. Additionally, the rotational support member (320) may perform physical rotation and electronic beamforming control in parallel.

[0060] The speaker body (330) is a part that emits audio and may be equipped with a diaphragm, voice coil, magnet unit, etc. for audio output. Additionally, an opening for audio emission may be formed on the front of the speaker body (330). The speaker body (330) may be rotated up and down and left and right by a motor (not shown) provided inside the driving unit (340).

[0061] The driving unit (340) is equipped with a motor inside and rotates the speaker body (330) in the up-down and left-right directions through the motor. The motor may be, for example, a servo motor, a stepping motor, a gear motor, etc., but the type and operation method of the motor are not particularly limited. This driving unit (340) may be formed in a spherical or hemispherical shape.

[0062] In this way, the directional speaker (120) is installed at a position corresponding to each seat in the vehicle and can be rotated up and down and left and right according to the position and direction of the passenger's face recognized by the AI ​​camera (110). At this time, the speaker control unit (130) can dynamically control the output of the directional speaker (120) corresponding to each seat based on the passenger situation information recognized by the AI ​​camera (110). To this end, the speaker control unit (130) can receive the AVN audio signal currently being played from the AVN system in the vehicle and provide it to the directional speaker corresponding to each seat.

[0063] FIG. 4 is a diagram illustrating the process of an AVN audio signal being input to a directional speaker for each seat according to an embodiment of the present invention.

[0064] Referring to FIG. 4, the speaker control unit (130) separates the AVN audio signal received from the AVN system into a navigation audio signal and a non-navigation audio signal, and then provides the navigation audio signal and the non-navigation audio signal to the directional speaker (120) corresponding to the driver's seat of the vehicle, and provides the non-navigation audio signal to the directional speaker (120) corresponding to a seat other than the driver's seat. Here, the navigation audio signal is an audio signal output through the in-vehicle navigation, and may be, for example, a navigation route guidance audio signal, a navigation speed limit notification audio signal, a navigation rest area guidance audio signal, etc. Additionally, the non-navigation audio signal is an audio signal that is not a navigation audio signal among the AVN audio signals received from the AVN system, and may be, for example, an audio signal output from the in-vehicle radio device, an audio signal output from the in-vehicle highway toll terminal, etc. Generally, navigation audio signals are intended for the driver, and there may be a relatively low need to provide them to passengers in seats other than the driver's seat. Accordingly, the speaker control unit (130) may provide navigation audio signals and non-navigation audio signals to the directional speaker (120) corresponding to the driver's seat of the vehicle, and provide only non-navigation audio signals to the directional speaker (120) corresponding to a seat other than the driver's seat.

[0065] Additionally, when the speaker control unit (130) receives an individual audio signal via short-range wireless communication from at least one of the passenger terminals carried by each passenger in each seat, it can replace the non-navigation audio signal output from the directional speaker (120) corresponding to the seat of the passenger terminal that transmitted the individual audio signal with the individual audio signal. Here, the short-range wireless communication may be, for example, Bluetooth communication, Wi-Fi communication, etc. Additionally, the passenger terminal may be a smartphone, tablet PC, laptop, etc. carried by the passenger.

[0066] FIG. 5 is an example of an audio signal input to each seat-specific directional speaker (120) according to one embodiment of the present invention, and FIG. 6 is an example of the operation of a personalized audio providing system (100) according to one embodiment of the present invention.

[0067] Referring to FIG. 5, the speaker control unit (130) can input a navigation audio signal among the AVN audio signals received from the AVN system to a directional speaker (120) corresponding to the driver's seat of the vehicle. Additionally, the speaker control unit (130) can input a non-navigation audio signal among the AVN audio signals received from the AVN system to a directional speaker (120) corresponding to all driver's seats in the vehicle. At this time, when the speaker control unit (130) receives an individual audio signal via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, the non-navigation audio signal output from the directional speaker (120) corresponding to the seat of the passenger terminal that transmitted the individual audio signal can be replaced with the individual audio signal.

[0068] As an example, when the speaker control unit (130) receives an individual audio signal via Bluetooth communication from a passenger terminal held by a passenger in the passenger seat, it can replace the non-navigation audio signal output from the directional speaker (120) corresponding to the passenger seat with the individual audio signal. Here, the individual audio signal is an audio signal output from the passenger terminal held by the passenger, and may be, for example, a video audio signal, a song audio signal, a call audio signal, etc. By replacing the non-navigation signal with an individual audio signal, passengers in each seat within the vehicle can individually select and listen to the individual audio signal they desire, rather than the AVN audio signal received from the AVN system. In this case, independent media playback for each seat within the vehicle, that is, the provision of personalized audio, becomes possible.

[0069] At this time, the AI ​​camera (110) can identify the passenger for each seat by utilizing the facial features of the passenger for each seat. As an example, the AI ​​camera (110) can identify the passenger for each seat by photographing the passenger's face whenever the passenger boards each seat, extracting the passenger's facial features, and comparing the extracted facial features with the facial features in the previously stored user information. The AI ​​camera (110) can, for example, determine a passenger with facial features that are repeatedly detected more than a set number of times as an existing passenger of the vehicle, and determine a passenger with facial features that are detected for the first time as a new passenger of the vehicle. Here, the existing passenger is a passenger who repeatedly boards the vehicle, and may be, for example, the driver, the driver's family, or a coworker. In this way, the AI ​​camera (110) can determine whether the passenger is an existing passenger of the vehicle or a new passenger based on the passenger's facial features whenever the passenger boards each seat.

[0070] The speaker control unit (130) can activate an automatic pairing function to enable short-range wireless communication between the passenger terminal carried by the identified passenger and the directional speaker (120) corresponding to the seat of the identified passenger when the information of the identified passenger matches the user information stored in this way.

[0071] As an example, the speaker control unit (130) can activate the Bluetooth automatic pairing function of the directional speaker (120) so that Bluetooth communication between the directional speaker (120) corresponding to the driver's seat and the passenger terminal held by the passenger in the driver's seat is possible when the information of the passenger in the driver's seat matches the previously stored user information. As another example, the speaker control unit (130) can activate the Bluetooth automatic pairing function of the directional speaker (120) so that Bluetooth communication between the directional speaker (120) corresponding to the passenger seat and the passenger terminal held by the passenger in the passenger seat is possible when the information of the passenger in the passenger seat matches the previously stored user information.

[0072] At this time, the speaker control unit (130) can activate an automatic pairing function to enable short-range wireless communication with a passenger terminal carried by the identified passenger, and at the same time, automatically load audio output profile information learned for the identified passenger and apply it to a directional speaker (120) corresponding to the seat of the identified passenger. In these embodiments, the audio output profile information may be information learned according to each passenger's output control command, such as equalizer information like the volume level of the audio output preferred by the identified passenger and frequency characteristics, and information regarding control rules for the audio output preferred by the identified passenger (e.g., volume reduction of audio output when performing a hand-raising gesture, audio output blocking during conversation, etc.). To this end, each passenger may input various output control commands related to audio output while seated, and the speaker control unit (130) may recognize passenger situation information in conjunction with the AI ​​camera (110) during this process, match it with the output control command, and store it.

[0073] As one example, the speaker control unit (130) learns audio output profile information such as "Passenger A's conversation situation - audio output blockage" when Passenger A inputs a command to block audio output while talking to Passenger B in the adjacent seat, and can automatically block audio output without Passenger A's output control command when Passenger A attempts to talk the next time. As another example, the speaker control unit (130) learns audio output profile information such as "Passenger C - audio volume increase by 3" when Passenger C inputs a command to repeatedly increase the volume of a predetermined audio output by 3, and can automatically increase the volume of audio output by 3 without Passenger C's output control command when Passenger C boards.

[0074] Accordingly, according to embodiments of the present invention, after identifying a passenger for each seat, if the information of the identified passenger matches the previously stored user information, an automatic pairing function is activated to enable short-range wireless communication between the directional speaker corresponding to the seat of the identified passenger and the passenger terminal carried by the identified passenger, and at the same time, audio output profile information previously learned for the identified passenger is automatically loaded and applied to the directional speaker corresponding to the seat of the identified passenger, thereby providing customized personalized audio for each seat that reflects the passenger's usual preferences.

[0075] Additionally, when the speaker control unit (130) receives an individual audio signal via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, it can individually control the output of the directional speaker corresponding to each seat differently for each seat based on the seat location of the passenger terminal that transmitted the audio signal and passenger situation information recognized through the AI ​​camera (110).

[0076] FIG. 7 is a diagram illustrating the process of controlling the output of a directional speaker for each seat according to passenger situation information recognized by an AI camera (110) according to an embodiment of the present invention.

[0077] Referring to FIG. 7, the AI ​​camera (110) analyzes the captured video of each seat based on an artificial intelligence model to identify the gender, age group, speech status, and behavior of the passenger for each seat, and classifies the passenger situation information into one of the following: infant boarding situation, passenger conversation situation, passenger sleeping situation, passenger phone call situation, and passenger disembarking situation. Subsequently, the speaker control unit (130) can individually control the output of the directional speaker (120) corresponding to each seat differently for each seat, taking into account the passenger situation information.

[0078] As an example, the speaker control unit (130) can reduce the maximum volume of the directional speaker (120) corresponding to the seat where the infant is seated from 10 to 7 when it is determined that the passenger situation information is that the infant is seated. In addition, in this case, the AI ​​camera (110) monitors the condition of the infant in real time and detects when the infant cries or makes a predetermined expression, for example, a distorted expression, and transmits the relevant information to the speaker control unit (130), and the speaker control unit (130) can provide voice guidance on the situation of the infant through the directional speaker (120) corresponding to the seat where the driver is seated.

[0079] As another example, if the speaker control unit (130) determines that the passenger situation information is a passenger conversation situation or a passenger call situation, it can reduce or block the volume of the directional speaker (120) corresponding to the seat of the passenger attempting to have a conversation or call by a predetermined amount. Subsequently, when the passenger's conversation or call ends, the speaker control unit (130) can restore the reduced or blocked volume of the directional speaker (120) to its original level.

[0080] As another example, the speaker control unit (130) can increase the volume of the directional speaker (120) corresponding to the passenger's seat to the maximum and trigger an alarm to the driver when the passenger in the sleeping state is determined to be in a sleeping state (or a drowsy state) and when the passenger in the sleeping state is not the driver, it can decrease or block the volume of the directional speaker (120) corresponding to the passenger's seat by a predetermined amount.

[0081] As another example, the speaker control unit (130) can block the output of the directional speaker (120) corresponding to the seat of the passenger who is disembarking when the passenger situation information is determined to be a passenger disembarking situation.

[0082] Additionally, the speaker control unit (130) can control the output of the directional speaker (120) by transmitting an output control signal corresponding to a specific gesture to the directional speaker (120) corresponding to the seat where the passenger is seated when the passenger takes a predetermined specific gesture. For example, when the passenger takes a gesture of raising their hand, the volume of the audio output from the directional speaker (120) corresponding to the seat where the passenger is seated increases, and when the passenger takes a gesture of lowering their hand, the volume of the audio output from the directional speaker (120) corresponding to the seat where the passenger is seated decreases.

[0083] In addition, the speaker control unit (130) may comprehensively consider not only the passenger situation information described above but also the vehicle situation information to individually control the output of the directional speaker (120) corresponding to each seat differently for each seat.

[0084] FIG. 8 is a diagram illustrating the process of controlling the output of each seat-specific directional speaker (120) according to passenger situation information and vehicle situation information according to one embodiment of the present invention.

[0085] Referring to FIG. 8, the speaker control unit (130) can combine passenger situation information and vehicle situation information to individually control the output of the directional speaker (120) corresponding to each seat differently for each seat. Here, the vehicle situation information is information determined based on sensor data collected from a sensor provided in the vehicle, and may include, for example, two or more of the following: the vehicle's driving speed, whether it has entered a tunnel, the window opening / closing status for each seat, the condition information of the road surface on which the vehicle is driving, and the current weather information. In the embodiments, the sensor may be a speed sensor, a position sensor, a camera, a lidar, a radar, an ultrasonic sensor, etc., but the type of sensor is not particularly limited.

[0086] As an example, the speaker control unit (130) can increase the volume of the directional speaker (120) corresponding to the seat where the passenger is seated when the vehicle enters a tunnel while the passenger is seated, but can decrease or block the volume of the directional speaker (120) corresponding to the seat where the passenger is seated even if the vehicle is currently traveling in a tunnel if the passenger situation information corresponds to at least one of the following: a situation where an infant is seated, a situation where the passenger is talking, a situation where the passenger is sleeping, a situation where the passenger is talking, and a situation where the passenger is disembarking.

[0087] As another example, the speaker control unit (130) can increase the volume of the directional speaker (120) corresponding to the seat where the passenger is seated when the vehicle's driving speed exceeds a reference value, when the window of the seat where the passenger is seated is open, and when it is currently raining or thundering with lightning, but can decrease or block the volume of the directional speaker (120) corresponding to the seat where the passenger is seated when the passenger situation information corresponds to at least one of the following: a situation where an infant is seated, a situation where the passenger is talking, a situation where the passenger is sleeping, a situation where the passenger is talking, and a situation where the passenger is disembarking.

[0088] In this way, according to embodiments of the present invention, when it is expected that the audio output from the directional speaker (120) will not be clearly heard by the passenger based on vehicle situation information determined by a sensor provided in the vehicle, the volume of the directional speaker (120) is automatically increased, but the volume of the directional speaker (120) is reduced or blocked in consideration of the passenger situation information, thereby enabling the output of audio that comprehensively considers the passenger situation information and the vehicle situation information.

[0089] Additionally, the speaker control unit (130) can analyze the coordinate change pattern of the face position of each passenger in each seat detected by the AI ​​camera (110) during a set time period to predict the listening position of each passenger in each seat, and can preemptively adjust the rotation direction and rotation angle of the directional speaker (120) according to the listening position. Generally, the sitting state, posture, and movement patterns of each passenger may differ, and there is a need for the audio output from the directional speaker (120) to change according to the passenger's posture change or movement. Accordingly, the speaker control unit (130) can analyze the posture and movement patterns of each passenger in each seat from the coordinate change pattern of the face position of each passenger in each seat detected by the AI ​​camera (110) during a set time period, and predict the listening position of each passenger in each seat from this. Subsequently, the speaker control unit (130) can preemptively adjust the rotation direction and rotation angle of the directional speaker (120) according to the predicted listening position.

[0090] As an example, when a passenger maintains a posture of looking out the window and repeatedly performs the action of turning their head forward when entering a tunnel, the speaker control unit (130) predicts the listening position to the front of the passenger according to the change in the passenger's face position as the vehicle's driving position changes from inside the tunnel to outside the tunnel, and can preemptively adjust the rotation direction and rotation angle of the directional speaker (120) according to the predicted listening position.

[0091] Additionally, the speaker control unit (130) can estimate the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle based on the geometric relationship between the audio output direction and output intensity from the directional speaker (120) corresponding to the passenger seat of the vehicle and the driver's seat of the vehicle, calculate the risk of driving interference based on the driving state of the vehicle and the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle, and control the output of the directional speaker (120) corresponding to the passenger seat according to the risk of driving interference. Generally, the vehicle has internal structures such as glass, a dashboard, a ceiling, and leather seats, and even if the audio travels straight toward the occupant's face through the directional speaker (120), reflected sound through the internal structures can be transmitted to the driver's seat. In particular, since the vehicle has a sealed structure and the distance between each seat is physically close, the possibility that some sound pressure other than the main beam will reach the driver's seat due to reflection cannot be ruled out. Accordingly, even if a directional speaker (120) is used, sound pressure may be transmitted to the driver's seat due to internal reflection and diffraction of the vehicle, so additional correction work is required.

[0092] To this end, the speaker control unit (130) may be configured to quantitatively estimate the effect of audio output from a directional speaker (120) corresponding to the passenger seat of the vehicle on the driver's seat of the vehicle, and to control the output of the directional speaker (120) corresponding to the passenger seat based thereon. Specifically, the speaker control unit (130) may estimate the expected sound pressure level at the driver's seat location by considering the geometric relationship between the audio output direction and output intensity of the directional speaker (120) corresponding to the passenger seat and the driver's seat location of the vehicle.

[0093] As an example, the speaker control unit (130) calculates distance information between the directional speaker (120) corresponding to the passenger seat and the face of the passenger in the driver's seat, and angle difference information between the audio radiation center axis of the directional speaker (120) corresponding to the passenger seat and the face of the passenger in the driver's seat, respectively, and can calculate the degree of attenuation when the audio radiated from the directional speaker (120) corresponding to the passenger seat reaches the face of the passenger in the driver's seat based on this distance information and angle difference information. At this time, the speaker control unit (130) can estimate the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle in real time based on the audio output intensity, a correction coefficient according to the internal reflection characteristics of the vehicle, etc. Subsequently, the speaker control unit (130) can calculate the risk of driving interference by combining the audio sound pressure from the passenger seat transmitted to the driver's seat and the vehicle's driving state information. Here, the vehicle's driving state information may include, for example, the vehicle's speed, acceleration, steering angle, whether it is changing lanes, whether it is braking suddenly, whether it is activating an autonomous driving mode, etc. As an example, the speaker control unit (130) may assign a higher risk of driving disturbance even at the same sound pressure when the vehicle is driving at high speed or in a situation of sudden steering or sudden braking. In this way, the risk of driving disturbance is not calculated based solely on the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle, but can be calculated by combining the audio sound pressure from the passenger seat transmitted to the driver's seat of the vehicle and the vehicle's driving state information.

[0094] The speaker control unit (130) can calculate the risk of driving interference, for example, according to the following mathematical formula 1. According to the following mathematical formula 1, the risk of driving interference may increase when the audio sound pressure estimated at the driver's seat position increases, when the vehicle's driving speed increases, when a situation of rapid acceleration or rapid deceleration occurs, or when a situation of lane change or rapid turning occurs. At this time, each variable of mathematical formula 1 may be normalized in advance or converted to the same scale and then weighted summed.

[0096] [Mathematical Formula 1]

[0097] R = w1·SPL_d + w2·V + w3·|A| + w4·|δ|

[0098] (Here, R represents the risk of driver distraction, and SPL_d represents the audio sound pressure level estimated at the driver's seat position. Additionally, V represents the vehicle's driving speed, A represents the vehicle's longitudinal acceleration (or acceleration), and δ represents the vehicle's steering angle. Furthermore, w1, w2, w3, and w4 represent the respective set weighting coefficients.)

[0100] The speaker control unit (130) can control the output of the directional speaker (120) corresponding to the passenger seat in stages according to the calculated risk of driving disturbance. As an example, the speaker control unit (130) can reduce the volume of the directional speaker (120) corresponding to the passenger seat when the calculated risk of driving disturbance exceeds a first threshold, switch to a Narrow-Beam Mode that narrows the audio radiation angle of the directional speaker (120) corresponding to the passenger seat when the calculated risk of driving disturbance exceeds a second threshold which is greater than the first threshold, thereby concentrating the audio only in a narrower area, and temporarily block the output of the directional speaker (120) corresponding to the passenger seat for a set period when the calculated risk of driving disturbance exceeds a third threshold which is greater than the second threshold. Here, the Narrow-Beam Mode refers to a mode that reduces the radiation angle of the audio relative to the reference radiation angle and concentrates the radiation into a narrower spatial area. The speaker control unit (130) can minimize listening discomfort for the passenger in the passenger seat caused by output control by gradually controlling the output of the directional speaker (120) corresponding to the passenger seat.

[0101] At this time, the speaker control unit (130) can adaptively adjust the first to third threshold values ​​based on the history of the driver’s driving interference risk accumulated over a long period. As an example, the speaker control unit (130) stores the result of calculating the driving interference risk and the corresponding output control history, and can adaptively adjust the first to third threshold values ​​corresponding to the driving interference risk based on the output control history. For example, if the driver repeatedly manually adjusts the volume at a specific sound pressure level, the speaker control unit (130) can lower the first to third threshold values ​​for that sound pressure range, and conversely, if there is no manual intervention, it can raise the first to third threshold values. The first to third threshold values ​​can be stored individually for each driver, and the first to third threshold values ​​corresponding to the driver can be applied according to the driver identification result. Accordingly, driving interference protection control that adapts to driver characteristics without relying on a fixed reference value is possible.

[0102] As such, according to embodiments of the present invention, beyond simply separating audio by seat, by quantitatively estimating the influence of audio from other seats transmitted to the driver's seat and dynamically controlling the output of audio from other seats based thereon, it is possible to enable independent audio usage by multiple passengers while protecting the driver's concentration on driving.

[0103] The aforementioned personalized audio provision system (100) can be applied not only to general vehicles but also to buses.

[0104] FIG. 9 is a diagram showing an example in which a personalized audio providing system (100) according to one embodiment of the present invention is applied to a bus.

[0105] Referring to FIG. 9, the bus includes multiple seats, and multiple directional speakers (120) may be placed in the ceiling of the vehicle interior. At this time, each directional speaker (120) is installed to face a corresponding seat and may be configured to concentrate and radiate audio to a specific seat area. In this case, passengers on each seat of the bus can individually select and listen to an individual audio signal of their choice, rather than an AVN audio signal received from the AVN system, and accordingly, independent media playback for each seat in the vehicle, that is, the provision of personalized audio, becomes possible.

[0106] FIG. 10 is a flowchart illustrating a method for providing personalized audio according to an embodiment of the present invention. In the illustrated flowchart, the method is described by dividing it into a plurality of steps, but at least some of the steps may be performed in a different order, combined with other steps and performed together, omitted, divided into detailed steps, or performed with one or more steps not illustrated added.

[0107] In step S102, the directional speaker (120) can set an initial position. As an example, the directional speaker (120) can project an LED to a reference position set for each seat (e.g., the center of the headrest) through an LED pointer, and store the rotation direction and rotation angle of the directional speaker (120) when the LED reaches the reference position as a zero point (i.e., a reference point).

[0108] In step S104, the AI ​​camera (110) recognizes whether a passenger is seated in each seat, the position of the face, the direction of the face, and the passenger situation information.

[0109] In step S106, the directional speaker (120) is rotated up and down and left and right according to the position and direction of the passenger's face recognized by the AI ​​camera (110).

[0110] In step S108, the speaker control unit (130) individually controls the output of each seat's directional speaker (120) differently.

[0111] FIG. 11 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments. In the illustrated embodiments, each component may have different functions and capabilities in addition to those described below, and may include additional components in addition to those not described below.

[0112] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may be a personalized audio providing system (100) or one or more components included in the personalized audio providing system (100).

[0113] The computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) can cause the computing device (12) to operate according to the exemplary embodiment described above. For example, the processor (14) can execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, and the computer-executable instructions may be configured to cause the computing device (12) to perform operations according to the exemplary embodiment when executed by the processor (14).

[0114] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by a processor (14). In one embodiment, the computer-readable storage medium (16) may be memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, other forms of storage media that are accessed by a computing device (12) and capable of storing desired information, or a suitable combination thereof.

[0115] The communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and the computer-readable storage medium (16).

[0116] The computing device (12) may also include one or more input / output interfaces (22) and one or more network communication interfaces (26) that provide interfaces for one or more input / output devices (24). The input / output interfaces (22) and network communication interfaces (26) are connected to a communication bus (18). The input / output devices (24) may be connected to other components of the computing device (12) through the input / output interfaces (22). An exemplary input / output device (24) may include an input device such as a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices and / or imaging devices, and / or an output device such as a display device, a printer, a speaker and / or a network card. An exemplary input / output device (24) may be included inside the computing device (12) as a component constituting the computing device (12), or it may be connected to the computing device (12) as a separate device distinct from the computing device (12).

[0118] Although the present invention has been described in detail above through representative embodiments, those skilled in the art will understand that various modifications can be made to the aforementioned embodiments without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof. Explanation of the symbols

[0120] 100: Personalized audio delivery system 110 : AI Camera 120: Directional speaker 310 : Fixed plate 320: Rotating support 330 : Speaker body 340 : Drive unit

Claims

Claim 1 One or more AI (Artificial Intelligence) cameras installed inside a vehicle to recognize whether a passenger is seated, face position, face direction, and passenger situation information for each seat; and directional speakers installed at positions corresponding to each seat inside the vehicle and rotating in the up-down and left-right directions according to the face position and face direction of the passenger recognized by the AI ​​cameras; The system includes a speaker control unit that receives an AVN audio signal currently being played from an AVN (Audio Video Navigation) system within the vehicle and provides it to a directional speaker corresponding to each seat, wherein when an individual audio signal is received via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, the speaker control unit controls the output of the directional speaker corresponding to each seat differently for each seat based on the seat location of the passenger terminal that transmitted the audio signal and the passenger situation information recognized through the AI ​​camera. The speaker control unit separates the AVN audio signal received from the AVN system into a navigation audio signal and a non-navigation audio signal, and then provides the navigation audio signal and the non-navigation audio signal to the directional speaker corresponding to the driver's seat of the vehicle, and provides the non-navigation audio signal to the directional speaker corresponding to a seat other than the driver's seat. When the speaker control unit receives an individual audio signal via short-range wireless communication from at least one of the passenger terminals carried by the passenger of each seat, the individual audio signal A personalized audio provision system based on seat recognition using artificial intelligence, which replaces the non-navigation audio signal output from a directional speaker corresponding to the seat of the transmitted passenger terminal with the individual audio signal. Claim 2 A personalized audio provision system based on seat recognition using artificial intelligence according to claim 1, wherein the AI ​​camera is configured to be installed at different locations within the vehicle to photograph the front seat and the rear seat of the vehicle, respectively, and to recognize the face position and face direction of the occupant in each seat by recognizing the body feature points of the occupant in each seat after determining whether the occupant in each seat is present, and the directional speaker is configured to rotate in the up-down and left-right directions according to the relative position between the face position of the occupant recognized by the AI ​​camera and the reference position set for each seat, and wherein the rotation direction and rotation angle are adjusted in real time according to the face direction of the occupant recognized by the AI ​​camera. Claim 3 delete Claim 4 A personalized audio provision system based on seat recognition using artificial intelligence, wherein, in claim 1, the AI ​​camera identifies the passenger for each seat using the facial features of the passenger for each seat, and the speaker control unit activates an automatic pairing function to enable short-range wireless communication between the directional speaker corresponding to the seat of the identified passenger and the passenger terminal carried by the identified passenger when the information of the identified passenger matches the user information stored in advance, and simultaneously automatically loads audio output profile information learned for the identified passenger and applies it to the directional speaker corresponding to the seat of the identified passenger. Claim 5 A personalized audio provision system based on seat recognition using artificial intelligence according to claim 1, wherein the AI ​​camera analyzes a captured image of each seat based on an artificial intelligence model to identify the gender, age group, speech status, and behavior of the passenger for each seat, and classifies the passenger situation information into one of an infant boarding situation, a passenger conversation situation, a passenger sleeping situation, a passenger calling situation, and a passenger disembarking situation based on the gender, age group, speech status, and behavior of the passenger for each seat, and the speaker control unit individually controls the output of a directional speaker corresponding to each seat differently for each seat in consideration of the passenger situation information. Claim 6 A personalized audio provision system based on seat recognition using artificial intelligence according to claim 1, wherein the speaker control unit dynamically controls the output of the directional speaker by transmitting an output control signal corresponding to a specific gesture to a directional speaker corresponding to the seat occupied by the passenger when the passenger takes a predetermined specific gesture. Claim 7 A step of recognizing the occupant status, face position, face direction, and occupant situation information for each seat using one or more AI (Artificial Intelligence) cameras installed inside a vehicle; a step of rotating a directional speaker installed at a position corresponding to each seat in the vehicle in the up-down and left-right directions according to the face position and face direction of the occupant recognized by the AI ​​camera; and a step of receiving an AVN (Audio Video Navigation) audio signal currently being played from an AVN system inside the vehicle and providing it to a directional speaker corresponding to each seat using a speaker control unit. The speaker control unit includes a step of individually controlling the output of a directional speaker corresponding to each seat differently for each seat based on the seat location of the passenger terminal that transmitted the audio signal and the passenger situation information recognized through the AI ​​camera, when receiving an individual audio signal via near-field wireless communication from at least one of the passenger terminals carried by the passenger of each seat in the speaker control unit; wherein the step of individually controlling the output of a directional speaker corresponding to each seat differently for each seat comprises separating the AVN audio signal received from the AVN system into a navigation audio signal and a non-navigation audio signal, providing the navigation audio signal and the non-navigation audio signal to the directional speaker corresponding to the driver's seat of the vehicle, and providing the non-navigation audio signal to the directional speaker corresponding to a seat other than the driver's seat, and when receiving an individual audio signal via near-field wireless communication from at least one of the passenger terminals carried by the passenger of each seat, the non- A method for providing personalized audio based on seat recognition using artificial intelligence, which replaces a navigation audio signal with the individual audio signal. Claim 8 In claim 7, the AI ​​camera is configured to be installed at different locations within the vehicle to photograph the front seat and rear seat of the vehicle, respectively, and to recognize the face position and face direction of the occupant in each seat by recognizing the body feature points of the occupant in each seat after determining whether the occupant in each seat is present, and the directional speaker is configured to rotate in the up-down and left-right directions according to the relative position between the face position of the occupant recognized by the AI ​​camera and the reference position set for each seat, and the rotation direction and rotation angle are adjusted in real time according to the face direction of the occupant recognized by the AI ​​camera. Claim 9 delete Claim 10 A method for providing personalized audio based on seat recognition using artificial intelligence, further comprising: a step of identifying a passenger for each seat using facial features of the passenger for each seat in the AI ​​camera prior to the step of individually controlling the output of a directional speaker corresponding to each seat differently for each seat; and a step in the speaker control unit of, when the information of the identified passenger matches previously stored user information, activating an automatic pairing function to enable short-range wireless communication between the directional speaker corresponding to the seat of the identified passenger and a passenger terminal carried by the identified passenger, while simultaneously automatically loading audio output profile information previously learned for the identified passenger and applying it to the directional speaker corresponding to the seat of the identified passenger. Claim 11 In claim 7, the step of recognizing whether a passenger is on board, face position, face direction, and passenger situation information for each seat comprises analyzing a captured image of each seat based on an artificial intelligence model to identify the gender, age group, speech status, and behavior of the passenger for each seat, and classifying the passenger situation information into one of an infant boarding situation, a passenger conversation situation, a passenger sleeping situation, a passenger calling situation, and a passenger disembarking situation based on the gender, age group, speech status, and behavior of the passenger for each seat. Claim 12 A method for providing personalized audio based on seat recognition using artificial intelligence, wherein, in claim 7, after the step of individually controlling the output of a directional speaker corresponding to each seat differently for each seat, the speaker control unit further comprises the step of dynamically controlling the output of the directional speaker by transmitting an output control signal corresponding to a specific gesture to the directional speaker corresponding to the seat where the passenger is seated when the passenger takes a predetermined specific gesture.

Images