Apparatus, system, and method for vehicle external communication

Abstract

The disclosure relates to an apparatus, a system, and a method for vehicle exterior communication. The apparatus for vehicle exterior communication according to the disclosure includes a processor configured to generate location information of a user through an external sensor of a vehicle, control a window of the vehicle based on the location information of the user, process respective voice signals transmitted from a plurality of microphones inside the vehicle based on the location information of the user, and output a guide voice through a speaker closest to the user among a plurality of speakers based on the location information of the user.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of Korean Patent Application No. 10-2025-0023491, entitled "Apparatus and Method for External Communication of Vehicles," filed on February 24, 2025, with the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to devices and methods for external vehicle communication, and more specifically, to devices and methods configured to enable users outside the vehicle to communicate via microphones and speakers inside the vehicle. Background Technology

[0004] Recently, technological advancements have been made in autonomous vehicles that transport passengers to their destinations. A prime example is the robotaxi, an autonomous vehicle capable of transporting passengers from a pick-up location to their desired destination solely through autonomous driving, without the need for a separate driver's seat or driver.

[0005] Typically, the standard process for using a robotaxi may include requesting a vehicle via a digital key application and, when the robotaxi arrives, commanding the server to unlock the doors via the application. However, even when using a valid digital key, there are situations where robotaxis services may be unavailable due to digital key authentication errors or when the terminal with the digital key application runs out of power, preventing passenger authentication.

[0006] External interfaces (e.g., physical buttons, external speakers, and / or external microphones) have been added to robotaxis, allowing passengers to press a call button mounted on the robotaxis and, when their smartphone battery dies after calling the robotaxis, engage in a voice call with a guide at an external control server. However, this approach increases hardware manufacturing costs and development complexity, and is susceptible to interference from external noise during calls. Specifically, using physical buttons requires additional maintenance, such as cleaning and fault management, and forces passengers to move to the side of the robotaxis to access the physical buttons, thus degrading the user experience. Furthermore, this approach is also disadvantageous in terms of robotaxis vehicle design due to the need for additional external hardware components. Summary of the Invention

[0007] This disclosure aims to provide an apparatus and method for external vehicle communication that enables a user to communicate with an external control server by utilizing a microphone, speaker, and communication unit disposed in the vehicle.

[0008] Furthermore, this disclosure aims to provide an apparatus and method for external vehicle communication that enables a user to communicate with an external control server by utilizing hardware implemented in existing vehicles, such as surround-view cameras, internal speakers, and internal microphones.

[0009] Furthermore, this disclosure aims to provide an apparatus and method for vehicle external communication that enables a user to communicate with an external control server via the vehicle by utilizing a surround view system, microphone beamforming control technology, and speaker output logic.

[0010] According to the purpose of this disclosure, an apparatus for external vehicle communication is provided. The apparatus may include: a processor configured to generate user location information via external sensors of the vehicle, control vehicle windows based on the user location information, process corresponding voice signals transmitted from multiple microphones inside the vehicle based on the user location information, and output guiding voice through a speaker closest to the user among multiple speakers based on the user location information.

[0011] According to an exemplary embodiment of this disclosure, the processor can be configured to: receive distance information between the vehicle and surrounding objects via an ultrasonic sensor; capture images of one or more of the front, rear, left, and right sides of the vehicle via a camera; detect the presence of a user based on the distance information and the images; and generate location information of the user based on the distance information and the images, wherein the location information includes the distance and direction of the user relative to the vehicle, and the direction is one of the left front, right front, left rear, and right rear of the vehicle.

[0012] According to an exemplary embodiment of this disclosure, the processor can be configured to: detect user regions where the user may be located based on distance information and image using a single-shot detector (SSD), determine a confidence score for each user region detected in the user regions, and determine the user's location based on the user region with the highest confidence score among the user regions whose confidence scores are equal to or greater than a predetermined threshold.

[0013] According to an exemplary embodiment of this disclosure, the processor can be configured to control the opening or closing of windows in the user's direction via a Body Domain Controller (BDC) based on a user's communication request signal received through an in-vehicle infotainment (IVI) system connected to an external control server.

[0014] According to an exemplary embodiment of this disclosure, the processor can be configured to adjust the opening degree of the window in the user's direction via BDC based on the user's distance relative to the vehicle.

[0015] According to an exemplary embodiment of this disclosure, the guidance voice may include one or more of the following: hands-free voice output through a call connection with an external control server, and guidance voice of the IVI system output in response to the user's voice signal.

[0016] According to an exemplary embodiment of this disclosure, the processor may be configured to: close the window via BDC based on one or more of the following: receiving a communication termination signal from a user's terminal, receiving a communication termination signal from an external control server, and not detecting the presence of a user; capture an interior image of the vehicle via an internal camera through an IVI system; and send the interior image of the vehicle to the external control server.

[0017] According to an exemplary embodiment of this disclosure, the processor can be configured to: adjust the beamforming regions of a plurality of microphones to cover the user's area based on the user's location information, and extract the voice signal corresponding to the beamforming region.

[0018] According to an exemplary embodiment of this disclosure, the processor may be configured to perform one or more of the following on the user's speech signal: noise suppression, automatic gain control (AGC), equalization, and digital filtering.

[0019] According to an exemplary embodiment of this disclosure, the processor can be configured to set the gain value of the external amplifier based on the distance of the user relative to the vehicle.

[0020] According to the purpose of this disclosure, a system for external vehicle communication is provided. The system may include a vehicle and a computing device. The computing device may be coupled to the vehicle. The computing device may include a processor and a memory. The memory may be configured to store instructions, which, when executed by the processor, are configured to cause the processor to perform the following steps: generating user location information via external sensors of the vehicle; controlling the vehicle's windows based on the user location information; processing corresponding voice signals transmitted from multiple microphones inside the vehicle based on the user location information; and outputting guiding voice through the speaker closest to the user among multiple speakers based on the user location information.

[0021] According to the purpose of this disclosure, a method for external vehicle communication is provided. The method may include performing the following steps using a computing device coupled to the vehicle, including a processor and memory: generating user location information via external sensors of the vehicle; controlling the vehicle windows based on the user location information; processing corresponding voice signals transmitted from multiple microphones inside the vehicle based on the user location information; and outputting guiding voice through the speaker closest to the user among multiple speakers based on the user location information.

[0022] According to an exemplary embodiment of this disclosure, generating user location information may include: receiving distance information between the vehicle and surrounding objects via an ultrasonic sensor; capturing images of one or more of the vehicle's front, rear, left, and right sides via a camera; detecting the presence of the user based on the distance information and the images; and generating user location information based on the distance information and the images, wherein the location information includes the user's distance and direction relative to the vehicle, and the direction is one of the vehicle's left front, right front, left rear, and right rear.

[0023] According to an exemplary embodiment of this disclosure, generating location information may include: detecting user regions where the user may be located using a single-shot detector (SSD) based on distance information and image, determining a confidence score for each user region detected, and determining the user's location based on the user region with the highest confidence score among user regions whose confidence scores are equal to or greater than a predetermined threshold.

[0024] According to an exemplary embodiment of this disclosure, control may include: controlling the opening or closing of a window in the user's direction via a Body Domain Controller (BDC) based on a communication request signal received from an in-vehicle infotainment (IVI) system connected to an external control server.

[0025] According to an exemplary embodiment of this disclosure, control may include: adjusting the degree of window opening in the user's direction via BDC based on the user's distance relative to the vehicle.

[0026] According to an exemplary embodiment of this disclosure, control may include: closing the window via BDC based on one or more of the following: receiving a communication termination signal from the user's terminal, receiving a communication termination signal from an external control server, and not detecting the presence of the user; capturing an interior image of the vehicle via an internal camera through an IVI system; and sending the interior image of the vehicle to an external control server.

[0027] According to an exemplary embodiment of this disclosure, the guidance voice may include one or more of the following: hands-free voice output through a call connection with an external control server, and guidance voice of the IVI system output in response to the user's voice signal.

[0028] According to an exemplary embodiment of this disclosure, the method may include: adjusting the beamforming regions of a plurality of microphones to cover the user's area based on the user's location information, and extracting a voice signal corresponding to the adjusted beamforming region.

[0029] According to an exemplary embodiment of this disclosure, extracting a speech signal may include performing one or more of the following on the user's speech signal: noise suppression, automatic gain control (AGC), equalization, and digital filtering.

[0030] According to an exemplary embodiment of this disclosure, outputting guidance voice may include setting the gain value of an external amplifier based on the user's distance relative to the vehicle. Attached Figure Description

[0031] The foregoing and other aspects, features, advantages, and the following detailed description of embodiments will be better understood when read in conjunction with the accompanying drawings. However, this disclosure is not intended to be limited to the details shown in the drawings, and various modifications and structural changes may be made without departing from the spirit of this disclosure and within the scope and range of equivalents of the claims. In the various drawings, the same reference numerals and symbols indicate the same elements.

[0032] Figure 1 This is a block diagram illustrating a vehicle external communication device according to an exemplary embodiment of the present disclosure.

[0033] Figure 2 This is a flowchart illustrating a vehicle external communication method according to an exemplary embodiment of the present disclosure.

[0034] Figure 3 This is a diagram illustrating the open and closed positions of a window according to an exemplary embodiment of the present disclosure.

[0035] Figure 4 This is a diagram illustrating a method of processing sensor data using a single-spot detector (SSD) algorithm to generate user location information according to an exemplary embodiment of the present disclosure.

[0036] Figure 5 This is a diagram illustrating the processing of corresponding voice signals transmitted from multiple microphones inside a vehicle based on a user's location information according to an exemplary embodiment of the present disclosure.

[0037] Figure 6 An example architecture of a vehicle according to an exemplary embodiment of this disclosure is shown.

[0038] Figure 7 Exemplary elements of a computing device according to an exemplary embodiment of the present disclosure are shown. Detailed Implementation

[0039] The exemplary embodiments of this disclosure will be described in detail below. These exemplary embodiments are implemented based on the technical solutions of this disclosure and illustrate specific implementation methods and specific operation processes, but the protection scope of this disclosure is not limited to the following exemplary embodiments.

[0040] The following specific embodiments are provided by way of example only and not by way of limitation. Furthermore, they are not intended to be limited to any express or implied theory presented in the foregoing background or the following specific embodiments.

[0041] Reference will now be made to various exemplary embodiments of the subject matter, examples of which are illustrated in the accompanying drawings. While various embodiments are discussed herein, it should be understood that they are not intended to limit these embodiments. Rather, the presented embodiments are intended to cover substitutions, modifications, and equivalents that may be included within the spirit and scope of the various embodiments defined by the appended claims. Furthermore, numerous specific details are set forth in this particular embodiment to provide a thorough understanding of embodiments of the subject matter. However, embodiments may be practiced without these specific details. In other examples, well-known methods, processes, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the described embodiments.

[0042] Some portions of the following detailed descriptions are presented in the form of processes, logic blocks, procedures, and other symbolic representations of operations on data within an electrical device. These descriptions and representations are means by which those skilled in the art of data processing most effectively communicate the substance of their work to others skilled in the art. In this application, processes, logic blocks, procedures, etc., are contemplated as one or more self-consistent processes or instructions that result in a desired outcome. These programs are those that require physical manipulation of physical quantities. Typically, although not essential, these physical quantities may take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated in electronic systems, devices, and / or components.

[0043] However, it should be remembered that these and similar terms will be associated with appropriate physical quantities and are merely convenient labels applied to those quantities. Unless otherwise specifically stated, as will be apparent from the following discussion, it should be recognized that throughout the description of the embodiments, the use of terms such as “determine,” “communicate,” “take,” “compare,” “monitor,” “calibrate,” “estimate,” “start,” “provide,” “receive,” “control,” “send,” “isolate,” “generate,” “align,” “synchronize,” “identify,” “maintain,” “display,” “switch,” etc., refers to the actions and processes of electronic items, such as: processors, sensor processing units (SPUs), processors of sensor processing units, application processors of electronic devices / systems, etc., or combinations thereof. This item manipulates data represented as physical (electronic and / or magnetic) quantities in registers and memories and converts that data into other data similarly represented as physical quantities within memory or registers or other such information storage, transmission, processing, or display components.

[0044] It should be understood that, as used herein, the terms "vehicle" or "of a vehicle" or other similar terms include motor vehicles in a broad sense, such as passenger cars, including sport utility vehicles (SUVs), buses, trucks, various commercial vehicles, vessels (including various boats and ships), aircraft, etc., and include hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other vehicles powered by alternative fuels (e.g., fuels derived from resources other than petroleum). As mentioned herein, a hybrid vehicle is a vehicle having two or more power sources, such as a vehicle powered by both gasoline and electricity. In several aspects, a vehicle may include internal combustion engine systems as disclosed herein.

[0045] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. These terms are intended only to distinguish one component from another, and these terms do not limit the nature, order, or sequence of the constituent components. It should also be understood that when the terms “comprises” and / or “comprising” are used in this specification, these terms specify the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or combinations thereof. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. Throughout the specification, unless explicitly stated otherwise, the word “comprise” and variations such as “comprises” or “comprising” shall be understood to imply the inclusion of the stated elements, but do not exclude any other elements. Furthermore, the terms “unit,” “device,” “machine,” and “module” described in the specification refer to a unit for performing at least one function and operation, and can be implemented by hardware components or software components and combinations thereof.

[0046] Although the exemplary implementation is described as using multiple units to perform the exemplary process, it should be understood that the exemplary process can also be performed by one or more modules. Furthermore, it should be understood that the term controller / control unit refers to a hardware device that includes a memory and a processor and is specifically programmed to perform the processes described herein. The memory is configured to store modules, and the processor is specifically configured to execute said modules to perform one or more processes further described below.

[0047] Furthermore, the control logic of this disclosure can be embodied in a non-transient computer-readable medium containing executable program instructions that are executed by a processor, controller, etc. Examples of computer-readable media include, but are not limited to, ROM, RAM, optical disc (CD)-ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage device. The computer-readable medium can also be distributed across a network-coupled computer system, enabling it to be stored and executed in a distributed manner, for example, via a telematics server or a controller area network (CAN).

[0048] Unless otherwise specified or obvious from the context, as used herein, the term “about” should be understood as being within the normal tolerance range in the field, such as within 2 standard deviations of the mean. “About” can be understood as being within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, 0.1%, 0.05%, or 0.01% of the stated value. Unless the context otherwise clarifies, all numerical values ​​provided herein are modified by the term “about”.

[0049] The embodiments described herein can be discussed in the general context of processor-executable instructions residing on some form of non-transient processor-readable medium (such as program modules) that are executed by one or more computers or other devices. Typically, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The functionality of program modules can be combined or distributed as needed in different embodiments.

[0050] In the diagrams, a single block can be described as performing a function or multiple functions. However, in practice, the function or multiple functions performed by that block can be performed in a single component or across multiple components, and / or can be performed using hardware, software, or a combination of hardware and software. To clearly illustrate this interchangeability between hardware and software, various illustrative components, blocks, modules, logic, circuits, and steps have been described in general terms of their functionality. Whether these functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of this disclosure. Furthermore, the example device vibration sensing system and / or electronic device described herein may include components other than those shown, including well-known components.

[0051] The various techniques described herein can be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a particular manner. Any feature described as a module or component may also be implemented together as an integrated logic device or separately as a discrete but interoperable logic device. If implemented in software, the technique may be implemented at least in part by a non-transient processor-readable storage medium comprising instructions that, when executed, perform one or more of the methods described herein. The non-transient processor-readable data storage medium may form part of a computer program product, which may include encapsulation material.

[0052] Non-transient processor-readable storage media may include random access memory (RAM) (e.g., synchronous dynamic random access memory (SDRAM)), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, and other known storage media. Additionally or alternatively, this technology may be implemented at least in part by a processor-readable communication medium that carries or conveys code in the form of instructions or data structures, and that such code can be accessed, read, and / or executed by a computer or other processor.

[0053] The various implementations described herein can be executed by one or more processors, such as one or more motion processing units (MPUs), sensor processing units (SPUs), host processors or their cores, digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), application-specific instruction set processors (ASIPs), field-programmable gate arrays (FPGAs), programmable logic controllers (PLCs), complex programmable logic devices (CPLDs), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein, or other equivalent integrated or discrete logic circuits. As used herein, the term "processor" may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. As used in this specification, the term "processor" may refer to substantially any computing processing unit or device, including but not limited to single-core processors; single-processors with software multithreading capabilities; multi-core processors; multi-core processors with software multithreading capabilities; multi-core processors with hardware multithreading technology; parallel platforms; and parallel platforms with distributed shared memory. Furthermore, processors can utilize nanoscale architectures, such as, but not limited to, molecular and quantum dot-based transistors, switches, and gates, to optimize space utilization or enhance the performance of user equipment. Processors can also be implemented as a combination of computing units.

[0054] Furthermore, in some aspects, the functionality described herein can be provided within dedicated software or hardware modules configured as described herein. Moreover, the technology can be fully implemented within one or more circuit or logic elements. The general-purpose processor can be a microprocessor, but alternatively, it can be any processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of an SPU / MPU and a microprocessor, multiple microprocessors, one or more microprocessors combined with an SPU core, an MPU core, or any other such configuration. One or more components of the SPU or electronic device described herein can be embodied in the form of a “chip,” a “package,” or one or more integrated circuits (ICs).

[0055] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0056] Figure 1 This is a block diagram of a vehicle external communication device according to one embodiment of the present disclosure, and Figure 2 This is a flowchart of a vehicle external communication method according to one embodiment of the present disclosure.

[0057] Now for reference Figure 1 According to an exemplary embodiment of the present disclosure, a vehicle external communication device 100 is illustratively depicted.

[0058] According to an exemplary embodiment, the vehicle external communication device 100 may include a location information generation unit 110, a control unit 120, a voice processing unit 130, a voice output unit 140, and / or other suitable components.

[0059] The location information generation unit 110 can be configured to detect the presence of a user and determine the user's orientation relative to the vehicle based on two-dimensional (2D) images acquired by a camera located outside the vehicle (e.g., vehicle system architecture 600), and to determine the user's distance relative to the vehicle based on depth information acquired by a distance sensor located outside the vehicle, thereby generating user location information including the user's orientation and distance (e.g., see [link to relevant documentation]). Figure 2 (S210). According to an exemplary embodiment, the camera and / or distance sensor may be coupled to the vehicle.

[0060] According to an exemplary embodiment, the location information generation unit 110 can be configured to detect the presence of a user using, for example, an ultrasonic sensor and a camera, and generate information about the user's direction and distance based on the detected user. For example, the location information generation unit 110 can be configured to acquire distance information from the vehicle to one or more surrounding objects using an ultrasonic sensor, and to acquire images of at least one of the vehicle's front, rear, left, and right sides using a camera. For example, the location information generation unit 110 can be configured to detect the presence of a user based on the distance information from the vehicle to one or more surrounding objects and the images.

[0061] Therefore, the location information generation unit 110 can be configured to generate the user's location information based on distance information from the vehicle to one or more surrounding objects and images, including the user's distance and direction relative to the vehicle.

[0062] The user's direction can be set to one of the following: front left, front right, rear left, or rear right of the vehicle, and / or it can be set to the direction from the center point of the location where multiple microphones are installed inside the vehicle toward the user.

[0063] Simultaneously, the location information generation unit 110 can be configured to use a single-shot detector (SSD) to detect user regions where the user may be located based on distance information and image detection, determine a confidence score for each detected user region, and identify the user's location based on the user region with the highest confidence score among user regions whose confidence scores are equal to or greater than a predetermined threshold. For example, the location information generation unit can be configured to identify the user's location using a processor implemented to perform object detection using an SSD.

[0064] According to an exemplary implementation, the confidence score can be between 0 and 1, and the closer the confidence score is to 1, the higher the probability that the corresponding user area actually includes the user.

[0065] Now for reference Figure 3 According to an exemplary embodiment of this disclosure, the open and closed positions of the window are illustratively depicted.

[0066] According to an exemplary implementation, the vehicle external communication device can be configured to process voice signals based on the user's location, and therefore does not limit the user's location.

[0067] Now for reference Figure 4 According to an exemplary embodiment of the present disclosure, a method for processing sensor data using a single-spot detector (SSD) algorithm to generate user location information is illustratively described.

[0068] According to an exemplary implementation, SSD may include an algorithm configured to detect the presence of one or more objects and determine the location of the one or more objects within an image or video frame. SSD may be configured to perform feature extraction and noise reduction on an input image, select candidate object regions where the probability of an object being present around the vehicle exceeds a predetermined threshold, assign a confidence score based on the probability that the selected candidate object regions actually contain the user, and determine the user's location based on the assigned confidence score.

[0069] Control unit 120 can be configured to control the vehicle's windows based on the user's location information (see, for example, see...). Figure 2 (S220). For example, control unit 120 can be configured to open the window in the user's direction when it receives an external communication execution signal from an external control server. Specifically, control unit can be configured to control the opening or closing of the window in the user's direction via body domain controller (BDC) when the in-vehicle infotainment (IVI) system receives a user's communication request signal from an external control server.

[0070] In addition, the control unit 120 can be configured to adjust the opening degree of the window via BDC based on the user's distance from the vehicle.

[0071] In addition, the control unit 120 can be configured to control the body domain, including, for example, door lift controllers, window lift controllers, electric side mirrors, air conditioning, central locking, etc., via a BDC implemented as an ECU.

[0072] For example, the control unit 120 can be configured to generate control signals for the motor driver of the actuator based on the position sensor of the monitored window, and can be implemented to adjust the opening and closing amount of a specific window.

[0073] For example, the control unit 120 can be configured to partially open the window to a lower degree as the detected distance of the user relative to the vehicle decreases.

[0074] Therefore, it can enhance vehicle safety and stability, improve speaker output efficiency, and maintain interior temperature more effectively through air conditioning and heating systems.

[0075] On the other hand, the control unit 120 can be configured to increase the opening degree of the window as the detected user's distance from the vehicle increases. Therefore, the microphone's voice recognition rate can be improved, and the speaker's output efficiency can be increased.

[0076] The voice processing unit 130 can be configured to process corresponding voice signals transmitted from multiple microphones inside the vehicle based on the user's location information (e.g., see [link]). Figure 2(S230). For example, the voice processing unit 130 can be configured to receive only voice signals from the user's direction and can be configured to remove voice signals or noise from other directions. For example, the voice processing unit 130 can be configured to adjust the beamforming areas of multiple microphones to cover the user's area based on the user's location information and extract the voice signal corresponding to the beamforming area.

[0077] Now for reference Figure 5 According to an exemplary embodiment of the present disclosure, an illustration is provided showing the processing of corresponding voice signals transmitted from multiple microphones inside a vehicle based on the user's location information.

[0078] According to an exemplary embodiment, the voice processing unit 130 can be configured to perform echo cancellation (echo canceller #1, echo canceller #2) on corresponding voice signals transmitted from microphones (microphone #1, microphone #2) based on user location information and vehicle control information received via the downlink from the head unit. Thereafter, the voice processing unit 130 can be configured to perform beamforming processing on the echo-cancelled voice signals. Then, the voice processing unit 130 can be configured to perform noise cancellation processing and post-processing on the beamformed voice signals.

[0079] For example, when performing post-processing, the speech processing unit 130 can be configured to reduce residual noise and distortion, and enhance signals (e.g., speech) in a specific frequency band.

[0080] For example, the voice processing unit 130 may be configured to perform at least one of noise suppression, automatic gain control (AGC), equalization, and digital filtering on the user's voice signal.

[0081] The voice output unit 140 can be configured to output guiding voice through the speaker closest to the user among a plurality of speakers, based on the user's location information (e.g., see [link]). Figure 2 (S240). For example, when the user's position is determined to be at the left front of the vehicle, the voice output unit 140 can be configured to output guiding voice through a speaker located in the left front door of the vehicle.

[0082] Furthermore, the voice output unit 140 can be configured to set the gain value of the external amplifier based on the user's distance relative to the vehicle.

[0083] For example, the voice output unit 140 can be configured to increase the sound output level as the user's distance from the vehicle increases.

[0084] According to an exemplary embodiment, the guidance voice may include at least one of a hands-free voice output through a call connection with an external control server and a guidance voice of the IVI system output in response to the user's voice signal.

[0085] For example, a vehicle external communication device according to an exemplary embodiment of this disclosure can be configured to allow a user to communicate with an operator by connecting to an external control server, perform user authentication through the external control server, or control the vehicle.

[0086] On the other hand, the vehicle external communication device according to an exemplary embodiment of the present disclosure can be configured to allow a user to perform user authentication or control the vehicle through the vehicle itself without connecting to an external control server.

[0087] The control unit 120 can be configured to close the window via BDC, capture an interior image of the vehicle via an interior camera through the IVI system, and send the interior image of the vehicle to the external control server based on at least one of receiving a communication termination signal from the user's terminal, receiving a communication termination signal from an external control server, and not detecting the presence of the user.

[0088] According to an exemplary implementation, when the application uses logic to terminate the communication operation as described above, pollution and damage to the vehicle can be prevented, thereby enhancing vehicle safety and security.

[0089] Now for reference Figure 6 This disclosure provides an example vehicle system architecture 600 for a vehicle according to an exemplary embodiment of the present disclosure. The following discussion of the vehicle system architecture 600 is sufficient to understand one or more components of the vehicle described above.

[0090] like Figure 6 As shown, vehicle system architecture 600 may include an engine, an electric motor or propulsion unit 602, and various sensors 604 to 618 for measuring various parameters of vehicle system architecture 600, such as, but not limited to, the parameters of the vehicle snapshot described above. In a gas-powered or hybrid vehicle with a fuel-powered engine, sensors 604 to 618 may include, for example, an engine temperature sensor 604, a battery voltage sensor 606, an engine revolutions per minute (RPM) sensor 608, and / or a throttle position sensor 610. If the vehicle is an electric vehicle or a hybrid vehicle, the vehicle may include an electric motor, and accordingly may include sensors such as a battery monitoring system 612 (for measuring battery current, voltage, and / or temperature), an electric motor current sensor 614 and an electric motor voltage sensor 616, and an electric motor position sensor 618 (such as a solver and encoder).

[0091] Operating parameter sensors shared by both types of vehicles may include, for example: a position sensor 634, such as an accelerometer, gyroscope, and / or inertial measurement unit; a speed sensor 636; and / or an odometer sensor 638. The vehicle system architecture 600 may also include a clock 642, which the system uses to determine the vehicle time and / or date during operation. The clock 642 may be encoded into the onboard computing device 620; it may be a separate device, or multiple clocks may be available.

[0092] Vehicle system architecture 600 may include various sensors that operate to collect information about the environment in which the vehicle is traveling. These sensors may include, for example: a position sensor 644 (e.g., a Global Positioning System (GPS) device); object detection sensors, such as one or more cameras 646; a LiDAR sensor system 648; and / or a radar and / or sonar system 650. These sensors may include environmental sensors 652, such as humidity sensors, precipitation sensors, light sensors, and / or ambient temperature sensors. The object detection sensors may be configured to enable vehicle system architecture 600 to detect objects within a given distance of the vehicle in any direction, while the environmental sensors 652 may be configured to collect data about environmental conditions within the vehicle's driving area. According to an exemplary embodiment, vehicle system architecture 600 may include one or more lights 654 (e.g., headlights, floodlights, strobes, etc.).

[0093] During operation, information can be transmitted from sensors to an onboard computing unit 620 (e.g., computing unit 700). The onboard computing unit 620 can be configured to analyze data captured by sensors and / or received from data providers, and can be configured to selectively control the operation of the vehicle system architecture 600 based on the results of the analysis. For example, the onboard computing unit 620 can be configured to: control braking via a brake controller 622; control direction via a steering controller 624; control speed and acceleration via a throttle controller 626 (in a gas-powered vehicle) or an electric motor speed controller 628 (such as a current level controller in an electric vehicle); control a differential gear controller 630 (in a vehicle with a transmission); and / or other controllers. The brake controller 622 may include a pedal force sensor and / or a simulator temperature sensor, as described herein.

[0094] Geographic location information can be transmitted from location sensor 644 to onboard computing device 620, which can then access a map of the environment corresponding to the location information to determine known fixed features of the environment, such as streets, buildings, stop signs, and / or stop / go signals. Images captured from camera 646 and / or object detection information captured from sensors such as LiDAR 648 can be transmitted from those sensors to onboard computing device 620. The object detection information and / or captured images can be processed by onboard computing device 620 to detect objects near the vehicle. Any known or potentially known techniques that can be used for object detection based on sensor data and / or captured images can be used in the embodiments disclosed in this document.

[0095] Now for reference Figure 7 A diagram illustrating an example architecture of computing device 700 is provided. According to exemplary embodiments, one or more functions of this disclosure may be implemented by a computing device, such as, for example, computing device 700 or a computing device similar to computing device 700. Computing device 700 may be a quantum computer, a classical computer, and / or have one or more components configured to perform one or more quantum and / or classical computing functions. Computing device 620 and / or vehicle external communication device 100 may be examples of computing device 700 and / or may include one or more components of computing device 700. According to exemplary embodiments, location information generation unit 110, control unit 120, voice processing unit 130, and / or voice output unit 140 may be components of computing device 700.

[0096] Figure 7 The hardware architecture represents an example implementation of a representative computing device configured to implement at least a portion of the system / device and method / control logic described herein.

[0097] Some or all of the components of the computing device 700 may be implemented as hardware, software, and / or a combination of hardware and software. Hardware may include, but is not limited to, one or more electronic circuits. Electronic circuits may include, but are not limited to, passive components (e.g., resistors and capacitors) and / or active components (e.g., amplifiers and / or microprocessors). Passive and / or active components may be adapted, arranged, and / or programmed to perform one or more of the methods, processes, or functions described herein.

[0098] like Figure 7As shown, computing device 700 may include a user interface 702 (e.g., a graphical user interface), a central processing unit (“CPU”) 706, a system bus 710, a memory 712 connected to and accessible by other parts of computing device 700 via the system bus 710, and a hardware entity 714 connected to the system bus 710. The user interface may include input and output devices configured to facilitate user-software interaction for controlling the operation of computing device 700. Input devices may include, but are not limited to, a physical and / or touch keyboard 740. Input devices may be connected to computing device 700 via a wired or wireless connection (e.g., Bluetooth® connection). Output devices may include, but are not limited to, a speaker 742, a display 744, and / or a light-emitting diode 746.

[0099] At least some of the hardware entities in hardware entity 714 may be configured to perform actions involving access to and use of memory 712, which may be random access memory (RAM), a disk drive and / or optical disc read-only memory (CD-ROM), and other suitable memory types. Hardware entity 714 may include a disk drive unit 716, which includes a computer-readable storage medium 718 on which one or more instruction sets 720 (e.g., program instructions such as, but not limited to, software code) configured to implement one or more of the methods, processes, or functions described herein may be stored. The instructions 720 may also reside wholly or at least partially within memory 712 and / or CPU 706 during execution by computing device 700.

[0100] The memory 712 and CPU 706 may also constitute a machine-readable medium. As used herein, the term "machine-readable medium" refers to a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store a set of one or more instructions 720. As used herein, the term "machine-readable medium" also refers to any medium capable of storing, encoding, or carrying a set of instructions 720 for execution by the computing device 700 and causing the computing device 700 to perform any one or more of the methods of this disclosure. Depending on the implementation, one or more computer application programs 724 may be stored on the memory 712.

[0101] The foregoing description includes examples of this disclosure. Of course, for the purposes of describing the subject matter, it is impossible to describe every conceivable combination of components or methods, but it should be understood that many further combinations and substitutions of this disclosure are possible. Therefore, the claimed subject matter is intended to cover all such changes, modifications, and variations falling within the spirit and scope of the appended claims.

[0102] In particular, and with regard to the different functions performed by the components, apparatuses, systems, etc., described above, unless otherwise indicated, the terminology used to describe such components (including references to “means”) is intended to correspond to any component that performs the specific function of the described component (e.g., a functional equivalent), even if such components are not structurally equivalent to the disclosed structure, which performs the function of the claimed subject matter in the exemplary aspects shown herein.

[0103] The aforementioned system and components have already been described regarding interactions between several components. It is understood that such a system and components may include those components or designated sub-components, some designated components or sub-components, and / or additional components, as well as different permutations and combinations thereof. Sub-components may also be implemented as components communicatively coupled to other components, rather than being included within a parent component (hierarchical). Furthermore, it should be noted that one or more components may be combined into a single component providing aggregation functionality or divided into several independent sub-components. Any component described herein may also interact with one or more other components not specifically described herein.

[0104] Furthermore, while a particular feature of the invention may be disclosed only with respect to one of several implementations, such features may be combined with one or more other features of other implementations, as may be desirable and advantageous for any given or particular application. Moreover, to the extent to which the terms “includes,” “including,” “having,” “comprising,” variations thereof, and other similar words are used in the Detailed Description or claims, these terms are intended to be inclusive in a manner similar to the term “comprising” as an open-ended transitional phrase, without excluding any additional or other elements.

[0105] Therefore, the embodiments and examples set forth herein are presented in order to best explain the various selected embodiments of this disclosure and their specific applications, and thereby enable those skilled in the art to make and use embodiments of this disclosure. However, those skilled in the art will recognize that the foregoing descriptions and examples have been given for purposes of illustration and example only. The descriptions set forth are not intended to be exhaustive or to limit the embodiments of this disclosure to the precise forms disclosed.

Claims

1. A device for external communication of a vehicle, comprising: The processor is configured as follows: The user's location information is generated using the vehicle's external sensors; Control the vehicle windows based on the user's location information; Based on the user's location information, the system processes corresponding voice signals transmitted from multiple microphones inside the vehicle. and Based on the user's location information, guiding voice is output through the speaker closest to the user among multiple speakers.

2. The device according to claim 1, wherein, The processor can also be configured to: The distance information between the vehicle and surrounding objects is received by an ultrasonic sensor; The camera captures images of one or more of the front, rear, left, and right sides of the vehicle. The presence of the user is detected based on the distance information and the image; and The user's location information is generated based on the distance information and the image, wherein the location information includes the user's distance and direction relative to the vehicle, and the direction is one of the vehicle's left front, right front, left rear, and right rear.

3. The device according to claim 2, wherein, The processor can also be configured to: A single detector is used to detect the user's potential location based on the distance information and the image; Determine the confidence score for each user region in the detected user regions; and The user's location is determined based on the user region with the highest confidence score among user regions whose confidence scores are equal to or greater than a predetermined threshold.

4. The device according to claim 1, wherein, The processor can also be configured to: Based on the user's communication request signal received by the in-vehicle infotainment system connected to an external control server, the opening or closing of the window in the user's direction is controlled by the vehicle domain controller.

5. The device according to claim 4, wherein, The processor can also be configured to: Based on the distance of the user relative to the vehicle, the opening degree of the window in the direction of the user is adjusted by the vehicle body domain controller.

6. The device according to claim 4, wherein, The guidance voice includes one or more of the hands-free voice output through a call connection with the external control server and the guidance voice of the in-vehicle infotainment system output in response to the user's voice signal.

7. The device according to claim 4, wherein, The processor can also be configured to: Based on one or more of the following: receiving a communication termination signal from the user's terminal, receiving a communication termination signal from the external control server, and not detecting the user's presence: The window is closed via the vehicle domain controller; The in-vehicle infotainment system captures images of the vehicle's interior via an internal camera; and The interior image of the vehicle is sent to the external control server.

8. The device according to claim 1, wherein, The processor can also be configured to: Based on the user's location information, the beamforming areas of the multiple microphones are adjusted to cover the user's area, and the voice signal corresponding to the beamforming area is extracted.

9. The device according to claim 1, wherein, The processor can also be configured to: The user's voice signal is subjected to one or more of the following: noise suppression, automatic gain control, equalization, and digital filtering.

10. The device according to claim 1, wherein, The processor can also be configured to: The gain value of the external amplifier is set based on the distance of the user relative to the vehicle.

11. A system for external communication of a vehicle, comprising: vehicle; as well as A computing device, coupled to the vehicle, wherein: The computing device includes a processor and memory, and The memory is configured to store instructions, which, when executed by the processor, are configured to cause the processor to perform the following steps: The user's location information is generated using the vehicle's external sensors; Control the vehicle windows based on the user's location information; Based on the user's location information, corresponding voice signals transmitted from multiple microphones inside the vehicle are processed; and Based on the user's location information, guiding voice is output through the speaker closest to the user among multiple speakers.

12. A method for external communication of a vehicle, comprising: The following steps are performed using a computing device, including a processor and memory, coupled to the vehicle: The user's location information is generated using the vehicle's external sensors; Control the vehicle windows based on the user's location information; Based on the user's location information, the system processes corresponding voice signals transmitted from multiple microphones inside the vehicle. and Based on the user's location information, guiding voice is output through the speaker closest to the user among multiple speakers.

13. The method according to claim 12, wherein, The location information of the user is generated by: The distance information between the vehicle and surrounding objects is received by an ultrasonic sensor; The camera captures images of one or more of the vehicle's front, rear, left, and right sides. The presence of the user is detected based on the distance information and the image; and The location information of the user is generated based on the distance information and the image, wherein the location information includes the distance and direction of the user relative to the vehicle, and the direction is one of the left front, right front, left rear and right rear of the vehicle.

14. The method according to claim 13, wherein, Generating the location information includes: using a single detector, based on the distance information and the image, detecting user regions where the user may be located, determining the confidence score of each user region in the detected user regions, and determining the user's location based on the user region with the highest confidence score among user regions whose confidence scores are equal to or greater than a predetermined threshold.

15. The method according to claim 12, wherein, Controlling the windows of the vehicle includes: controlling the opening or closing of the windows in the direction of the user via a vehicle domain controller based on a communication request signal received from the user's in-vehicle infotainment system connected to an external control server.

16. The method according to claim 15, wherein, Controlling the windows of the vehicle includes: Based on the user's distance relative to the vehicle, the opening degree of the window in the user's direction is adjusted by the vehicle domain controller; or Based on one or more of the following: receiving a communication termination signal from the user's terminal, receiving a communication termination signal from the external control server, and not detecting the user's presence, the window is closed via the vehicle domain controller; the interior image of the vehicle is captured via the in-vehicle infotainment system using an internal camera; and the interior image of the vehicle is sent to the external control server.

17. The method according to claim 15, wherein, The guidance voice includes one or more of the hands-free voice output through a call connection with the external control server and the guidance voice of the in-vehicle infotainment system output in response to the user's voice signal.

18. The method of claim 12, further comprising: Based on the user's location information, the beamforming areas of the multiple microphones are adjusted to cover the user's area; and Extract the speech signal corresponding to the adjusted beamforming region.

19. The method according to claim 18, wherein, Extracting the speech signal includes performing one or more of the following on the user's speech signal: noise suppression, automatic gain control, equalization, and digital filtering.

20. The method according to claim 12, wherein, Outputting the guidance voice includes setting the gain value of the external amplifier based on the distance of the user relative to the vehicle.

Images