Vehicle driving control device including sound sensor and vehicle driving control method
By installing a sound sensor composed of multiple vibrators of different resonance frequencies on an autonomous vehicle, combining the processor to identify sound objects and control the vehicle's driving route and speed, the problem of identifying obstacles outside the line of sight is solved and the safety of the vehicle is improved and obstacle avoidance ability.
Patent Information
- Application Number
- CN201911402855.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-01-25
- Filing Date
- 2019-12-30
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2039-12-30
AI Technical Summary
Existing technologies for identifying external environments of autonomous vehicles are difficult to identify potential obstacles outside sight in real time, especially objects from sound sources, resulting in the vehicle's possible collision with undetected sound objects.
A sound sensor consisting of a plurality of vibrators arranged in different directions and having different resonance frequencies is used to identify the sound object through the vibration frequency, direction and intensity of the vibrator, and the vehicle driving route and speed are controlled in combination with the processor to avoid collisions.
Real-time identification of sound objects outside the line of sight and vehicle driving control, improve the safety and obstacle avoidance capabilities of autonomous driving vehicles, and enhance the perception of the external environment.
Smart Images

Figure CN111483461B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application is based on and claims the benefit of priority from Korean Patent Application No. 10-2019-0010069 filed on January 25, 2019, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. Technical Field
[0003] The present disclosure relates to a vehicle driving control device using a sound sensor and a vehicle driving control method of the vehicle driving control device. Background Art
[0004] As interest in autonomous vehicles increases, technologies for achieving autonomous driving have attracted attention. In order to make a vehicle move on its own without driver operation, the following technologies are used: (1) technology for recognizing the external environment of the vehicle, (2) technology for synthesizing the recognized information, determining operations such as acceleration, stopping, and turning, and determining the driving route, and (3) technology for controlling the movement of the vehicle by using the determined information. All technologies must be organically combined to achieve autonomous driving, but technology for recognizing the external environment of the vehicle is becoming increasingly important. This is because recognizing the external environment is the first element of autonomous driving, and the fusion of electrical and electronic technology and information technology is performed to recognize the external environment.
[0005] Technologies for recognizing the external environment can be roughly classified into sensor-based recognition technologies and connection-based recognition technologies. Sensors installed on vehicles for autonomous driving include ultrasonic sensors, cameras, radars, and lidars. Sensors installed on vehicles recognize the external environment and terrain of the vehicle alone or in conjunction with other sensors and provide information to the driver and the vehicle. Summary of the invention
[0006] Provided are a vehicle driving control device using a sound sensor and a vehicle driving control method using the vehicle driving control device.
[0007] Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented embodiments.
[0008] According to an embodiment, a vehicle driving control device includes: a communication interface configured to receive a signal corresponding to a sound generated in an external environment from a sound sensor; and a processor configured to: identify the sound object by obtaining the type of the sound object generating the sound based on the received signal, and either or both of the direction of the sound object and the distance from the sound object to a vehicle including the vehicle driving control device; and control the driving of the vehicle based on the identified sound object.
[0009] The processor may also be configured to change either or both of a driving path of the vehicle and a speed of the vehicle based on the identified sound object.
[0010] The processor may also be configured to control driving of the vehicle to prevent a collision between the vehicle and the identified sound object.
[0011] The vehicle driving control device may further include: an output interface configured to output information of any one or any combination of a type of the sound object, a direction of the sound object, and a distance from the sound object to the vehicle.
[0012] The output interface may include a display configured to display an image related to the external environment, the image including information on the type of the sound object.
[0013] The display may also be configured to display the relative position of the sound object with respect to the vehicle on the displayed image.
[0014] The sound sensor may be mounted on the vehicle.
[0015] The sound sensor may include a plurality of vibrators arranged in different directions and having different resonance frequencies, and at least two vibrators among the plurality of vibrators may be configured to vibrate in response to the sound.
[0016] Any one or any combination of the plurality of vibrators may be arranged parallel to a traveling direction of the vehicle.
[0017] The processor may also be configured to: obtain a frequency band of the sound based on a resonance frequency of a vibrator that is vibrating among the at least two vibrators among the multiple vibrators; obtain a type of the sound object based on the obtained frequency band of the sound; obtain a direction of the sound object relative to the vehicle based on a direction of the vibrating vibrator; obtain a distance from the sound object to the vehicle based on an output intensity of the vibrating vibrator; and obtain a change in the distance from the sound object to the vehicle based on a change in the output intensity of the vibrating vibrator over time.
[0018] The sound object may not be recognized by an optical sensor mounted on the vehicle.
[0019] The sound object may be in the vehicle's exterior environment in an area outside the line of sight of the optical sensor.
[0020] The optical sensor may include an image sensor.
[0021] The processor may be further configured to: obtain one or more candidate sound objects that potentially generate the sound based on the result output from the sound sensor; and identify one of the one or more candidate sound objects that is not recognized by the optical sensor as the sound object.
[0022] According to an embodiment, a method for controlling driving of a vehicle includes: receiving a signal corresponding to a sound generated in an external environment from a sound sensor; identifying the sound object by obtaining the type of the sound object generating the sound based on the received signal, and either or both of the direction of the sound object and the distance from the sound object to the vehicle; and controlling the driving of the vehicle based on the identified sound object.
[0023] Controlling the driving of the vehicle may include changing any one or any combination of a driving route of the vehicle and a speed of the vehicle based on the identified sound object.
[0024] Controlling the driving of the vehicle may include controlling the driving of the vehicle so that the vehicle and the recognized sound object do not collide with each other.
[0025] The method may further include outputting information of any one or any combination of a type of the sound object, a direction of the sound object, and a distance from the sound object to the vehicle.
[0026] The outputting may include: displaying an image related to the external environment, the image including information on the type of the sound object.
[0027] Displaying the image may include displaying a relative position of the sound object with respect to the vehicle on the displayed image.
[0028] The sound object may not be recognized by an optical sensor mounted on the vehicle.
[0029] The sound object may be in the vehicle's exterior environment in an area outside the line of sight of the optical sensor.
[0030] Identifying the sound object may include: obtaining one or more candidate sound objects that potentially generate the sound based on the result output from the sound sensor; and identifying one of the one or more candidate sound objects that is not identified by the optical sensor as the sound object.
[0031] The sound sensor may be mounted on a vehicle and include a plurality of vibrators arranged in different directions and having different resonance frequencies, and at least two vibrators among the plurality of vibrators may be configured to vibrate in response to the sound.
[0032] Identifying a sound object may include: obtaining a frequency band of the sound based on a resonance frequency of a vibrator that is vibrating among the at least two vibrators among the plurality of vibrators; obtaining a type of the sound object based on the obtained frequency band of the sound; obtaining a direction of the sound object relative to the vehicle based on a direction of the vibrating vibrator; obtaining a distance from the sound object to the vehicle based on an output intensity of the vibrating vibrator; and obtaining a change in the distance from the sound object to the vehicle based on a change in the output intensity of the vibrating vibrator over time.
[0033] Identifying the sound object may include: obtaining at least one vibrator having an intensity variation equal to or greater than a reference range among the plurality of vibrators based on a result output from the sound sensor; and identifying the sound object based on a result output from the at least one vibrator among the plurality of vibrators obtained.
[0034] According to an embodiment, a vehicle driving control device of a vehicle includes: a sound sensor configured to obtain a signal corresponding to a sound generated by a sound object in an external environment; and a processor configured to: obtain the type of the sound object, the direction of the sound object, and the distance between the sound object and the vehicle based on the received signal; and control the driving of the vehicle based on the obtained type of the sound object, the obtained direction of the sound object, and the obtained distance.
[0035] The sound sensor may include: a sound inlet; a plurality of vibrators disposed on the same plane as the sound inlet and disposed around the sound inlet, the plurality of vibrators being configured to vibrate in response to sound; and a plurality of sound outlets disposed facing the plurality of vibrators.
[0036] Each of the plurality of vibrators may include: a fixed portion fixed to a support of the sound sensor; a moving portion configured to move in response to sound; a sensing portion on which the moving portion is disposed, the sensing portion being configured to sense movement of the moving portion; and a mass body disposed on the moving portion.
[0037] The plurality of vibrators may be respectively arranged into subgroups, and in each of the subgroups, lengths of the plurality of vibrators may gradually increase in a clockwise or counterclockwise direction. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The above and other aspects and features of the embodiments of the present disclosure will become more apparent from the following description in conjunction with the accompanying drawings, in which:
[0039] Figure 1 is a diagram showing an automated vehicle according to an embodiment;
[0040] Figure 2is a reference diagram showing an object in an area out of sight according to an embodiment;
[0041] Figure 3 is a block diagram of a vehicle driving control device including a sound sensor according to an embodiment;
[0042] Figure 4 is a plan view of a schematic structure of an acoustic sensor according to an embodiment;
[0043] Figure 5 is along Figure 4 A cross-sectional view of the sound sensor taken along line AA';
[0044] Fig. 6A and Figure 6B The different cross sections are shown Figure 4 A cross-sectional view of the structure of a vibrator provided in a sound sensor;
[0045] Figure 7 is a plan view of a schematic structure of an acoustic sensor according to an embodiment;
[0046] Figure 8 is a plan view of a schematic structure of an acoustic sensor according to an embodiment;
[0047] Fig. 9 is a plan view of a schematic structure of an acoustic sensor according to an embodiment;
[0048] Fig.10 is a flowchart illustrating a method for controlling driving of a vehicle using a sound sensor according to an embodiment;
[0049] Fig.11 is a flowchart illustrating a method of recognizing an object by using a sound sensor according to an embodiment;
[0050] Fig.12 is a flow chart illustrating a method of identifying an object not identified by another sensor in a vehicle according to an embodiment;
[0051] Fig.13 is a flowchart showing a method of recognizing a sound with a large intensity change as an object according to an embodiment;
[0052] Fig.14A and Fig. 14B is a reference diagram illustrating a method of controlling driving of a vehicle using a sound sensor according to an embodiment;
[0053] Fig.15A and Fig. 15B is a reference diagram illustrating a method of controlling driving of a vehicle using a sound sensor according to an embodiment; and
[0054] Fig.16 is a block diagram of a vehicle driving control device according to an embodiment. DETAILED DESCRIPTION
[0055] The present disclosure will now be described more fully with reference to the accompanying drawings, in which embodiments of the present disclosure are shown. In the accompanying drawings, for clarity and ease of description, the width and thickness of a layer or region may be exaggerated. In the accompanying drawings, like reference numerals represent like elements.
[0056] It should be understood that the term “include” or “comprising” may not be interpreted as including all elements or steps described in the specification, and some elements or some steps may be omitted, or additional elements or steps may be further included.
[0057] Furthermore, terms such as “unit” or “module” refer to units that perform at least one function or operation, and these units may be implemented as hardware or software or a combination of hardware and software.
[0058] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. Expressions such as "at least one of..." when following a list of elements modify the entire list of elements rather than modifying the individual elements in the list.
[0059] Figure 1 is a diagram showing an automatic vehicle 10 according to an embodiment.
[0060] The vehicle 10 supporting autonomous driving may include various sensors, such as an inertial measurement unit (IMU) 11 for measuring vehicle speed, vehicle direction, gravity, acceleration, etc., a radar 12 for emitting electromagnetic waves and recognizing situations through reflected patterns, a lidar 13 for emitting low-power lasers and recognizing surrounding situations through reflected patterns, and a camera 14 for capturing images. The vehicle driving control device 100 (see Figure 3 ) can receive sensing data from sensors installed on the vehicle 10. It can be driven by the driver 120 (see Figure 3 ) (eg, steering controller 16 and / or speed controller 17) controls vehicle 10 based on information received from various sensors.
[0061] As a device for controlling the driving of the vehicle, the vehicle driving control device 100 may be installed on the vehicle 10 as a separate device different from the vehicle 10, or may be included in the vehicle 10, or may be at least a part of the electronic equipment that controls the vehicle 10. The vehicle driving control device 100 may include Figure 1Any one or any combination of the sensors shown in (e.g., radar 12, lidar 13, camera 14, and global positioning system (GPS) 15), and may include at least a portion of a steering controller 16 and a speed controller 17.
[0062] In addition, the vehicle driving control device 100 may store map data including accurate road information of an area in which the vehicle 10 will travel. The vehicle driving control device 100 may control the speed and / or direction of the vehicle 10 by using a high-speed GPU to drive an artificial intelligence program and analyze information input from sensors and road information.
[0063] The vehicle driving control device 100 according to the embodiment may further include a sound sensor 110 for sensing the sound generated in the external environment. Sensors such as the radar 12, the laser radar 13, and the camera 14 may only recognize the situation in the area within the line of sight of the external environment. In addition, sensors such as the GPS 15 and the IMU 11 may recognize the situation in the area outside the line of sight, but it is difficult to recognize the external environment in real time. Regardless of the line of sight, the sound sensor 110 may immediately detect the generation of the sound.
[0064] Figure 2 is a reference diagram showing an object in an area out of sight according to an embodiment. Figure 2 As shown, an object 10a in an area adjacent to the vehicle 10 in the external environment (i.e., in a line of sight area) can be identified by a sensor such as a radar, a laser radar, or a camera. However, an object 10b that is not adjacent to the vehicle 10 may exist in an area out of sight due to the object 10a adjacent to the vehicle 10. Even if the object 10b exists in the area out of sight, the object 10b may be an object that affects the driving of the vehicle (e.g., an ambulance). Therefore, the vehicle driving control device identifies the object 10b in real time. Because the sound sensor 110 according to the embodiment (see Figure 3 ) senses sound, so the sound sensor 110 can be used to identify the object that produces the sound regardless of the line of sight.
[0065] Figure 3 is a block diagram of a vehicle driving control device 100 including a sound sensor 110 according to an embodiment. Figure 3, the vehicle driving control device 100 may include: a sound sensor 110 for detecting sound, a driver 120 for driving the vehicle, an output interface 130 for outputting information related to the driving of the vehicle, a memory 140 for storing data related to the driving of the vehicle, and a processor 150 for controlling the driving of the vehicle based on the result detected by the sound sensor 110. Although the vehicle driving control device 100 is shown as including one processor, the embodiment is not limited thereto, and the vehicle driving control device 100 may include a plurality of processors. Hereinafter, at least some of the operations and functions of the processor 150 may be performed by a plurality of processors.
[0066] The sound sensor 110 may include a plurality of vibrators arranged in different directions, and may detect sound by vibrating one or more vibrators arranged in a direction corresponding to the direction in which the sound is generated. The sound sensor 110 may be mounted on the vehicle, for example, outside the vehicle. The vibrators of the sound sensor 110 may be arranged parallel to the driving direction of the vehicle.
[0067] Figure 4 is a plan view of a schematic structure of an acoustic sensor 110 according to the embodiment, and Figure 5 is along Figure 4 FIG. 1 is a cross-sectional view of the sound sensor 110 taken along line AA′. Fig. 6A and Figure 6B The different cross sections are shown Figure 4 FIG. 1 is a cross-sectional view of a structure of a vibrator provided in the acoustic sensor 110 .
[0068] refer to Figure 4 and Figure 5 , the sound sensor 110 may include: a sound inlet 234 through which sound is input; a sound outlet 235 through which sound input via the sound inlet 234 is output; and a plurality of vibrators 210_k arranged between the sound inlet 234 and the sound outlet 235. Herein, when the number of vibrators 210_k is N, k is an integer from 1 to N.
[0069] The physical angular resolution of the sound sensor 110 may be determined by the number N of vibrators 210_k. The physical angular resolution may be expressed as 360° / N. The sound sensor 110 may detect the direction of an input sound by comparing the output magnitudes of a plurality of vibrators 210_k. As the number of vibrators 210_k to be compared with each other increases, a higher resolution may be obtained.
[0070] The plurality of vibrators 210_k are arranged so that any one or any combination thereof selectively reacts according to the direction of the sound input to the sound inlet 234. The plurality of vibrators 210_k may be arranged to surround the sound inlet 234. The plurality of vibrators 210_k may be arranged in a planar manner without overlapping each other, and all of the plurality of vibrators 210_k may be arranged to be exposed to the sound inlet 234. Figure 4 As shown, a plurality of vibrators 210_k may be arranged on the same plane, and may be arranged around a center point C on the same plane, vertically facing the center of the sound inlet 234. In the accompanying drawings, a plurality of vibrators 210_k are shown as being arranged around the center point C in a circular shape, but this is only an example. The arrangement of the plurality of vibrators 210_k is not limited thereto, and may be arranged in various forms having symmetry with respect to the center point C. For example, a plurality of vibrators 210_k may be arranged in the shape of a polygon or an elliptical trajectory.
[0071] The plurality of sound outlets 235 may be arranged to face the plurality of vibrators 210 — k .
[0072] The size and shape of the sound inlet 234 and the sound outlet 235 are not particularly limited, and the sound inlet 234 and the sound outlet 235 may have any size and shape capable of exposing the plurality of vibrators 210 — k to the same extent.
[0073] The housing 230 having openings corresponding to the shapes of the sound inlet 234 and the sound outlet 235 may be used to form the sound inlet 234 and the sound outlet 235 .
[0074] The housing 230 may include one of various materials capable of shielding sound. For example, the housing 230 may include a material such as aluminum. The sound inlet 234 and the sound outlet 235 are not limited to Figure 4 and Figure 5 Shape shown.
[0075] The support member 220 may be located in the housing 230, and the support member 220 supports the plurality of vibrators 210_k and provides a space in which the plurality of vibrators 210_k vibrate in response to sound. The support member 220 may be formed by forming a through hole TH in a substrate, such as Figure 4 As shown. A plurality of vibrators 210_k may be supported by a support member 220 and may be arranged to face the through hole TH. The through hole TH provides a space in which the vibrator 210_k vibrates due to an external force, and the shape and size of the through hole TH are not particularly limited as long as the through hole TH provides such a space. The support member 220 may include one of various materials, such as a silicon substrate.
[0076] like Fig. 6A and Figure 6BAs shown, each of the vibrators 210_k includes a fixed portion 211 fixed to the support 220, a moving portion 213 movable in response to a signal, and a sensing portion 212 for sensing movement of the moving portion 213. The vibrator 210_k may further include a mass body 214 for providing the moving portion 213 with mass m.
[0077] The moving part 213 may include an elastic membrane. The elastic membrane may have a length L and a width W, and determines the resonance characteristics of the vibrator 210 — k and the mass m of the mass body 214. The elastic membrane may include a material such as silicon, metal, or polymer.
[0078] The sensing portion 212 may include a sensor layer for sensing the movement of the moving portion 213. The sensing portion 212 may include, for example, a piezoelectric element, and in this case, the sensing portion 212 may have a structure in which an electrode layer, a piezoelectric material layer, and an electrode layer are stacked. The piezoelectric material layer may include ZnO, SnO, PZT, ZnSnO 3 , polyvinylidene fluoride (PVDF), poly(vinylidene fluoride-trifluoroethylene) (P(VDF-TrFE)), AlN or PMN-PT. The electrode layer may include a metal material or any one of various conductive materials.
[0079] The specific values of the width and thickness of the vibrator 210_k may be determined in consideration of the desired resonance frequency of the vibrator 210_k. For example, the vibrator 210_k may have a width of about several microns to several hundred microns, a thickness of about several microns or less, and a length of about several millimeters or less, but is not limited thereto. The vibrator 210_k having such a small size may be manufactured by a micro-electromechanical system (MEMS) process.
[0080] The vibrator 210_k vibrates up and down in the Z direction in response to an external signal and exhibits an output proportional to the displacement z. The displacement z satisfies the motion equation according to Equation 1 below.
[0081] [Equation 1]
[0082]
[0083] Here, c is the damping coefficient, k is the elastic modulus, and F 0 cosωt is a driving force that indicates the action caused by the signal input to the vibrator 210_k. The value of k is determined by the physical properties and shape of the moving part 213.
[0084] The vibrator 210_k exhibits a resonance frequency f 0 The frequency response characteristics are near the bandwidth.
[0085] The resonant frequency f is the center frequency 0 It is expressed by the following equation 2.
[0086] [Equation 2]
[0087]
[0088] Therefore, the vibrator 210_k provided in the sound sensor 210 can sense a frequency band near the designed center frequency. Therefore, when designing the center frequency, a frequency band with high availability in a given environment can be set, and the vibrator 210_k can be implemented to have a shape and physical properties suitable for the set frequency band.
[0089] In the sound sensor 110 , the vibrators 210 — k provided at various positions are provided to have the same length to have the same resonance frequency. However, the embodiment is not limited thereto, and the vibrators 210 — k may be modified to have different resonance frequencies.
[0090] In the sound sensor 110, one or more vibrators 210_k located in a path into which the directional sound is input among the plurality of vibrators 210_k vibrate in response to the directional sound. Figure 5 As shown, when sound is input to path ①, one or more vibrators including vibrator 210_1 located on path ① may vibrate. When sound is input to path ②, one or more vibrators including vibrator 210_9 located on path ② may vibrate. Therefore, the direction of the input sound can be determined based on the position and output of the vibrator 210_k that selectively responds based on the direction of the input sound.
[0091] Figure 7 is a plan view showing a schematic structure of a sound sensor 110 a according to the embodiment.
[0092] The sound sensor 110a includes a plurality of subgroups, namely, first to fourth subgroups SG_1, SG_2, SG_3, and SG_4. The first to fourth subgroups SG_1, SG_2, SG_3, and SG_4 may be formed by grouping a plurality of vibrators oriented adjacent to each other to represent different directivities. For example, the first to fourth subgroups SG_1, SG_2, SG_3, and SG_4 may represent directions forming an angle of 90° with each other.
[0093] The difference between the sound sensor 110a and the sound sensor 110 is that the sound sensor 110a includes vibrators 210_1_k, 210_2_k, 210_3_k, and 210_4_k having different resonance frequencies. Figure 7As shown, the sound sensor 110a includes subgroups that vary in length to have a frequency range, rather than including 64 vibrators having one resonant frequency, so that the signals of the vibrators in one subgroup can be integrated to use the signals. Figure 7 As shown, by implementing the sound sensor 110a using a device having four subgroups, physical 4 orientations can be provided instead of 64 orientations, and the frequency band in each subgroup can be widened to the entire audible frequency band instead of a single resonant frequency band.
[0094] Hereinafter, the differences will be mainly described.
[0095] The first subgroup SG_1 includes a plurality of vibrators 210_1_k arranged along a periphery of a first circular sector having a central angle of 90°, the second subgroup SG_2 includes a plurality of vibrators 210_2_k arranged along a periphery of a second circular sector having a central angle of 90° (the second circular sector is adjacent to the first circular sector), the third subgroup SG_3 includes a plurality of vibrators 210_3_k arranged along a periphery of a third circular sector having a central angle of 90° (the third circular sector is adjacent to the second circular sector), and the fourth subgroup SG_4 includes a plurality of vibrators 210_4_k arranged along a periphery of a fourth circular sector having a central angle of 90° (the fourth circular sector is adjacent to the third circular sector).
[0096] The vibrators 210_i_k (where k is an integer from 1 to N) included in each subgroup SG_i (where i is an integer from 1 to 4) are set to have corresponding resonant frequencies to detect sounds of a frequency range Δf. The interval between the resonant frequencies of adjacent vibrators 210_i_k can be determined by considering the number N of vibrators 210_i_k included in each subgroup SG_i and the frequency range Δf.
[0097] The vibrators 210_i_k in each subgroup SG_i may be arranged in the order of magnitude of their resonance frequencies. In addition, the arrangement direction according to the magnitude order may be the same in the subgroup SG_i. Figure 7 As shown, in each subgroup SG_i, the vibrator 210_i_k may be arranged in an order in which the length of the vibrator 210_i_k gradually increases in the clockwise direction. However, the embodiment is not limited thereto, and the vibrator 210_i_k may be arranged in an order in which the length of the vibrator 210_i_k gradually increases in the counterclockwise direction. Alternatively, in some subgroups SG_i, the vibrator 210_i_k may be arranged in an order in which the length of the vibrator 210_i_k gradually increases in the clockwise direction, and in some other subgroups SG_j (where j≠i), the vibrator 210_j_k may be arranged in an order in which the length of the vibrator 210_j_k gradually increases in the counterclockwise direction.
[0098] Relatively accurate direction sensing can be achieved by using four sets of vibrators arranged to distinguish all directions using four directivities. With this in mind, the sound sensor 110a can accurately detect the direction of the input sound. In addition, since the sound sensor 110a uses vibrators 210_i_k with different resonant frequencies, the sound sensor 110a can detect signals of a wider frequency band than a sound sensor including vibrators 210_i_k with the same resonant frequency.
[0099] exist Figure 7 In the embodiment, the sound sensors of multiple subgroups are arranged in one housing. However, the embodiment is not limited thereto. Some of the multiple subgroups may be formed in different housings. For example, the first subgroup SG_1 and the second subgroup SG_2 may be formed in one housing, and the third subgroup SG_3 and the fourth subgroup SG_4 may be formed in another housing. Therefore, the first subgroup SG_1 and the second subgroup SG_2 may be installed at the front of the vehicle, and the third subgroup SG_3 and the fourth subgroup SG_4 may be installed at the rear of the vehicle. The processor may easily identify the direction according to the installation position of the sound sensor.
[0100] Figure 8 is a plan view showing a schematic structure of a sound sensor 110 b according to the embodiment.
[0101] The sound sensor 110b and Figure 7 The sound sensor 110a is different from the sound sensor 110b in that the sound sensor 110b includes 8 subgroups, and the rest of the configuration of the sound sensor 110b is substantially the same as that of the sound sensor 110a.
[0102] The sound sensor 110b is arranged so that 8 subgroups SG_i (where i is an integer from 1 to 8) can represent directions forming an angle of 45° with each other. Vibrators 210_i_j having different resonance frequencies are arranged in each subgroup SG_i.
[0103] The sound sensor 110b may have a Figure 7 The angular resolution of the sound sensor 110a is higher than that of the sound sensor 110b, but the frequency bandwidth that the sound sensor 110b can sense may become narrower. In view of this, the number of sub-groups can be set to be suitable for the environment in which the sound sensor 110b will be used.
[0104] Fig. 9 is a plan view showing a schematic structure of a sound sensor 110 c according to the embodiment.
[0105] The sound sensor 110c and Figure 8The difference between the sound sensor 110b and the sound sensor 110c is that the arrangement order of the plurality of vibrators 210_i_j in the adjacent subgroups SG_i (where i is an integer from 1 to 8) of the sound sensor 110c is in the reverse order. That is, in some subgroups, for example, in the subgroups SG_1, SG_3, SG_5, and SG_7, the vibrators 210_i_j may be arranged in the order of increasing length in the clockwise direction, while in the remaining subgroups SG_2, SG_4, SG_6, and SG_8, the vibrators 210_i_j may be arranged in the order of increasing length in the counterclockwise direction.
[0106] In the description of the sound sensors 110a, 110b and 110c including vibrators with different resonant frequencies, some examples of vibrator arrangements have been described. However, the vibrator arrangement is not limited to this. The arrangement intervals (e.g., frequency intervals or spatial intervals) of the vibrators in each subgroup may be different, and for example, various arrangements considering the coupling between adjacent vibrators may be used.
[0107] Figure 3 The processor 150 may control driving of the vehicle based on the results received from the sound sensor 110 .
[0108] Fig.10 is a flowchart showing a method of controlling driving of a vehicle using the sound sensor 110 according to an embodiment. Fig.10 , the processor 150 receives the result from the sound sensor 110 (operation S310). As described above, the sound sensor 110 may include a plurality of vibrators arranged in different directions, and the plurality of vibrators may have different resonant frequencies. Each vibrator in the sound sensor 110 may output an electrical signal corresponding to the input sound. In the sound sensor 110, a vibrator having a resonant frequency included in the frequency of the input sound may vibrate to output an electrical signal (e.g., voltage), and a vibrator having a resonant frequency not included in the frequency of the input sound may not output an electrical signal. Therefore, the sound sensor 110 may output an electrical signal subdivided by frequency.
[0109] The processor 150 may identify the object by using the electrical signal received from the sound sensor 110 (operation S320). For example, the processor 150 may obtain a sound pattern by using the electrical signal received from the sound sensor 110. The sound pattern may include a sound characteristic value that can identify the sound. The processor 150 may obtain the sound characteristic value based on the electrical signal received from the sound sensor 110 by using fast Fourier transform (FFT), two-dimensional discrete cosine transform (DCT), dynamic time warping (DTW), artificial neural network, vector quantization (VQ), Gaussian mixture model (GMM), etc., and may obtain the sound pattern according to the sound characteristic value. In addition, the processor 150 may identify whether the detected sound is the sound of which object by comparing the detected sound with an object template that defines the sound of the object. A plurality of object templates may be provided. For example, a plurality of object templates may include an object template corresponding to the sound generated by an ambulance, an object template corresponding to the sound generated by a sudden stop of a vehicle, an object template corresponding to the sound generated by a scream, an object template corresponding to the sound generated by the chassis of a vehicle, etc.
[0110] The processor 150 may compare the sound pattern with the object template to determine the similarity. The similarity may be determined by a maximum likelihood estimation method, etc. When the similarity is greater than a reference value, the processor 150 may determine that the detected sound includes the sound of an object corresponding to the object template. For example, when the sound pattern is compared with a plurality of object templates and it is determined that the similarity is equal to or greater than a reference value, the processor 150 may determine that the detected sound is generated by a plurality of objects. Hereinafter, an object generating a sound may be referred to as a sound object.
[0111] The processor 150 may control the driving of the vehicle in response to the identified object (operation S330). The processor 150 may control the driver 120 so that either or both of the driving route and driving speed of the vehicle change correspondingly to the identified object. In the memory 140, a vehicle control description matching the object may be pre-stored. The processor 150 may read the vehicle control description matching the object from the memory 140 to control the driving of the vehicle. However, the embodiment is not limited thereto. The processor 150 may use a learning network model of an artificial intelligence (AI) system to control the driving of the vehicle corresponding to the object. For example, the processor 150 may control the driving of the vehicle to prevent a collision between the identified object and the vehicle.
[0112] The sound sensor 110 may be configured to include at least a portion of the processor 150 described below. For example, in addition to detecting sound, the sound sensor 110 may also perform operations such as correcting an electrical signal of the sound, calculating characteristics of the electrical signal, etc. In this case, the sound sensor 110 may be a functional module having a hardware module and a software module.
[0113] Fig.11 is a flow chart illustrating a method of recognizing an object by using a sound sensor according to an embodiment. The identification of the object may include the type and location of the object.
[0114] The processor 150 may determine the type of the object based on the resonant frequency of the vibrator of the sound sensor 110 that is vibrating (operation S410). Each object has a unique sound. Because the sound pattern includes a frequency band, the sound sensor 110 may output an electrical signal corresponding to the input sound. In the sound sensor 110, a vibrator having a resonant frequency included in the frequency of the sound may vibrate to output an electrical signal (e.g., a voltage), and a vibrator having a resonant frequency not included in the frequency of the sound may not output an electrical signal. Therefore, the signal output from the sound sensor 110 may include information about the frequency band of the sound, that is, the resonant frequency of the resonant vibrator. The processor 150 may obtain a sound pattern based on the electrical signal subdivided by frequency received from the sound sensor 110, and determine the type of the sound by comparing the sound pattern with the object template.
[0115] The processor 150 may determine the direction of the object based on the direction of the vibrator of the sound sensor 110 that is vibrating (operation S420). The processor 150 may include a plurality of vibrators having the same resonance frequency. Even if the vibrators have the same resonance frequency, the output intensity of the vibrators may be different depending on the direction of the sound input. The direction of the vibrator that outputs the maximum electrical signal among the vibrators having the same resonance frequency may be determined as the direction of the object.
[0116] The processor 150 may determine the distance to the object based on the output strength of the vibrator (operation S430). The memory 140 may store the strength of each object template by distance. The processor 150 may determine the distance to the object by comparing the strength of the sound pattern that determines the type of the object with the strength of the object template according to the distance.
[0117] In addition to the sound sensor 110, various sensors such as radar, laser radar, and camera are installed on the vehicle 10. Radar, laser radar, and camera can be used to recognize the external environment near the vehicle. The sound sensor 110 can only recognize objects that are not recognized by other sensors.
[0118] Fig.12is a flow chart illustrating a method of identifying an object not identified by another sensor in a vehicle according to an embodiment.
[0119] The processor 150 may determine a candidate object by using the result received from the sound sensor 110 (operation S510). The method of determining the candidate object may be similar to the method of referring to Fig.10 The object recognition method is the same as described above. That is, the result received from the sound sensor 110 may be used to generate a sound pattern, and then the sound pattern may be compared with a sound template to recognize the object included in the sound pattern.
[0120] The processor 150 determines whether each candidate object is an object recognized by another sensor installed on the vehicle (operation S520). In addition, the processor 150 can determine the candidate object that is not an object recognized by another sensor among the candidate objects as a valid object (operation S530). The object recognized by the optical sensor included in the camera or laser radar may have been used to control the driving of the vehicle. By determining the type, distance and direction of the object not recognized by another sensor, the signal processing load of the vehicle driving control device 100 can be reduced. In addition, the sound generated in the object recognized by another sensor can also be used as a source of vehicle driving, and thus the driving of the vehicle can be controlled more accurately. Whether to recognize the object recognized by another sensor can be controlled by the user, the external environment, etc.
[0121] The sound sensor 110 detects all existing sounds. However, some of the existing sounds may be noise (e.g., driving noise generated by passing vehicles), and some of the existing sounds may be effective sounds for driving the vehicle. The vehicle driving control device 100 can determine the effective sound based on the intensity of the sound.
[0122] Fig.13 is a flowchart illustrating a method of recognizing a sound having a large intensity variation as an object according to an embodiment.
[0123] The processor 150 receives the result from the sound sensor 110 (operation S610). Each vibrator in the sound sensor 110 may output an electrical signal corresponding to the input sound. When each vibrator in the sound sensor 110 outputs an electrical signal corresponding to the input sound, the processor 150 may receive the electrical signal subdivided by frequency.
[0124] The processor 150 may determine the frequency of the electrical signal whose intensity variation is equal to or greater than the reference range (operation S620). When a collision occurs between vehicles in the surrounding environment of the vehicle, a loud sound is generated instantaneously due to the collision. Then, the size of the electrical signal corresponding to the collision sound output from the sound sensor 110 may instantaneously exceed the reference range.
[0125] The processor 150 may identify the object based on the result output from the vibrator whose intensity changes are equal to or greater than the reference range (operation S630). That is, the processor 150 may generate a sound pattern using the electric signal subdivided according to the frequency whose intensity changes are equal to or greater than the reference range among the frequencies of the electric signal received from the sound sensor 110, and may compare the sound pattern with the sound template to identify the object.
[0126] exist Fig.13 , a sound whose intensity variation is equal to or greater than a reference range is determined as a valid sound. However, the embodiment is not limited thereto. A sound whose intensity is greater than a reference value may be determined as a valid sound. For example, the siren of an ambulance may be a factor that affects the driving of a vehicle. However, when the ambulance is far away from the vehicle, or when the ambulance is in the opposite lane relative to the vehicle, the siren of the ambulance does not affect the driving of the vehicle. Therefore, when the siren gradually increases over time, or when the intensity of the siren is equal to or greater than a reference value, the processor 150 may determine the siren of the ambulance as a valid sound. The processor 150 may then determine the direction of the ambulance and the distance between the ambulance and the vehicle, and control the driving of the vehicle.
[0127] Fig.14A and Fig. 14B is a reference diagram of a method of controlling driving of a vehicle using the sound sensor 110 according to an embodiment.
[0128] The vehicle driving control device 100 can identify the alarm sound based on the result received from the sound sensor 110. The vehicle driving control device 100 can determine that the object generating the alarm sound is an ambulance based on the alarm sound, and can determine the direction of the ambulance and the distance from the ambulance. The processor 150 can output information about the ambulance through the output interface 130.
[0129] For example, Fig.14A As shown, the display may display an image representing the surrounding environment as a kind of output interface 130. The position of the vehicle 710 and information 720 related to the object recognized by the sound sensor 110 may be shown on the image 700. That is, the display may display the type of object (e.g., an ambulance), the direction of the object based on the vehicle, and the distance between the vehicle 710 and the object on the image 700.
[0130] like Fig. 14B As shown, the vehicle driving control device 100 can provide driving information to the driver by outputting a driving control instruction 730 according to the result obtained by the sound sensor 110 .
[0131] Fig.15A and Fig. 15Bis a reference diagram of a method of controlling driving of a vehicle using the sound sensor 110 according to an embodiment.
[0132] The sound sensor 110 may detect the alarm sound. The vehicle driving control device 100 may recognize the alarm sound based on the result received from the sound sensor 110. The vehicle driving control device 100 may determine that the object generating the alarm sound is a collision between vehicles based on the alarm sound, and may determine the collision point, etc. Then, the processor 150 may output information about the object via the display.
[0133] For example, Fig.15A As shown, the display may display an image 800 representing the surrounding environment. The position of the vehicle 810 and information 820 related to the object recognized by the sound sensor 110 may be shown on the image 800. That is, an indicator indicating a vehicle collision may be displayed at the point where the vehicle collision occurs on the image 800. Fig. 15B As shown, the vehicle driving control device 100 can provide driving information to the driver by outputting driving control instructions 830 according to the results obtained by the sound sensor 110.
[0134] The sound sensor 110 may detect sounds beyond the line of sight in real time, and thus may provide information on various external environments to the vehicle 810 .
[0135] exist Figure 3 In the embodiment, the vehicle driving control device 100 includes the sound sensor 110 , but is not limited thereto. The vehicle driving control device 100 may be configured as a device separate from the sound sensor 110 and the driver 120 .
[0136] Fig.16 is a block diagram of a vehicle driving control device 100 a according to the embodiment.
[0137] Compare Figure 3 and Fig.16 , the vehicle driving control device 100a may further include a communication interface 160 for communicating with the sound sensor 110 and the driver 120. The communication interface 160 may apply the result received from the sound sensor 110 to the processor 150, and may send a control instruction of the processor 150 to the driver 120. The communication interface 160 may include a short-range wireless communication interface, a mobile communication interface, and the like. The vehicle driving control device 100a may be a cellular phone as an electronic device separate from the vehicle 10. In addition, the vehicle driving control device 100a may provide various information to an external output device through the communication interface 160 without including the output interface 130.
[0138] The disclosed embodiments may be implemented in a software program including instructions stored on a computer-readable storage medium.
[0139] As a device capable of calling stored instructions from a storage medium and performing operations according to the disclosed embodiments according to the called instructions, a computer may include the image sending device and the image receiving device according to the disclosed embodiments.
[0140] The computer-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means that the storage medium does not include a signal and is tangible, but it is not possible to distinguish whether data is stored on the storage medium semi-permanently or temporarily.
[0141] In addition, the electronic device or method according to the disclosed embodiment may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product.
[0142] A computer program product may include a software program and a computer-readable storage medium on which the software program is stored. For example, a computer program product may include a product (e.g., a downloadable application) in the form of a software program distributed electronically via a manufacturer of an electronic device or an electronic market (e.g., Google PlayStore or App Store). For electronic distribution, at least a portion of the software program may be stored on a storage medium, or may be temporarily generated. In this case, the storage medium may be a storage medium of a manufacturer's server, a server of an electronic market, or a relay server for temporarily storing software programs.
[0143] In a system consisting of a server and a terminal (e.g., a vehicle driving control device 100 or a navigation server), the computer program product may include a storage medium of the server or a storage medium of the terminal. Alternatively, when there is a third device (e.g., a smart phone) that communicates with the server or the terminal, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include the software program itself that is sent from the server to the terminal or the third device or from the third device to the terminal.
[0144] In this case, one of the server, the terminal, and the third device may execute the computer program product to perform the method according to the disclosed embodiment. Alternatively, two or more of the server, the terminal, and the third device may execute the computer program product to perform the method according to the disclosed embodiment in a distributed manner.
[0145] For example, a server (e.g., a cloud server or an artificial intelligence server) may execute a computer program product stored on the server to control a terminal communicating with the server to perform a method according to the disclosed embodiments.
[0146] As another example, the third device may execute a computer program product to control a terminal communicating with the third device to execute a method according to the disclosed embodiment. As a specific example, the third device may remotely control an image sending device or an image receiving device to send or receive a packaged image.
[0147] When the third device executes the computer program product, the third device may download the computer program product from the server and execute the downloaded computer program product. Alternatively, the third device may execute a computer program product provided in a preloaded manner to execute the method according to the disclosed embodiment.
[0148] While the present disclosure has been shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the following claims.
Claims
1. A vehicle driving control device, include: a communication interface configured to receive a signal corresponding to a sound generated in an external environment from a sound sensor; as well as The processor is configured as: identifying the sound object by obtaining, based on the received signal, a type of the sound object generating the sound, and either or both of a direction of the sound object and a distance from the sound object to a vehicle including the vehicle driving control device; as well as controlling driving of the vehicle based on the identified sound object, The processor is further configured to identify the type of the sound object by performing the following operations: receiving a frequency-divided electrical signal from the sound sensor, the electrical signal corresponding to a plurality of frequencies; determining at least one frequency among the plurality of frequencies whose intensity variation is equal to or greater than a reference range; generating a sound pattern corresponding to the at least one frequency having an intensity variation equal to or greater than a reference range; and The type of the sound object is identified by comparing the sound pattern with a sound template.
2. The vehicle driving control device according to claim 1, in, The processor is further configured to change either or both of a driving path of the vehicle and a speed of the vehicle based on the identified sound object.
3. The vehicle driving control device according to claim 1, in, The processor is also configured to control driving of the vehicle to prevent a collision between the vehicle and the identified sound object.
4. The vehicle driving control device according to claim 1, further comprising: include: The output interface is configured to output information of any one item or any combination of the type of the sound object, the direction of the sound object, and the distance from the sound object to the vehicle.
5. The vehicle driving control device according to claim 4, in, The output interface includes: a display configured to display an image related to the external environment, wherein the image includes information on the type of the sound object.
6. The vehicle driving control device according to claim 5, in, The display is also configured to display a relative position of the sound object with respect to the vehicle on the displayed image.
7. The vehicle driving control device according to claim 1, in, The sound sensor is mounted on the vehicle.
8. The vehicle driving control device according to claim 1, in, The sound sensor includes a plurality of vibrators arranged in different directions and having different resonance frequencies, and At least two vibrators of the plurality of vibrators are configured to vibrate in response to the sound.
9. The vehicle driving control device according to claim 8, in, Any one or any combination of the plurality of vibrators is arranged parallel to a traveling direction of the vehicle.
10. The vehicle driving control device according to claim 8, in, The processor is further configured to: obtaining a frequency band of the sound based on a resonance frequency of a vibrator that is vibrating among the at least two vibrators among the plurality of vibrators; Based on the obtained frequency band of the sound, obtaining the type of the sound object; obtaining a direction of the sound object relative to the vehicle based on a direction of the vibrating vibrator; obtaining a distance from the sound object to the vehicle based on an output intensity of the vibrating vibrator; as well as Based on the change over time in the output intensity of the vibrating vibrator, a change in the distance from the sound object to the vehicle is obtained.
11. The vehicle driving control device according to claim 1, in, The sound object is not recognized by an optical sensor mounted on the vehicle.
12. The vehicle driving control device according to claim 11, in, The sound object is in the external environment of the vehicle in an area outside the line of sight of the optical sensor.
13. The vehicle driving control device according to claim 11, in, The optical sensor includes an image sensor.
14. The vehicle driving control device according to claim 11, in, The processor is further configured to: Based on the result output from the sound sensor, obtaining one or more candidate sound objects that potentially generate the sound; and A candidate sound object among the one or more candidate sound objects that is not recognized by the optical sensor is recognized as the sound object.
15. A method for controlling the driving of a vehicle, the method include: receiving a signal corresponding to a sound generated in an external environment from a sound sensor; identifying the sound object by obtaining, based on the received signal, a type of the sound object generating the sound, and either or both of a direction of the sound object and a distance from the sound object to the vehicle; as well as controlling driving of the vehicle based on the identified sound object, The type of the sound object is identified by the following operations: receiving a frequency-divided electrical signal from the sound sensor, the electrical signal corresponding to a plurality of frequencies; determining at least one frequency among the plurality of frequencies whose intensity variation is equal to or greater than a reference range; generating a sound pattern corresponding to the at least one frequency having an intensity variation equal to or greater than a reference range; and The type of the sound object is identified by comparing the sound pattern with a sound template.
16. The method according to claim 15, in, Controlling the driving of the vehicle includes changing any one or any combination of a driving route of the vehicle and a speed of the vehicle based on the identified sound object.
17. The method according to claim 15, in, Controlling the driving of the vehicle includes controlling the driving of the vehicle so that the vehicle and the recognized sound object do not collide with each other.
18. The method according to claim 15, further comprising: include: Information of any one or any combination of the type of the sound object, the direction of the sound object, and the distance from the sound object to the vehicle is output.
19. The method according to claim 18, in, The outputting includes: displaying an image related to the external environment, the image including information on the type of the sound object.
20. The method according to claim 15, in, The sound object is not recognized by an optical sensor mounted on the vehicle.
21. The method according to claim 20, in, Identifying the sound object comprises: Based on the result output from the sound sensor, obtaining one or more candidate sound objects that potentially generate the sound; and A candidate sound object among the one or more candidate sound objects that is not recognized by the optical sensor is recognized as the sound object.
22. The method according to claim 15, in, The sound sensor is mounted on the vehicle and includes a plurality of vibrators arranged in different directions and having different resonance frequencies, and At least two vibrators of the plurality of vibrators are configured to vibrate in response to the sound.
23. The method according to claim 22, in, Identifying the sound object comprises: obtaining a frequency band of the sound based on a resonance frequency of a vibrator that is vibrating among the at least two vibrators among the plurality of vibrators; Based on the obtained frequency band of the sound, obtaining the type of the sound object; obtaining a direction of the sound object relative to the vehicle based on a direction of the vibrating vibrator; obtaining a distance from the sound object to the vehicle based on the output intensity of the vibrating vibrator; and Based on the change over time in the output intensity of the vibrating vibrator, a change in the distance from the sound object to the vehicle is obtained.
24. A vehicle driving control device for a vehicle, the vehicle driving control device include: a sound sensor configured to obtain a signal corresponding to a sound generated by a sound object in an external environment; as well as The processor is configured as: Based on the obtained signal, obtain the type of the sound object, the direction of the sound object, and the distance between the sound object and the vehicle; as well as controlling the driving of the vehicle based on the obtained type of the sound object, the obtained direction of the sound object, and the obtained distance, The processor is further configured to identify the type of the sound object by performing the following operations: receiving a frequency-divided electrical signal from the sound sensor, the electrical signal corresponding to a plurality of frequencies; determining at least one frequency among the plurality of frequencies whose intensity variation is equal to or greater than a reference range; generating a sound pattern corresponding to the at least one frequency having an intensity variation equal to or greater than a reference range; and The type of the sound object is identified by comparing the sound pattern with a sound template.
