Sound source visualization device and method

Abstract

The present invention relates to a sound source visualization device and method. According to an embodiment, the sound source visualization device includes: a sound source sensing module that senses a sound source signal including surrounding sounds, sound sources, and noise through multiple sound source sensing sensors; a preprocessing module that filters noise from the sensed sound source signal and amplifies the sound source signal; a calculation module that analyzes the preprocessed sound source signal and calculates a simple sound source location including the separation distance, direction, sound source intensity, two-dimensional position, and height information of the sound source from the sound source visualization device; a search module that uses the height information of the sound source sensed by each of the sound source sensing sensors to generate multiple pseudo planes including the height information, screens the multiple pseudo planes, and applies the height information obtained using the selected planes to the simple sound source location to generate three-dimensional sound source location information; and a visualization module that outputs the sound source information including the three-dimensional sound source location information, sound source height information, and sound source intensity information to a system host, or converts the sound source information including the three-dimensional sound source location information, sound source height information, and sound source intensity information to a visualization signal and displays it.

Description

Technical Field

[0001] The present invention relates to a sound source visualization device and method, and in particular to a sound source visualization device and method that uses multiple sound source sensing sensors to grasp sound source information such as sound source position, height, direction, intensity, and interval distance, allowing a user to visually and intuitively identify the position, height, intensity, and interval distance of the sound source. Background Art

[0002] Unless otherwise indicated in this specification, the content described in this section is not prior art to the claims in this application and is not intended to be prior art even if included in this section.

[0003] Self-driving cars are vehicles that can drive themselves without the driver operating the steering wheel, brakes, or pedals. They use sensors to understand surrounding conditions, avoid obstacles, and choose the optimal route to their destination. Self-driving cars require systems such as highway driving assistance, lane departure warning systems, lane keeping assistance systems, rear cross-traffic alert systems, advanced smart cruise control to maintain a constant distance between vehicles, and automatic emergency braking systems. Self-driving cars are at the core of future automotive technology and are an area of ​​active research and development for multinational companies such as Google.

[0004] The technological development stages of autonomous vehicles are generally categorized into levels 0 to 5. Level 0 represents no autonomous driving technology, while Level 1 requires driver intervention. Level 2 allows the car to autonomously steer and accelerate and decelerate, while Level 3 also allows it to avoid road obstacles. Starting at Level 3, the driver's attention is significantly reduced due to the shift in monitoring criteria from human to system. Level 4 addresses a wide range of road and driving conditions, while Level 5 achieves fully autonomous driving without the need for a driver's license. Autonomous vehicles achieving Level 3 and 4 technology are now gradually entering the market.

[0005] For existing Level 3 autonomous and self-driving car technologies, obstacles and surrounding objects are primarily detected visually using sensors such as cameras. However, in real-world driving, humans use sound to identify surrounding objects and often use it to detect critical situations. For example, emergency signals such as the sounds of objects, collisions, impacts, and sirens are important sound sources that drivers must interpret.

[0006] However, the trend has been to develop technologies that utilize visual signals to judge driving conditions. To achieve Level 5 autonomous driving technology that is as perfect as human-driven driving in the future, sound source signal processing and visualization technologies for autonomous vehicles are required. Summary of the Invention

[0007] Technical issues

[0008] The present invention provides a sound source visualization device and method that uses multiple sound source sensing sensors to identify various sound source information such as the direction, position, height, intensity, and interval distance of the sound source, allowing users to intuitively and quickly identify the position of the sound source.

[0009] Furthermore, the height of the sound source is included in the simple sound source position information including the direction, intensity, and position of the sound source, or by correcting the sound source height information and position information, the user can identify the surrounding sound source positions in a three-dimensional manner including the sound source height information.

[0010] Furthermore, all sound source information, such as the direction of movement, movement, and size of the sound source, is visualized, allowing for a more intuitive understanding of the size and dangerousness of the sound source.

[0011] Technical Solution

[0012] According to an embodiment, a sound source visualization device includes: a sound source sensing module that senses sound source signals including ambient sounds, sound sources, and noise using a plurality of sound source sensing sensors arranged diagonally at different heights; a preprocessing module that filters noise from the sensed sound source signals and amplifies the sound source signals; a calculation module that analyzes the preprocessed sound source signals and calculates a simple sound source location including a separation distance, direction, sound source intensity, two-dimensional position, and height information of the sound source from the sound source visualization device; a search module that generates a plurality of pseudo planes including height information using height information of the sound source sensed by each of the sound source sensing sensors, selects the plurality of pseudo planes, and generates three-dimensional sound source location information by including height information obtained using a selected plane in the simple sound source location; and a visualization module that outputs the sound source information including three-dimensional sound source location information, sound source height information, and sound source intensity information to a system host, or converts the sound source information including three-dimensional sound source location information, sound source height information, and sound source intensity information to a visualization signal for display.

[0013] According to another embodiment, a method for visualizing a sound source includes the following steps: (A) sensing a sound source signal including surrounding sounds, sound sources, and noise in a sound source sensing module through a plurality of sound source sensing sensors; (B) filtering noise from the sound source signal sensed in a preprocessing module and amplifying the sound source signal; (C) analyzing the preprocessed sound source signal in a calculation module and calculating a simple sound source position including a separation distance, direction, sound source intensity, two-dimensional position, and height information of the sound source from the sound visualization device; (D) generating a plurality of pseudo-planes including the height information using the height information of each sound source sensed in the sound source sensing sensor in a search module, screening the plurality of pseudo-planes, and including the height information obtained from the selected plane in the simple sound source position to generate three-dimensional sound source position information; and (E) outputting the sound source information including the three-dimensional sound source position information, the sound source height information, and the sound source intensity information to a system host in a visualization module, or converting the sound source information into a visualization signal and displaying it.

[0014] Beneficial effects

[0015] The sound source visualization device and method described above can be used to obtain a variety of sound source information including the height, direction, distance from the user, position, and intensity of the sound source, so that the existence of objects around the user and danger signals can be more accurately understood through the sound source.

[0016] Furthermore, in an autonomous vehicle, by visually converting surrounding sound source signals and providing them as visual information to the system or a fellow driver, the system or a fellow driver can grasp emergency situations based on the movements or sounds of surrounding objects that cannot be identified by cameras or vision.

[0017] Furthermore, a pseudo plane representing the sound source position in three dimensions is generated for each sound source sensing sensor and unified to correct the sound source height information. This allows accurate sound source height information to be provided to the user even when the sound source does not exist on a plane.

[0018] Furthermore, when the sound source visualization apparatus according to the embodiment is provided to a hearing-impaired user or an elderly person, the hearing-impaired person can visually grasp the sound source signal.

[0019] Furthermore, when the sound source visualization device according to the embodiment is provided in exercise assisting equipment for the elderly, etc., the elderly with relatively poor hearing can grasp auditory information through vision, thereby preventing safety accidents.

[0020] The effects of the present invention are not limited to the above-mentioned effects, but should be understood to include all effects that can be inferred from the detailed description of the present invention or the configuration of the invention described in the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a diagram illustrating a sound source visualization device according to an embodiment.

[0022] Figure 2 : is a diagram showing a data processing block of the sound source visualization apparatus according to the embodiment.

[0023] Figure 3 1 is a diagram showing more specific data processing blocks of the sound source visualization device according to the embodiment.

[0024] Figure 4 FIG. 1 is a diagram illustrating a visualization template of a sound source visualization apparatus according to an embodiment.

[0025] Figure 5 is a flowchart illustrating a sound source information visualization process according to an embodiment.

[0026] Figure 6 FIG. 1 is a diagram further specifically illustrating the simplified sound source position information generation and extraction signal correction process according to the embodiment.

[0027] Figure 7 is a flowchart illustrating in further detail the process of generating and correcting sound source position information according to an embodiment.

[0028] Figure 8 is a diagram for explaining the error correction process according to the embodiment in more detail.

[0029] Figure 9a It is a diagram for explaining the distance between sound sources and the position correction calculation process according to the embodiment.

[0030] Figure 9b This is a diagram for explaining a method for calculating a simple sound source position including a separation distance, direction, sound source intensity, two-dimensional position, and height information of the sound source in a calculation module and generation of a false plane according to an embodiment.

[0031] Figure 10 is a flowchart illustrating a sound source information visualization process according to an embodiment.

[0032] Best Practice

[0033] According to an embodiment, a sound source visualization device includes: a sound source sensing module that senses sound source signals including ambient sounds, sound sources, and noise using a plurality of sound source sensing sensors arranged diagonally at different heights; a preprocessing module that filters noise from the sensed sound source signals and amplifies the sound source signals; a calculation module that analyzes the preprocessed sound source signals and calculates a simple sound source location including a separation distance, direction, sound source intensity, two-dimensional position, and height information of the sound source from the sound source visualization device; a search module that generates a plurality of pseudo planes including height information using height information of the sound source sensed by each of the sound source sensing sensors, selects the plurality of pseudo planes, and generates three-dimensional sound source location information by including height information obtained using a selected plane in the simple sound source location; and a visualization module that outputs the sound source information including three-dimensional sound source location information, sound source height information, and sound source intensity information to a system host, or converts the sound source information including three-dimensional sound source location information, sound source height information, and sound source intensity information to a visualization signal for display. DETAILED DESCRIPTION

[0034] References and Attachments Figure 1 The advantages and features of the present invention, as well as methods for achieving these advantages, will be made clearer by the following detailed embodiments. However, the present invention can be implemented in a variety of different forms and is not limited to the embodiments disclosed below. These embodiments are provided solely to complete the disclosure of the present invention and to fully inform those skilled in the art of the present invention of the scope of the invention. The present invention is defined solely by the scope of the claims. Throughout this specification, the same reference numerals refer to the same components.

[0035] When describing the embodiments of the present invention, if it is determined that a detailed description of a related known function or structure may cause unnecessary confusion to the main purpose of the present disclosure, the detailed description will be omitted. Furthermore, the terms described below are defined based on the functions in the embodiments of the present invention and may vary depending on the intentions or practices of the user or operator. Therefore, it is necessary to define them based on the content throughout this specification.

[0036] Figure 1 is a diagram illustrating a sound source visualization device according to an embodiment.

[0037] Reference Figure 1 According to the embodiment, the sound source visualization device identifies the sound source information including the size, position, direction, and height of the sound source around the user, converts the sound source information into a visual object, and displays it to the user. In the embodiment, although a clock-shaped radial visualization template is proposed, the sound source visualization template is not limited to Figure 1In an embodiment, for a sound source sensed by multiple sound source identification sensors, each item of sound source information such as the size, direction, and distance from the user of the sound source can be converted into a visualization object corresponding to the sound source visualization template, thereby providing the user with detailed sound source information in a visual manner. Figure 1 As shown, the visualization template of the sound source information can be displayed on the upper surface (A) or the side surface (B) of the sound source visualization device.

[0038] The sound source visualization device according to the embodiment can be applied to various systems and devices such as autonomous driving vehicles. In this case, the sound source visualization device can display the sound source information through a visualization template pre-set in the system.

[0039] Figure 2 : is a diagram showing a data processing block of the sound source visualization apparatus according to the embodiment.

[0040] Reference Figure 2 According to the embodiment, the sound source visualization device can be configured to include a sound source sensing module 210, a pre-processing module 230, a calculation module 250, a search module 270, and a visualization module 290. The term "module" used in this specification should be interpreted as including software, hardware, or a combination thereof according to the context in which the term is used. For example, software can be machine language, firmware, embedded code, or application software. As another example, hardware can be a circuit, a processor, a computer, an integrated circuit, an integrated circuit core, a sensor, a micro-electro-mechanical system (MEMS), a passive device, or a combination thereof.

[0041] The sound source sensing module 210 uses a plurality of sound source sensing sensors arranged diagonally at the same height or at different heights to sense sound source signals including surrounding sounds, sound sources, and noise. In an embodiment, each of the four sound source sensing sensors can sense surrounding sounds, sound sources, and noise.

[0042] The pre-processing module 230 filters noise from the sensed sound source signal and amplifies the sound source signal. For example, the pre-processing module 230 can master the frequency and intensity of the sound source collected by the sensor to filter out the noise signal and amplify the main sound source signal to above a predetermined decibel or level.

[0043] The calculation module 250 analyzes the preprocessed sound source signal and calculates a simplified sound source location, including the distance, direction, sound source intensity, two-dimensional position, and simplified height information of the sound source. In an embodiment, the simplified sound source location is obtained by preliminarily calculating sound source information including the relative position, size, and height of the sound source at the user's location based on the sound source signal received from the preprocessing module 230.

[0044] The search module 270 generates multiple pseudo planes using the height information of the sound source sensed by each sound source sensing sensor, selects the multiple pseudo planes, and includes the height information obtained from the selected planes in the sound source simplified location, thereby generating three-dimensional sound source location information. In an embodiment, the search module 270 corrects the sound source height information and sound source location information preliminarily calculated by the calculation module 250. Specifically, the search module 270 can use the sound source information sensed by each sensor to generate a pseudo plane including the location (x, y) coordinates of the sound source and the height (z) coordinates of the sound source, and select each generated plane from the sensor to correct the sound source height information. In an embodiment, the sound source height information can be corrected using the average, median, mode, or mean of the z coordinates of each plane generated by the sensor as the sound source height information. In an embodiment, the corrected information can be generated as the sound source height information.

[0045] The visualization module 290 outputs the sound source information including the three-dimensional sound source position information, the sound source height information and the sound source intensity information to the system host or converts the sound source information into a visualization signal and a visualization object corresponding to the visualization template and displays the information.

[0046] Figure 3 1 is a diagram showing more specific data processing blocks of the sound source visualization device according to the embodiment.

[0047] Reference Figure 3 According to the embodiment, the calculation module 250 may include a sound source intensity calculation unit 251 and a sound source position calculation unit 253, the search module 270 may include a sound source height search unit 271 and a correction unit 273, and the visualization module 290 may include a conversion unit 291, a display unit 293 and a sound source height display unit 295.

[0048] The sound source intensity calculation unit 251 of the calculation module 250 grasps the sound source data and the size of the sound source signal identified by the sound source sensing sensor, and calculates the sound source intensity level corresponding to the grasped size. For example, the sound source intensity calculation unit 251 grasps the sound source intensity and calculates the sound source intensity level corresponding to the grasped sound source intensity. Specifically, the sound source intensity level can be calculated in such a way that 0-50 decibels corresponds to level 1, 51-100 decibels corresponds to level 2, and 101-150 decibels corresponds to level 3, and the range and interval of the sound source intensity level can be adjusted when designing the system. In an embodiment, each of the set sound source levels can be matched with a color, the brightness of the lighting (lux), or the flicker frequency, so that the intensity of the sound source can be displayed visually.

[0049] The sound source position calculation unit 253 analyzes each sound source sensing signal acquired by the sound source sensing sensor and processed in the pre-processing module to calculate the occurrence position and movement position of the sound source.

[0050] The sound source height search unit 271 of the search module 270 uses the calculated sound source position information and the height data of the sound source included in the sound source information sensed by each of the sound source sensing sensors to generate a plane representing the height information of the sound source recognized by each of the sound source sensing sensors, and extracts information from the generated plane to grasp the height of the sound source.

[0051] The correction unit 273 adds the searched sound source height information to the simple sound source position information, or corrects the generated sound source position, sound source distance, intensity, and sound source height.

[0052] The conversion unit 291 of the visualization module 290 converts the direction of the sound source and the distance between the user and the sound source, identified based on a preset visualization template such as a radial pattern, into a visualization object corresponding to the preset visualization template. Alternatively, the converted sound source information is output to the system host.

[0053] The display unit 293 matches the color or light intensity corresponding to the intensity of the recognized sound source, and displays the visualized object while changing the color or light brightness according to the intensity of the sound source.

[0054] The sound source height display unit 295 converts the acquired sound source height into an angle corresponding to the visualization template and displays the converted angle, or displays the height information of the sound source together with the visualization object converted from the sound source.

[0055] Figure 4 FIG. 1 is a diagram illustrating a visualization template of a sound source visualization apparatus according to an embodiment.

[0056] Reference Figure 4In an embodiment, the sound source visualization template may be constructed in a radial form. The center of the radial template represents the user position, and the sound source position relative to the user position may be calculated and displayed as a visual object.

[0057] A radial visualization template can convert the direction and distance of a sound source from the user into a clockwise direction and the diameter of a circle for relative display. Furthermore, in embodiments, impact sounds, friction sounds, and sound sources above a certain intensity can be visualized as impacts, or the intensity of the sound source can be matched with color, a flashing light frequency, and other factors. Utilizing the radial sound source visualization template according to embodiments, in an autonomous vehicle, surrounding sound source signals are visually converted and provided as visual information to the vehicle's main system or a fellow passenger, allowing the driver to identify the movement or situation of surrounding objects that are not visible through a camera. Specifically, when the sound source visualization method and program according to embodiments are utilized in a system or device for an autonomous vehicle, the positional information of surrounding objects identified as sound sources can be output separately from the template provided by the visualization device according to embodiments. For example, in a system equipped with a sound source visualization program, information about surrounding objects identified through sound source information can be output using a pre-configured template, or as audio or visual information.

[0058] Furthermore, when the sound source visualization template and the sound source visualization device according to the embodiment are set in elderly exercise assisting equipment, etc., elderly people with relatively poor hearing can visually recognize auditory information, thereby preventing safety accidents.

[0059] Figure 5 is a flowchart illustrating a sound source information visualization process according to an embodiment.

[0060] Reference Figure 5 In step S510, the sound source sensing module uses multiple sound source sensing sensors to sense sound source signals including surrounding sounds, sound sources, and noise. In step S530, the preprocessing module filters noise from the sensed sound source signals and amplifies the sound source signals. In step S550, the calculation module analyzes the preprocessed sound source signals and calculates a simplified sound source location from the sound source visualization device, including the distance, direction, sound source intensity, two-dimensional position, and height information of the sound source, thereby determining the intensity of the sound source around the user. In step S570, the search module uses the height information of each sound source sensed by the sound source sensing sensors to generate multiple pseudo planes including height information, selects the multiple pseudo planes, and applies the height information obtained from the selected planes to the simplified sound source location to generate three-dimensional sound source position information. In step S590, the visualization module converts the sound source information, including three-dimensional sound source position information, sound source height information, and sound source intensity information, into a visual signal and displays it.

[0061] Figure 6 FIG. 1 is a diagram further specifically illustrating the process of generating simplified sound source position information and correcting extracted signals according to an embodiment.

[0062] Reference Figure 6 In step S551, the signals extracted after the preprocessing of each sensor are combined in the calculation module to calculate the sound source intensity, and the intensity level is generated according to each sound source.

[0063] In step S552, the calculation module uses the sound source signals extracted by each sensor to generate the respective inter-aural time differences (ITDs). The ITD, a time difference caused by the path difference between the sound source's location and the two sensors, primarily helps perceive the location of horizontal sound sources. In this embodiment, generating ITDs for the same sound source from each sensor enables more accurate calculation of the distance between the sound source and the user, as well as ascertainment of the sound source's direction and location.

[0064] In step S553, the search module excludes characteristics related to altitude based on the signals input from the four sound sensors and searches for directions on the horizontal plane assuming the same horizontal plane, thereby identifying a simplified position.

[0065] In step S554 , the same horizontal plane including the height is generated using the position identified by the position search, and the height with respect to the sound source is searched.

[0066] In step S555, the generated simplified positions and simplified heights can be used to create a plane alignment for each sensor, enabling signal extraction. For example, a plane z coordinate can be generated for each sensor, and the sound source height information used to achieve plane alignment can be calculated using the mode, median, or average values. In step S557, the signals of each sensor are reconstructed from the sensor positions reconstructed through plane alignment to extract the signal. In step S559, the extracted signal is corrected and output through verification and simulation of the extracted data.

[0067] Figure 7 is a flowchart illustrating in further detail the process of generating and correcting sound source position information according to an embodiment.

[0068] In step S575 , the calculation module calculates the sound source position using the ITD generated from each sensor.

[0069] In step S577, in the height calculation of the calculation module, the height of the sound source is calculated using the generated ITD and the position calculation result.

[0070] In step S579 , the distance between users is calculated using the ITD generated by each sensor and the calculated position and height information of the sound source.

[0071] In step S581, the calculation module corrects the error in the sound source location information that occurred during the height calculation process in step S577. It also corrects the error between the sound source location and the user that occurred during the distance calculation process in step S579. After error correction is complete, the process proceeds to step S590, where the sound source information is converted into a visual object.

[0072] Figure 8 is a diagram for explaining the error correction process according to the embodiment in more detail.

[0073] Reference Figure 8 In step S583, the sound source signal extracted after being corrected and pre-processed by the calculation module is position-corrected in the correction unit to be a final signal and output.

[0074] In step S585, the calibration unit performs altitude calibration, correcting the altitude calculation result based on the position identified by the position calibration and outputting the result. In step S587, the calibration unit corrects the distance between the user and the sound source by reflecting the position calibration result and the altitude calibration result. In step S589, the calibration result of the calibration unit is finally confirmed and outputted.

[0075] Figure 9a It is a diagram for explaining the distance between sound sources and the position correction calculation process according to the embodiment.

[0076] Reference Figure 9a , when the sound source sensing sensors equipped in the sound source visualization device according to the embodiment are A, B, C, and D, and the sound source is S, the arrival time from the sound source to each sensor and the distance from each sensor to the sound source are measured to be all different. If calculated based on the sound speed of 340m / s, the distances a, b, and c between each sensor and the sound source can be grasped based on the time difference when each sensor arrives at a specific sound source. If the ITD of sensor D and sensor C is known, the distance b can be regarded as the same as the distance between the sound source S and point p (the point closest to sensor D on the line connecting the sound source S and sensor C) and the distance y can be calculated. Since the distance x is a known value, the angle θ can be calculated assuming that y and z form a right angle, thereby grasping the direction and height of the sound source. In Figure 9aThe method for calculating angle θ shown in the figure assumes that the sound source S is separated by a distance that is sufficiently large to negligibly account for the difference between the distance b and the distance between the sound source S and point p. Therefore, in this embodiment, angle θ can be roughly calculated by calculating y and z based on an indicator such as the ITD generated based on the distance between the sound source S and each sensor, thereby determining the direction and height of the sound source. Furthermore, in this embodiment, during the sound source height and position correction process, the sound source height and direction can be corrected by recalculating y and z or resetting y and z to correct angle θ.

[0077] Figure 9b This is a diagram for explaining a method of calculating a simple sound source position including a separation distance, direction, sound source intensity, two-dimensional position, and height information of the sound source in a calculation module according to an embodiment.

[0078] In the calculation module 250 according to the embodiment, the intensity of the sound source can be detected by adding the sound source signals from each sensor. For example, the calculation module 250 adds or compares the signal outputs of the four sensors to determine the intensity and basic direction of the sound source, and calculates the two-dimensional simple occurrence position, direction and separation distance by triangulation using the two ITDs obtained from the three sensors, wherein the three sensors are formed at right angles by three of the four sensors. Figure 9b The sound source intensity is selected based on the sound source intensity calculated from the BDC, DCA, CAB, or ABD sensor combination. In one embodiment, the direction of movement is determined by comparing the front and rear positions obtained by periodically calculating the occurrence position. Furthermore, the height information of the sound source can be simply calculated by comparing the distance calculated by the ITD (Identity Detection) obtained from two diagonally positioned sensors in the sensor combination with a predetermined diagonal distance in the module.

[0079] Furthermore, in an embodiment, the search module can obtain the height information of the sound source as described below. As described above, the search module also adds or compares the signal outputs of the four sensors to determine the intensity and basic direction of the sound source, and uses the two ITDs obtained from the three sensors to calculate the two-dimensional simple occurrence position, direction and separation distance by triangulation, wherein the three sensors are selected from a sensor combination of BDC, DCA, CAB or ABD that form three right angles among the four sensors. At this time, the moving direction is obtained by comparing the results of the front and back positions obtained in the calculation of the periodic occurrence position. Furthermore, the height information of the sound source can be simply calculated by comparing the distance calculated by the ITD with the predetermined diagonal distance in the module, and the ITD is obtained from the two sensors arranged on the diagonal line in the combination of sensors. Afterwards, the search module generates multiple pseudo planes, screens the generated multiple pseudo planes, and then obtains the height information using the selected planes. As Figure 9bAs shown, in the embodiment, a plurality of planes constituted as ADC' are generated by three ADC sensors, and height information and position information obtained by using a plane selected from the plurality of planes are generated.

[0080] Figure 10 is a flowchart illustrating a sound source information visualization process according to an embodiment.

[0081] In step S591 , according to a preset radial visualization template, the identified direction of the sound source and the distance between the user and the sound source are converted into a visualization object corresponding to the preset radial visualization template.

[0082] In step S593 , the color or light intensity corresponding to the intensity of the recognized sound source is matched, and the color or light brightness of the visualized object is changed and displayed according to the intensity of the sound source.

[0083] In step S595, the corrected height information of the sound source is displayed together with the visual object after the sound source conversion. In the embodiment, the display device can use not only LED but also LCD, OLED, etc.

[0084] The aforementioned sound source visualization device and method can capture a variety of sound source information, including the height, location, and intensity of the sound source, thereby more accurately detecting the presence of objects around the user and danger signals based on the sound source. Furthermore, by generating a pseudo plane that includes the sound source location in three dimensions for each sound source sensing sensor and aligning them to correct the sound source height information, the user can accurately be provided with sound source height information even when the sound source is not located on a plane.

[0085] Furthermore, when the sound source visualization device according to the embodiment is provided to hearing-impaired users or elderly individuals, the sound source signal is visualized and provided to surrounding objects and sound source information, enabling rapid understanding of the visualized auditory information. Furthermore, the sound source visualization process can be applied to autonomous vehicles to understand surrounding objects.

[0086] The disclosed content is merely illustrative and can be variously modified and implemented by persons having ordinary knowledge in the technical field without departing from the spirit of the claims. Therefore, the scope of protection of the disclosed content is not limited to the above-mentioned specific embodiments.

[0087] Industrial applicability

[0088] In an autonomous vehicle, by visually converting surrounding sound source signals and providing the sound source information as visual information to the system or a fellow driver, the system or the fellow driver can grasp an emergency situation based on the movements or sounds of surrounding objects that cannot be identified by a camera or vision.

[0089] Furthermore, a pseudoplane representing the sound source position in three dimensions is generated for each sound source sensing sensor and unified to correct the sound source height information, thereby accurately providing the user with sound source height information even when the sound source does not exist on a plane. When the sound source visualization device according to the embodiment is provided to users with poor hearing and the elderly, people with poor hearing can grasp the sound source signal through vision. When the sound source visualization device according to the embodiment is set in a sports assist device for the elderly, etc., the elderly with relatively poor hearing can recognize the auditory information through vision, thereby preventing safety accidents.

Claims

1. A sound source visualization device, comprising: The sound source sensing module uses four sound source sensing sensors to sense sound source signals including surrounding sounds, sound sources and noise; a pre-processing module, filtering noise from the sensed sound source signal and amplifying the sound source signal; a calculation module that analyzes the preprocessed sound source signal and calculates a simplified sound source location including a separation distance, direction, sound source intensity, two-dimensional position, and height information of the sound source from the sound source visualization device; a search module that generates a plurality of pseudo planes reflecting the height information using the height information of the sound source sensed by each of the sound source sensing sensors, and selects the plurality of pseudo planes so that the height information obtained using the selected pseudo planes is included in the sound source simplified position to generate three-dimensional sound source position information and height information; as well as The visualization module outputs the sound source information including the three-dimensional sound source position information, sound source height information and sound source intensity information to a preset system host, or converts it into a visual signal and displays it, The calculation module includes: The sound source intensity calculation unit grasps the sound source data and the magnitude of the sound source signal recognized by the sound source sensing sensor, and calculates a sound source intensity level corresponding to the magnitude of the grasped sound source signal. The search module includes: a sound source height search unit that generates a plurality of pseudo planes including z-coordinates as height information of the sound source recognized by each of the sound source sensing sensors using the height data of the sound source included in the sound source information sensed by each of the sound source sensing sensors, and filters the generated pseudo planes to grasp the sound source height; and A correction unit, which corrects the sound source position, sound source distance, sound source intensity and sound source height; The four sound source sensing sensors have a plurality of sensor combinations consisting of three sensors forming a right angle, By using the time difference of the sound source obtained from three sensors selected from any one of the plurality of sensor combinations, the two-dimensional simplified occurrence position, direction and separation distance of the sound source are calculated by triangulation. The height information of the sound source is simply calculated by comparing a distance calculated from a time difference of the sound source obtained from two sensors disposed at different heights on a diagonal line in the selected sensor combination with a diagonal distance between the two sensors, The correction unit corrects the sound source height information using an average value, a median value, a mode, or a simulation of the z coordinates as the sound source height information in the pseudo plane selected by the sound source height search unit. The visualization module includes: a conversion unit, which converts the identified direction of the sound source and the distance between the user and the sound source into a visualization object corresponding to the preset visualization template according to the preset visualization template; a sound source intensity display unit that matches a color or light intensity corresponding to the intensity of the recognized sound source, and displays the color or light brightness of a visualized object by changing the color or light brightness according to the intensity of the sound source; and The sound source height display unit displays corrected height information of the sound source together with a visualized object converted from the sound source.

2. The sound source visualization device according to claim 1, characterized in that The calculation module also includes: The sound source position calculation unit analyzes the sound source sensing signals obtained by each of the sound source sensing sensors and the sound source signal transmitted from the pre-processing module to calculate the occurrence position and movement direction of the sound source.

3. A sound source visualization method, the sound source visualization method comprising the following steps: (A) In the sound source sensing module, four sound source sensing sensors are used to sense sound source signals including surrounding sounds, sound sources, and noise; (B) filtering noise from the sound source signal sensed in the pre-processing module and amplifying the sound source signal; (C) analyzing the pre-processed sound source signal in the calculation module and calculating a simplified sound source location including a distance, direction, sound source intensity, two-dimensional position, and height information of the sound source from the sound visualization device; (D) generating, in a search module, a plurality of pseudo planes using the height information of each sound source sensed by the sound source sensing sensor, screening the plurality of pseudo planes, and generating three-dimensional sound source position information by including the height information obtained from the selected pseudo plane in the sound source simplified position; and (E) converting the sound source information including the three-dimensional sound source position information, the sound source height information, and the sound source intensity information into a visual signal and displaying the signal in a visualization module; The four sound source sensing sensors have a plurality of sensor combinations consisting of three sensors forming a right angle, The step (C) comprises the following steps: Acquire the sound source data and the size of the sound source signal recognized by the sound source sensing sensor, and calculate the sound source intensity level corresponding to the acquired size of the sound source signal. The step (D) comprises the following steps: generating a plurality of pseudo planes including z coordinates as height information of a sound source recognized by each of the sound source sensing sensors using height data of the sound source included in the sound source information sensed by each of the sound source sensing sensors; Screening the generated pseudo plane to grasp the sound source height, and correcting the sound source position, sound source distance, sound source intensity and sound source height; In the step (D), the two-dimensional simplified occurrence position, direction and separation distance of the sound source are calculated by triangulation based on the time difference of the sound source obtained from three sensors selected from any one of the plurality of sensor combinations. The height information of the sound source is simply calculated by comparing a distance calculated from a time difference of the sound source obtained from two sensors disposed at different heights on a diagonal line in the selected sensor combination with a diagonal distance between the two sensors, When correcting the sound source height, the sound source height information is corrected using the average value, median value, mode or analog of the z coordinate as the sound source height information in the pseudo plane selected from the sound source height search unit. The step (E) comprises the following steps: According to a preset visualization template, converting the identified direction of the sound source and the distance between the user and the sound source into a visualization object corresponding to the preset visualization template; matching the color or light intensity corresponding to the intensity of the identified sound source, and displaying the visualized object by changing the color or light brightness according to the intensity of the sound source; The corrected height information of the sound source is displayed together with the visualized object after the sound source is changed.

4. The sound source visualization method according to claim 3, characterized in that: The step (C) further comprises the following steps: The sound source sensing signals obtained by each of the sound source sensing sensors and the sound source signal transmitted from the pre-processing module are analyzed to calculate the occurrence position and the movement position of the sound source.

5. The sound source visualization method according to claim 3, characterized in that: The step (E) further comprises the following steps: The system provided with the sound source visualization program outputs the corrected height information of the sound source together with the visualized object after the sound source is changed.

Images