System for providing stereophonic sound in physical space using selective tracking of moving objects in images

Abstract

According to the present invention, a system for providing stereo sound in a physical space using selective tracking of a moving object in a video, the system providing stereo sound in a physical space in which a plurality of speaker units are arranged by obtaining video image information applied to a display unit, the system comprises a video source acquisition unit for acquiring video image information applied to a display unit; an image processing unit that selectively extracts a moving object in the image from the video image information, and tracks and recognizes a motion trace of the moving object; and a sound source processing unit that extracts a sound source with respect to the image from the video image information, and individually extracts a moving sound source originating from the moving object, and selectively transmits the moving sound source to the plurality of speaker units.

Description

Technical Field

[0001] The present invention relates to a system for providing stereo sound to users present in a physical space, and more particularly, to a system for dividing a moving object and a non-moving object in an image, separately extracting a signal due to the moving object, and separately transmitting the sound to a physically divided single speaker, thereby providing separate stereo sound according to the movement of the moving object. Background Art

[0002] According to the forecast of ReportLinker, a global market research company, in April 2023, the global audio equipment market size is expected to reach 7.74 billion US dollars (about 10.2438 trillion won) this year. Considering that the audio market size in 2022 was estimated to be 7.25 billion US dollars (about 9.6 trillion won) last year, it has increased by about 6.76%.

[0003] According to the report link, the speaker market, which includes home audio equipment such as subwoofers, wall-mounted speakers, and TV sound bars, will continue to grow. The CAGR is expected to reach 6.7% by 2027.

[0004] In particular, the "home audio" market has shown significant growth. According to analysis, as more people enjoy TV content on TV at home during the new coronavirus pandemic, more people are paying attention to audio equipment, etc., and the report link states that "users are becoming more and more aware of and demanding sound quality," which means "willing to spend more money on speakers." As the number of ordinary consumers with a high level of audio knowledge increases, the sound bar market is also expanding.

[0005] On the other hand, the Digital Entertainment Group (DEG) of the United States also published a report that overall spending on "home entertainment" such as TVs and audio equipment has increased. Just as TV sales have increased significantly during the pandemic, the audio equipment market has also grown. The report link stated that "as the popularity of home entertainment audio equipment increases, manufacturers such as Samsung, LG, Sony, and Bosch have released products and platforms."

[0006] This is why TV giants Samsung Electronics and LG Electronics are also gradually upgrading their sound bar products. That is, they provide a rich audio experience by connecting to TVs. In addition, they can also increase the sales of sound bars, which can be said to be "killing two birds with one stone."

[0007] Recently, the fiercely competitive Samsung and LG have launched new organic light-emitting diode (OLED) TV products one after another, and there is also a trend of launching new sound bar products. The high-level technology that the two companies are proud of is also attracting attention. (Refer to Edaily report on March 11, 2023)

[0008] As a conventional patent document, there is "Stereo sound providing method and apparatus by generating sound signal in virtual space (Korean Patent Publication No. 10-1919508, Patent Document 1)".

[0009] The invention of patent document 1 provides a stereo providing device for generating a sound signal in a virtual space, which includes: a frequency energy analyzer for analyzing the frequency energy of a sound source; a sound source category determiner for determining the sound source category by comparing the frequency energy of each frequency segment analyzed by the frequency energy analyzer; a virtual sound source configurator for setting a virtual sound source configuration in a virtual space corresponding to instrument configuration information of the sound source category; a virtual sound source generator for generating a virtual sound source by upmixing and downmixing; and an equalizer for equalizing each virtual sound source according to the positional relationship between the virtual sound source set in the virtual space and the listener.

[0010] There is a "stereo adjustment method and apparatus (Korean Patent Publication No. 10-1682105, Patent Document 2)".

[0011] The embodiment of patent document 2 provides a stereo adjustment method and device, which outputs a graphical user interface screen in which one or more corresponding objects are configured in a virtual space. When receiving input information for adjusting the size of the virtual space, at least one of the disappearance time, vibration amplitude and parallax of each sound signal in the multi-channel sound signal is adjusted based on the input information and output, thereby providing a surround effect with a sense of stereo without having a separate additional speaker for realizing three-dimensional sound, and at the same time being able to provide the audience with a sense of presence as if they were listening to music in various actual performance venues.

[0012] There is a "method and device for providing multi-viewpoint images and three-dimensional stereo sound (Korean authorization number 10-1534295, patent document 3)".

[0013] The embodiment of patent document 3 provides a method and device for providing multi-viewpoint images and three-dimensional stereo sound, which configures a stereoscopic image and a multi-channel image of one or more corresponding objects in a virtual space and displays them in a preset channel display area, and outputs a multi-channel sound signal corresponding to the multi-channel image. When change information of the corresponding object is recognized, a reconstructed stereoscopic image is generated according to the change information, and the output of each sound signal in the multi-channel sound signal is adjusted and output, thereby displaying a three-dimensional spatial sense, and providing a surround effect with a sense of stereo and presence without a separate additional speaker for realizing three-dimensional sound.

[0014] Patent Document 1: Korean Grant No. 10-1919508

[0015] Patent Document 2: Korean Grant No. 10-1682105

[0016] Patent Document 3: Korean Grant No. 10-1534295 Summary of the invention

[0017] Technical issues to be solved

[0018] A system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention is proposed to solve the above-mentioned problems, and the following problems are proposed to be solved.

[0019] First, stereo sound is provided to the user based on image-based information.

[0020] Second, specific objects in the image, mainly moving objects, and the sounds from the moving objects are designed to be stereo.

[0021] Third, based on the moving parts in the image, a stereo effect with the moving objects as the main feature is provided to the user.

[0022] The problems solved by the present invention are not limited to those mentioned above, and those skilled in the art can clearly understand other problems that are not mentioned from the following description.

[0023] Methods used to solve problems

[0024] In order to solve the above-mentioned problem, a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention has the following means for solving the problem.

[0025] With regard to the system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention, the system provides stereo sound in a physical space configured with a plurality of speaker units by obtaining video image information applied to a display unit, and is characterized in that it comprises: a video source acquisition unit, which acquires the video image information applied to the display unit; an image processing unit, which selectively extracts a moving object in the image from the video image information, and tracks and identifies a motion trace of the moving object; and a sound source processing unit, which extracts a sound source relative to the image from the video image information, and separately extracts a motion sound source originating from the moving object, and selectively transmits the motion sound source to the plurality of speaker units.

[0026] As for the system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention, it is characterized in that the image processing unit includes: an image extraction unit, which extracts the image from the video image information and separates it from the sound source; a space recognition unit, which separately extracts a background space image from the image separately extracted by the image extraction unit; and a moving object recognition unit, which separately extracts and recognizes the moving object from the image separately extracted by the image extraction unit.

[0027] With regard to the system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention, it is characterized in that the space identification unit can divide the background space image into a plurality of virtual grids, and identify the spatial movement of the moving object through the background space image divided by the plurality of virtual grids.

[0028] With regard to the system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention, it is characterized in that the image processing unit may also include: a motion tracking identification unit, which three-dimensionally identifies and tracks trajectory information of the moving object in the background space image based on the areas of the multiple virtual grids occupied by the moving object in the background space image and the coordinate information of the multiple virtual grids transferred by the moving object in the background space image.

[0029] With regard to the system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention, it is characterized in that the sound source processing unit may include: a motion sound source separation unit, which extracts the sound source relative to the image from the video image information, and separately extracts the motion sound source originating from the moving object.

[0030] With regard to the system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention, it is characterized in that the sound source processing unit may also include: a multi-channel allocation unit, which divides the motion sound source of the moving object extracted separately by the motion sound source separation unit into multiple spatial signals, and selectively allocates a separate channel to each of the multiple spatial signals.

[0031] As for the system for providing stereo sound in a physical space by selectively tracking moving objects in an image according to the present invention, it is characterized in that the sound source processing unit may also include: a physical space synchronization unit, which configures the multiple speaker units in the physical space of the room where the user is located according to preset space designation information, and selectively distributes the spatial signal to each of the multiple speaker units.

[0032] With regard to the system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention, it is characterized in that the sound source processing unit may also include: a moving object sound source tracking unit, which three-dimensionally identifies the trajectory information identified by the motion tracking identification unit, selectively sends the multiple spatial signals selectively assigned to the separate channels to the multiple speaker units configured in the physical space, and provides stereo sound to the user through the multiple speaker units in the physical space based on the trajectory information of the moving object.

[0033] With regard to the system for providing stereo sound in a physical space by selectively tracking moving objects in an image according to the present invention, it is characterized in that the physical space synchronization unit can virtually infer a preset positioning zone (predetermined positioned zone) in the space where the user is located, and configure the multiple speaker units in front, back, left or right with the preset positioning zone as the center, and set the matter of configuring the multiple speaker units in front, back, left or right with the preset positioning zone as the center as the preset space designation information.

[0034] The system for providing stereo sound in a physical space by selectively tracking moving objects in an image according to the present invention is characterized in that the multiple virtual grids form multiple grid-type virtual grids up, down, left, and right of the video image information applied to the display unit, and the cells of the grid-type virtual grids can be selectively expanded or reduced.

[0035] Effects of the Invention

[0036] According to the system for providing stereo sound in a physical space by selectively tracking a moving object in an image of the present invention having the above-described structure, the following effects are provided.

[0037] First, a stereo effect can be presented with the center of the element which is considered to be the most important element of the image where the user is most concentrated.

[0038] Secondly, a video codec is used to pre-divide and obtain the moving part and the non-moving part of the video, and the sound mainly in the moving part is extracted separately, and a stereo effect mainly in the moving part is selectively provided.

[0039] Third, by tracking the moving image, i.e., trail tracking, separate signals are provided to speakers that are physically surrounded by the user, thereby producing a stereo effect.

[0040] The effects of the present invention are not limited to those mentioned above, and those skilled in the art can clearly understand other effects not mentioned from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 The present invention is a conceptual diagram showing an example form in which a speaker unit is arranged around a user and a display unit in the physical space where the user is located, in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention.

[0042] Figure 2 This is a conceptual diagram showing how images and sound sources are extracted from video image information in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention.

[0043] Figure 3 A conceptual diagram showing selectively extracting a moving object in an image in a system for providing stereo sound in a physical space by selectively tracking the moving object in the image according to an embodiment of the present invention.

[0044] Figure 4 The present invention shows a system for providing stereo sound in a physical space by selectively tracking a moving object in an image, and separately extracting a motion signal from the moving object and a reflection signal caused by a reflection object.

[0045] Figure 5 The present invention is a conceptual diagram showing how to obtain a ghost object in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention.

[0046] Figure 6 is a block diagram showing each structure of a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention.

[0047] Figure 7 The block diagram of an image processing unit and its lower-level structures in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention.

[0048] Figure 8 The present invention is a block diagram showing a sound source processing unit and its lower-level structures in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention.

[0049] Fig. 9 This is a conceptual diagram for explaining a function of a separation screen portion in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention can be modified in various ways and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific implementation form, but should be understood to include all changes, equivalents and substitutes within the technical concept and technical scope of the present invention.

[0051] Figure 1 The present invention is a conceptual diagram showing an example form in which a speaker unit is arranged around a user and a display unit in the physical space where the user is located, in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention. Figure 2 This is a conceptual diagram showing how images and sound sources are extracted from video image information in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention. Figure 3 A conceptual diagram showing selectively extracting a moving object in an image in a system for providing stereo sound in a physical space by selectively tracking the moving object in the image according to an embodiment of the present invention. Figure 4 The present invention shows a system for providing stereo sound in a physical space by selectively tracking a moving object in an image, and separately extracting a motion signal from the moving object and a reflection signal caused by a reflection object. Figure 5 The present invention is a conceptual diagram showing how to obtain a ghost object in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention. Figure 6 is a block diagram showing each structure of a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention. Figure 7 The present invention is a block diagram of an image processing unit and its lower-level structures in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention. Figure 8 The present invention is a block diagram showing a sound source processing unit and its lower-level structures in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention. Fig. 9 This is a conceptual diagram for explaining a function of a separation screen portion in a system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to an embodiment of the present invention.

[0052] The system for providing stereophonic sound in a physical space by selectively tracking a moving object in an image according to the present invention is equivalent to the following system: Figure 1As shown, video image information applied to a display unit 20 such as a television, a projector, or other projectors is obtained, and in a physical space (refer to Figure 1 Provides stereo sound in a room.

[0053] The system according to the present invention obtains access rights to the video image information applied to the display unit 20 by means of a chip embedded in the display unit 20, installed software, or a USB or HDMI connection provided by the applicant of this application, and provides the stereo effect described later by obtaining image information.

[0054] First, with regard to a system for providing stereophonic sound in a physical space by selectively tracking a moving object in an image according to the present invention, Figure 6 As shown, it may include a video source acquisition unit 100, an image processing unit 200 and an audio source processing unit 300.

[0055] Next, as described above, the video source acquisition unit 100 is configured to acquire video image information by acquiring video image information applied to the display unit 20 while the display unit 20 is granted access rights.

[0056] Preferably, the video source acquisition unit 100 acquires video image information before the display unit 20 displays the picture, and has a technical function of providing stereo sound to the plurality of speaker units 10 by separating the sound source and analyzing the image through the acquired video image information.

[0057] The video image information acquired by the video source acquisition unit 100 is processed into source information of the image processing unit 200 and the sound source processing unit 300 described later.

[0058] The image processing unit 200 is structured as follows: as described above, it selectively extracts a moving object in an image from video image information, and tracks and recognizes a moving trace of the moving object, ie, a track and trajectory of the moving object.

[0059] The image processing unit 200 divides and identifies the background space image corresponding to the background and the moving object not corresponding to the background from the video image information, and obtains a typical pattern of the part with a large change between frames, such as a person, a car, a flying object, an animal or other object that moves in the image and attracts the user's attention, based on the codec compression standard technology, which separates and compresses the parts with small changes between frames and the parts with large changes between frames in the image, and obtains the typical pattern of the part with a large change between frames, such as a person, a car, a flying object, an animal or other objects that move in the image and attract the user's attention. That is, the image processing unit 200 extracts the moving object from the background space image in a way that is combined with the codec method, and further identifies and obtains it through the size change and position change of the moving object described later.

[0060] The sound source processing unit 300 is configured to extract the sound source corresponding to the image from the video image information, and separately extract the motion sound source originating from the moving object, and selectively transmit the motion sound source to the plurality of speaker units 10 .

[0061] As described above, the motion sound source corresponds to a sound source linked to a moving object, and the motion sound source of the moving object is extracted separately by utilizing the characteristic that the motion sound source is synchronized with the moving object appearing, expanding, and approaching from the image.

[0062] The motion sound source originating from the moving object has different characteristics from other sound sources flowing out of the background. Due to the frequency characteristics, the background sound source has characteristics such as white noise, while the motion sound source has a characteristic of being concentrated in the mid-bass or a bandwidth of a specific wavelength band in the audible frequency band depending on the type of the moving object. The motion sound source of the moving object is different from the background sound source or other sound sources, and can also be extracted through a separate filter by utilizing the characteristics of modularization through a separate channel.

[0063] As for the image processing unit 200 of the system for providing stereo sound in a physical space using selective tracking of a moving object in an image according to the present invention, Figure 7 As shown, it may include an image extraction unit 210 , a space recognition unit 220 and a moving object recognition unit 230 .

[0064] The image extraction unit 210 is a technical structure that extracts images from video image information separately from sound sources as described above. Since the function of extracting only images from video image information separately from sound sources may be a very obvious part, a separate description is omitted.

[0065] Afterwards, the space recognition unit 220 extracts the background space image separately from the image extracted by the image extraction unit 210 , and at the same time, the moving object recognition unit 230 extracts and recognizes the moving object separately from the image extracted by the image extraction unit 210 .

[0066] like Figure 2 and Figure 3 As shown, the moving object recognition unit 230 and the space recognition unit 220 divide the background space image corresponding to the background in the video image information and the moving object not corresponding to the background in the extracted image, and extract and recognize them separately. As described above, based on the video compression codec standard technology, the parts with smaller changes between frames and the parts with larger changes in the image are separated and compressed and transmitted, and the parts with larger changes between frames are mainly used and based on the typical pattern of the parts with larger changes, the moving objects are extracted and recognized separately. The moving object can be, for example, a person, a car, a flying object, an animal, or moves in other images, so as to obtain an object that attracts the user's attention. In addition, as far as the moving object recognition unit 230 and the space recognition unit 220 are concerned, the image processing unit 200 extracts the moving object from the background space image together with the codec method, and further recognizes and obtains it through the size change of the moving object and the position change of the moving object described later.

[0067] like Figure 3 As shown, the space recognition unit 220 divides the background space image into a plurality of virtual grids, and recognizes the spatial motion of the moving object through the background space image divided by the plurality of virtual grids.

[0068] Here, the plurality of virtual grids is characterized in that a plurality of lattice-type virtual grids are formed in an orthogonal manner in the vertical direction and the horizontal direction of the video information applied to the display unit 20, and the cells of the lattice-type virtual grid can be selectively expanded or reduced.

[0069] The density of multiple grid-type virtual grids, for example, the configuration of horizontal or vertical virtual orthogonal type virtual lines can be processed differently according to the fps (frames per second) degree between frames or according to the resolution of pixels, and the density of the grid can be set differently according to the time processing speed processed by the present invention.

[0070] In addition, of course, it can also be set differently according to the speed of the moving object in the image. Figure 3 During the processing of 60fps frames, the virtual grid for processing can be expanded according to the speed at which the moving object moves in the background space image. When the speed is fast, the virtual grid for processing can be expanded. Conversely, when the moving object moves slowly in the background space image, the virtual grid for processing can be reduced.

[0071] The system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to the present invention may further include a motion trace recognition unit 240 .

[0072] like Figure 3As shown, the motion tracking identification unit 240 has the following structure: based on the areas of multiple virtual grids occupied by the moving object in the background space image and the coordinate information of multiple virtual grids transferred by the moving object in the background space image, the trajectory information of the moving object in the background space image is three-dimensionally identified and tracked.

[0073] The motion tracer recognition unit 240 can capture the motion object Figure 3 The movement on the orthogonal grids from which grid on the virtual grid to which grid on the background space image is calculated according to the type of the moving object, the specific movement mode and the movement on the moving grid to infer the future movement traces, i.e., tracks and trajectories.

[0074] like Figure 3 As shown, the motion tracking identification unit 240 not only identifies the spatial movement of the moving object based on the motion capture and inference of its trace trajectory, but also identifies whether the moving object is approaching or moving away and disappearing based on the number of grids occupied by the moving object in the process of moving toward moving object #1-1, moving object #1-2, moving object #1-3, and moving object #1-4.

[0075] In terms of a system for providing stereophonic sound in a physical space using selective tracking of a moving object in an image according to the present invention, Figure 8 As shown, the sound source processing unit 300 may further include a motion sound source separation unit 310 .

[0076] The motion sound source separation unit 310 is configured to extract the sound source corresponding to the image from the video image information as described above, and to separately extract the motion sound source originating from the moving object.

[0077] As described above, the motion sound source corresponds to a sound source linked to a moving object, and the motion sound source of the moving object is extracted separately by utilizing the characteristic that the motion sound source is synchronized with the moving object appearing, expanding, and approaching from the image.

[0078] The motion sound source originating from the moving object has different characteristics from other sound sources flowing out of the background. Due to the frequency characteristics, the background sound source has characteristics such as white noise, while the motion sound source has a characteristic of being concentrated in the mid-bass or a bandwidth of a specific wavelength band in the audible frequency band depending on the type of the moving object. The motion sound source of the moving object is different from the background sound source or other sound sources, and can also be extracted through a separate filter by utilizing the characteristics of modularization through a separate channel.

[0079] like Figure 8As shown, the sound source processing unit 300 may also include a multi-channel allocation unit 320, which has the following structure: the motion sound source of the motion object extracted separately by the motion sound source separation unit 310 is divided into multiple spatial signals, and a separate channel is selectively allocated to each of the multiple spatial signals.

[0080] In addition, the sound source processing unit 300 may also include a physical space synchronization unit 330, which has the following structure: each of a plurality of speaker units is configured according to preset space designation information in the physical space of the room where the user is located, and each of the spatial signals is selectively distributed to each of the plurality of speaker units.

[0081] In addition, the sound source processing unit 300 may further include a moving object sound source tracking unit 340 .

[0082] The moving object sound source tracking unit 340 three-dimensionally recognizes the trajectory information recognized by the motion trace recognition unit 240, such as Figure 1 As shown, a plurality of spatial signals to which separate channels are selectively allocated are selectively transmitted to a plurality of speaker units 10 arranged in a physical space, respectively, thereby providing stereo sound to a user through the plurality of speaker units 10 in the physical space (room) according to trajectory information of a moving object.

[0083] like Figure 3 As shown, a moving object (refer to a helicopter) is observed on multiple virtual grids of a background space image, and the moving object moves from the upper left end to the lower center of the right side. In this process, the number of grids occupied by the moving object increases, so it can be interpreted as the moving object approaching the lower center of the right side from the upper left end, which is identified by the motion tracking identification unit 240.

[0084] The motion sound source of the moving object originates from the moving object, and the motion sound source is divided into a plurality of spatial signals and divided into a plurality of channels, and selectively allocated to separate channels, which is managed by the multi-channel allocation unit 320. Figure 1As shown, a plurality of speakers 10 may be configured (for example, based on the user, a 10FL speaker is configured on the left side in front of the user, a 10FR speaker may be configured on the right side in front, based on the user, a 10RL speaker is configured on the left side behind the user, and based on the user, a 10RR speaker is configured on the right side behind the user). This spatial configuration is equivalent to an example of preset spatial designation information, and the physical space synchronization unit 330 manages this information. After that, the physical space synchronization unit 330 distributes the motion sound source divided into a plurality of spatial signals to the plurality of speakers. As far as the motion object sound source tracking unit 340 is concerned, the 10FL speaker on the left side in front of the user manages the sound of the helicopter in the image when it is far away on the upper left side, and the 10RR speaker manages the lower right side of the user. The frequency band and amplitude of the sound sent to each speaker separately as a spatial signal are set to be different (for example, the sound of 10FL is small and the frequency is in the low frequency band, while the sound of 10RR has a large amplitude but a relatively high frequency band), and the sound transfer from 10FL to 10RR is set to perform continuous sound amplification and frequency band transfer.

[0085] To this end, with respect to the physical space synchronization unit 330, Figure 1 As shown, a preset positioned zone in the space where the user is located is virtually inferred, and multiple speaker units are configured in front, back, left or right with the preset positioned zone as the center, and the matter of configuring multiple speaker units in front, back, left or right with the preset positioned zone as the center is set as preset space designation information.

[0086] As for the image processing unit 200, Figure 7 As shown, a reflective object recognition unit 250 may also be included.

[0087] like Figure 4 As shown, in addition to the moving object, in addition to the motion sound source appearing in the moving object, when the same mode as the motion sound source, with a time difference with the motion sound source, with the same frequency band and the sound source is obtained separately, as shown in Figure 4 As shown, the presence of a reflective object in the video image information can be detected. The reflective object identification unit 250 infers that the motion tracer identification unit 240 is located on the predicted path of the moving object, and finally calculates the reflective object based on the patterns of other shapes such as trees, buildings, exterior walls, cars, etc.

[0088] Corresponding to the reflection object identification unit 250 , the sound source processing unit 300 may further include a reflection object sound source tracking unit 350 .

[0089] like Figure 4As shown, by estimating the position of the reflecting object, the relative position of the moving object and the motion trace of the moving object, the movement of the moving sound source and the sound source reflected and popped out from the reflecting object are obtained. As described above, after the multi-channel allocation unit 320 diverts the reflected and popped-out sound source, namely the reflected object sound source, into multiple spatial signals, and selectively allocates a separate channel to each of the multiple spatial signals, the physical space synchronization unit 330 sends these reflected object sound sources to each speaker unit of the multiple speaker units 10 separately, thereby also generating separate stereo for the reflected sound source.

[0090] As for the separation screen part 360, Fig. 9 As shown, the file of the reflection target sound source is formed in a bit array of 1 and 0 in the order of columns and rows.

[0091] As for the separation screen portion 360, a preset perforated grid is set and applied to the matrix as described above, and the reflection object sound source equivalent to the preset perforated grid is separated and extracted from the original information and stored separately, and the corresponding degree of damage to the reflection object sound source as the original information is obtained.

[0092] For rows and columns having information equivalent to a preset punched square omitted, the upper separation portion and the lower separation portion are respectively applied.

[0093] As for the upper separation part, each column descends independently in the direction downward from the upper end, and each column descending downward descends independently.

[0094] On the contrary, as for the lower separation portion, each column rises independently in the direction upward from the lower end, and each column rising upward rises independently.

[0095] The lower separation portion does not infringe upon the area where the upper separation portion descends downward, which means that the upper separation portion and the lower separation portion do not overlap.

[0096] The upper separation part and the lower separation part split the rows and columns with information omitted equivalent to the preset punched squares into upper and lower parts respectively, and then the reflection object sound source as the original information is separated up and down by the upper separation part and the lower separation part, so the inherent information of the reflection object sound source file cannot be known due to a third party or external access.

[0097] As for the image processing unit 200, Figure 7 As shown, a ghost object calculation unit 241 may also be included.

[0098] like Figure 5 As shown, the space recognition unit 220 two-dimensionally expands the plurality of virtual grids and applies them to the video image information.

[0099] Afterwards, the motion tracer identification unit 310 is structured as follows: when there is a ghost signal, i.e., a motion sound source generated before the moving object appears in the image, it is located on the extension line of the motion tracer, and the position of the ghost object and its motion tracer are additionally calculated outside the video image information.

[0100] Corresponding to the ghost object calculation unit 241 , the sound source processing unit 300 may further include a ghost object sound source tracking unit 341 .

[0101] On the extension line of the moving object sound source tracking unit 340, if multiple spatial signals of the moving sound source are split and allocated to multiple channels, the ghost object sound source tracking unit 341 also splits and transmits the spatial signals allocated to the separate channels to the multiple speaker units 10. Figure 5 In the example, based on the user, starting from the speaker at the rear right, motion sound sources can be sent to the front left and front right.

[0102] The scope of the claims of the present invention is determined by the matters recorded in the claims. The brackets used in the claims are used to clarify the constituent elements, rather than to record selective limitations. The records in the brackets should also be interpreted as essential constituent elements.

Claims

1. A system for providing stereo sound in a physical space by selectively tracking a moving object in an image, wherein the system provides stereo sound in a physical space provided with a plurality of speaker units by obtaining video image information applied to a display unit, characterized in that: include: a video source acquisition unit, for acquiring the video image information applied to the display unit, an image processing unit, selectively extracting a moving object in the image from the video image information, and tracking and identifying a moving tracer of the moving object, and The sound source processing unit extracts the sound source relative to the image from the video image information, and separately extracts the motion sound source originating from the moving object, and selectively transmits the motion sound source to the plurality of speaker units.

2. The system for providing stereo sound in a physical space by selectively tracking a moving object in an image according to claim 1, characterized in that: The image processing unit comprises: An image extraction unit extracts the image from the video image information and separates it from the sound source, a space recognition unit that extracts a background space image separately from the image extracted by the image extraction unit, and The moving object recognition unit extracts and recognizes the moving object separately from the image separately extracted by the image extraction unit.

3. The system for providing stereophonic sound in a physical space by selectively tracking a moving object in an image according to claim 2, wherein: The space recognition unit divides the background space image into a plurality of virtual grids, and recognizes the spatial motion of the moving object through the background space image divided by the plurality of virtual grids.

4. The system for providing stereophonic sound in a physical space by selectively tracking a moving object in an image according to claim 3, wherein: The image processing unit also includes: The motion tracking recognition unit three-dimensionally recognizes and tracks the trajectory information of the moving object in the background space image based on the areas of the multiple virtual grids occupied by the moving object in the background space image and the coordinate information of the multiple virtual grids transferred by the moving object in the background space image.

5. The system for providing stereophonic sound in a physical space by selectively tracking a moving object in an image according to claim 4, wherein: The sound source processing unit comprises: The motion sound source separation unit extracts the sound source corresponding to the image from the video image information, and separately extracts the motion sound source originating from the moving object.

6. The system for providing stereophonic sound in physical space by selectively tracking moving objects in images according to claim 5, characterized in that: The sound source processing unit also includes: The multi-channel allocating unit divides the motion sound source of the moving object extracted independently by the motion sound source separating unit into a plurality of spatial signals, and selectively allocates an independent channel to each of the plurality of spatial signals.

7. The system for providing stereophonic sound in a physical space by selectively tracking a moving object in an image according to claim 6, wherein: The sound source processing unit also includes: The physical space synchronization unit configures a plurality of the speaker units in the physical space of the space where the user is located according to preset space designation information, and selectively distributes the space signal to each of the plurality of speaker units.

8. The system for providing stereophonic sound in a physical space by selectively tracking a moving object in an image according to claim 7, wherein: The sound source processing unit also includes: a moving object sound source tracking unit that three-dimensionally recognizes the trajectory information recognized by the motion tracer recognition unit, and selectively sends the plurality of spatial signals to which the individual channels are selectively allocated to the plurality of speaker units arranged in the physical space, respectively; According to the trajectory information of the moving object, stereo sound is provided to the user through the plurality of speaker units in the physical space.

9. The system for providing stereophonic sound in a physical space by selectively tracking a moving object in an image according to claim 7, wherein: The physical space synchronization unit virtually infers a preset positioning area in the space where the user is located, The plurality of speaker units are arranged in front, back, left or right with the preset positioning area as the center, and the arrangement of the plurality of speaker units in front, back, left or right with the preset positioning area as the center is set as the preset space designation information.

10. The system for providing stereo sound in physical space by selectively tracking moving objects in images according to claim 3, characterized in that: The plurality of virtual grids form a plurality of lattice-type virtual grids in the upper, lower, left, and right directions of the video image information applied to the display unit, and cells of the lattice-type virtual grid can be selectively expanded or reduced.

Images