System and method for automatic multimedia control based on UWB position data
The UWB-based zone control system addresses manual operation limitations by automatically adjusting multimedia effects based on user location and movement, enhancing multimedia productions with synchronized audiovisual experiences and reducing manpower inefficiencies.
Patent Information
- Application Number
- KR1020260115967
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2026-06-25
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2046-06-25
AI Technical Summary
Current multimedia productions in performing arts and exhibitions face challenges in linking audiovisual effects with user movement due to manual operation methods requiring extensive training, prone to errors, and difficulty in real-time control for multiple users, leading to inefficiencies and disconnects.
A zone-based multimedia automatic control system using UWB location data to track users, dividing indoor spaces into zones, and automatically controlling multimedia effects based on user location, movement, and number of users, eliminating the need for manual intervention.
Enables reliable, real-time, and efficient multimedia production by tracking user locations across the entire indoor space, allowing diverse and three-dimensional effects, reducing manpower inefficiencies and communication issues, and ensuring synchronized audiovisual experiences.
Smart Images

Figure 112026077208703-PAT00010_ABST
Abstract
Description
Technology Field
[0001] The present disclosure relates to a zone-based multimedia automatic control system and method based on UWB location data. Background Technology
[0003] In the fields of contemporary performing arts and exhibitions, as multimedia productions incorporating digital technology have become commonplace, active attempts are being made to combine various elements such as sound, video, and lighting to provide audiences with synesthetic experiences and expand the scope of expression. In such multimedia productions, linking audiovisual effects with the position of users moving within the stage or space is considered a key factor in enhancing immersion and interactivity, and there is a growing interest in spatial productions where the user's movement becomes an integral part of the performance.
[0004] However, such location-linked staging currently relies on a method where the director familiarizes themselves with the user's movement path in advance and manually controls sound, lighting, and other elements based on their own senses. This manual operation method has limitations, such as requiring a large workforce and extensive training, being prone to errors during communication between operators, and easily causing a sense of disconnect between the user's actual location and the staging effects. Furthermore, since it is practically difficult to individually control effects corresponding to each user's location in real-time when multiple users are present simultaneously, there is a growing demand for technology that can reliably track user locations and automate linked multimedia staging without human intervention.
[0005] The information described above is intended to enhance understanding of the background of the present invention and may include information that does not constitute prior art. The problem to be solved
[0007] To solve the above-mentioned problems, the present invention provides a zone-based multimedia automatic control system and method based on UWB location data. means of solving the problem
[0009] The present disclosure may be implemented in various ways, including a method, an apparatus (system), or a computer program stored on a readable storage medium.
[0010] A multimedia automatic control system based on UWB location data according to one embodiment of the present disclosure for solving technical problems may include: a receiving unit configured to receive location data in the form of a string representing the location of a user, which is output from a plurality of anchors installed in an indoor space and a tag attached to at least one user; a generating unit configured to generate refined location data by extracting a tag identifier and two-dimensional coordinates through regular expression-based parsing from the location data; a determining unit configured to determine the zone where the user is located by applying the refined location data to a zone map that divides the indoor space into a plurality of zones; a selection unit configured to select a multimedia effect corresponding to the determined zone from among multimedia effects for a plurality of multimedia components, including an audio device and a video device, which are set for each of the plurality of zones; a parameter calculation unit configured to calculate control parameters of the selected multimedia effect based on the refined location data; and a control unit configured to automatically control a plurality of multimedia components in real time according to the determined zone by converting the control parameters into a communication protocol corresponding to each multimedia component and transmitting them.
[0011] According to one embodiment of the present disclosure, a receiving unit and a generating unit are provided in a first computing device, and a determination unit, a selection unit, a parameter calculation unit and a control unit are provided in a second computing device, and the first computing device receives location data via USB serial communication from a listener that transmits location data output from a tag, and can transmit the refined location data wirelessly to the second computing device via UDP (User Datagram Protocol) communication.
[0012] According to one embodiment of the present disclosure, tags are provided in a plurality of numbers, each attached to a plurality of users, and a determination unit is further configured to determine the area where a user is located for each tag and to calculate the number of users in each of the plurality of areas, and a parameter calculation unit can change at least one of the volume of an audio device and the display form of an object of a video device corresponding to the area according to the number of users in each of the plurality of areas.
[0013] According to one embodiment of the present disclosure, a plurality of zones includes a first zone corresponding to a stage area of an indoor space and a second zone located outside the stage area and corresponding to a user's entry or exit path, and a determination unit is further configured to determine an entry state in which a user moves from the second zone to the first zone and an exit state in which a user moves from the first zone to the second zone based on changes over time in refined position data, and a selection unit can select a preset first multimedia effect when the user is in an entry state and select a preset second multimedia effect different from the first multimedia effect when the user is in an exit state.
[0014] According to one embodiment of the present disclosure, the parameter calculation unit can calculate the left and right panning values and the volume attenuation amount for the sound output by the acoustic device based on the following [Equation 1] and [Equation 2].
[0015] [Mathematical Formula 1]
[0016]
[0017]
[0018] [Mathematical Formula 2]
[0019]
[0020] (Here, P_voice is the left / right panning value of the user's original voice, P_proc is the left / right panning value of the modified processed voice, x is the x-coordinate of the refined position data, W is the width of the room space, G_dB is the volume attenuation amount, y is the y-coordinate of the refined position data, D is the depth of the room space, and α is the attenuation coefficient) Effects of the invention
[0022] According to some embodiments of the present disclosure, by utilizing UWB-based location tracking, the location of a user can be reliably tracked over an entire indoor space including non-visible areas without constraints on field of view or lighting conditions, unlike camera-based methods.
[0023] According to some embodiments of the present disclosure, instead of directly corresponding the user's location data to an effect, by selecting a multimedia effect corresponding to the area where the user is located, diverse and three-dimensional multimedia productions can be realized depending on the location.
[0024] According to some embodiments of the present disclosure, by distributing a first computing device responsible for receiving and refining location data and a second computing device responsible for multimedia control and wirelessly linking them, the limitations based on the physical distance between the location where location tracking is performed and the location where multimedia is controlled can be eliminated.
[0025] According to some embodiments of the present disclosure, by individually recognizing multiple users, grouping them by zone, and automatically adjusting multimedia effects according to the number of users in each zone, dynamic production responding to the location distribution of users can be implemented without separate manual operation.
[0026] According to some embodiments of the present disclosure, by setting the outer area of the stage area as a zone and identifying entry or exit based on the direction of movement of the user, multimedia effects linked to the user's entry and exit from the stage can be automatically applied.
[0027] According to some embodiments of the present disclosure, by calculating the left-right panning value and volume attenuation amount of sound based on the user's coordinates, the sense of separation, localization, and distance of the sound can be expressed in real time according to the user's position and movement. In addition, by localizing the original sound to the side where the user is located and localizing the processed sound to the opposite side, the user's left-right movement can be expressed as a change in the gap between the sound image of the original sound and the sound image of the processed sound, and the entire width of the indoor space can be utilized for sound production.
[0028] According to some embodiments of the present disclosure, by automating multimedia production linked to a user's location without relying on manual human operation, inefficiency in manpower management and communication problems between operators can be resolved, and stable and efficient production can be achieved at a low cost.
[0029] According to some embodiments of the present disclosure, discrete control that switches multimedia effects when a user's location or derived indicator satisfies a specific condition, and control that continuously changes control parameters according to changes in the user's location, are performed together within a single system, thereby enabling the integrated implementation of condition-based transitions and continuous direction linked to location.
[0030] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art to which the present disclosure pertains (referred to as "person skilled in the art") from the description in the claims. Brief explanation of the drawing
[0032] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, wherein similar reference numerals indicate similar elements, but are not limited thereto. FIG. 1 is a diagram showing the overall process of a UWB-based multimedia automatic control system according to one embodiment of the present disclosure. FIG. 2 is a schematic diagram showing a configuration in which a user terminal, a tag, a multimedia automatic control system, and an external device are connected to enable communication in order to provide multimedia automatic control according to one embodiment of the present disclosure. FIG. 3 is a block diagram schematically showing the internal configuration of a user terminal, tag, and multimedia automatic control system according to one embodiment of the present disclosure. FIG. 4 is a block diagram showing the internal configuration of a multimedia automatic control system according to one embodiment of the present disclosure. FIG. 5 is a flowchart illustrating a method for automatically controlling multimedia according to one embodiment of the present disclosure. FIG. 6 is a diagram illustrating a multimedia control operation according to one embodiment of the present disclosure. FIG. 7 is a diagram illustrating a multimedia control operation based on the number of users per zone according to one embodiment of the present disclosure. FIG. 8 is a diagram illustrating a multimedia control operation based on zone location according to one embodiment of the present disclosure. Specific details for implementing the invention
[0033] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions regarding widely known functions or configurations will be omitted if there is a risk that the gist of the present disclosure may be unnecessarily obscured.
[0034] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Additionally, in the description of the following embodiments, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.
[0035] The advantages and features of the disclosed embodiments and the methods for achieving them will become clear by referring to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below but may be implemented in various different forms, and the embodiments provided are merely to make the present disclosure complete and to fully inform those skilled in the art of the scope of the invention.
[0036] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification have been selected to be as generally used as possible, taking into account their functions in this disclosure; however, these terms may vary depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Additionally, in specific cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.
[0037] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.
[0038] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. Components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts' provided therein, or further separated into additional components and 'modules' or 'parts'.
[0039] According to one embodiment of the present disclosure, the term "processor" should be broadly interpreted to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some contexts, the term "processor" may refer to application-specific integrated circuits (ASICs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), etc. The term "processor" may also refer to combinations of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term "memory" should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.
[0040] In addition, the terms "first," "second," etc., used in the following embodiments are used merely to distinguish one component from another, and the essence, order, or sequence of the said component is not limited by such terms.
[0041] Additionally, in the following embodiments, where it is stated that one component is 'connected', 'coupled', or 'joined' to another component, it should be understood that the component may be directly connected or joined to the other component, but that another component may also be 'connected', 'coupled', or 'joined' between each component.
[0042] Additionally, the terms 'comprises' and / or 'comprising' as used in the following embodiments do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.
[0043] In the present disclosure, the 'system' may include at least one of a server device and a cloud device, but is not limited thereto. For example, the system may be composed of one or more server devices. As another example, the system may be composed of one or more cloud devices. As yet another example, the system may be configured and operated with both a server device and a cloud device.
[0044] In the present disclosure, 'display' may refer to any display device associated with a computing device, for example, any display device capable of displaying any information / data controlled by or provided by the computing device.
[0045] In the present disclosure, 'each of a plurality of A' or 'each of a plurality of A' may refer to each of all components included in a plurality of A, or each of some components included in a plurality of A.
[0046] In addition, terms such as first, second, A, B, (a), (b), etc. used in the following embodiments are used merely to distinguish one component from another, and the essence, order, or sequence of the said component is not limited by such terms.
[0047] In the present disclosure, 'user' may refer to a performer with a tag attached or a performer with a wireless chip attached, etc.
[0048] The flowcharts and descriptions illustrated in the drawings of the present disclosure below are merely examples, and some embodiments may be implemented differently from the described embodiments. For example, one or more steps may be omitted, the order of each step may be changed, one or more steps may be performed in a superposition, or one or more steps may be performed repeatedly.
[0049] FIG. 1 is a diagram showing the overall process of a UWB-based multimedia automatic control system according to one embodiment of the present disclosure.
[0050] Referring to FIG. 1, a multimedia automatic control system (100) according to one embodiment of the present disclosure tracks the location and movement of a user (130) located within an indoor space in real time based on Ultra-wideband (UWB), and can automatically control multimedia including sound, video, etc. in response to the tracked location. In the present disclosure, the multimedia automatic control system (100) may refer to a system that automatically performs a series of processes ranging from receiving location data of the user (130) to controlling multimedia components.
[0051] In the present disclosure, the user (130) may refer to an object subject to location tracking. For example, the user (130) may be a performer performing on a stage (110), but is not limited thereto, and may include various objects subject to location tracking, such as a viewer moving within an exhibition space or an experiencer experiencing within a media art space. Additionally, in the present disclosure, the indoor space may be a performance venue including a stage (110) where a performance takes place, but is not limited thereto, and may include various indoor environments where multimedia is produced in response to the location of the user (130), such as an exhibition hall, a media art space, or a smart stage.
[0052] In one embodiment, a UWB method may be employed for location tracking in the present disclosure. Although a method relying on image information, such as a camera or infrared sensor, may be used to track the location of a user, such a method requires the tracking target to be within the device's field of view and may be significantly subject to environmental constraints, such as lighting conditions or obstruction of view by obstacles. On the other hand, since the UWB method according to the present disclosure does not rely on visual information, the location of the user (130) can be stably tracked over the entire indoor space, including areas where a field of view is not secured.
[0053] Referring to FIG. 1, a plurality of anchors (120) may be installed in an indoor space. The anchors (120) may function as reference points fixedly installed at known locations within the indoor space, for example, at each corner of the stage (110). Meanwhile, a tag (140) may be attached to a user (130). The tag (140) may be attached in a form that adheres closely to the user's (130) body, attached to the user (130) in the form of a wireless chip with built-in UWB functionality, or carried by the user (130) in the form of a smart device with built-in UWB functionality. The tag (140) transmits and receives wireless signals with the plurality of anchors (120) and can calculate its own location based on distance and time information with each anchor (120). The calculated location may be output as location data indicating the location of the user (130).
[0054] Here, location data output from the tag (140) can be collected through a listener (150). The listener (150) can function as a configuration that receives location data output from the tag (140) and transmits it to a multimedia automatic control system (100). In one embodiment, the anchor (120), the tag (140), and the listener (150) may each have their roles individually assigned to the same type of UWB module, and accordingly, a configuration for location tracking can be provided without having to provide separate different hardware.
[0055] Referring to FIG. 1, a multimedia automatic control system (100) can receive location data transmitted through a listener (150), process it, and perform a series of processes to control multimedia. In one embodiment, the multimedia automatic control system (100) may be implemented by including at least one computing device, and may be implemented by including a first computing device (160) and a second computing device (170) as shown in FIG. 1. However, this is exemplary, and the multimedia automatic control system (100) may be implemented as a single computing device or distributed among three or more computing devices. Specific embodiments in which the first computing device (160) and the second computing device (170) are separately provided and interact with each other will be described later.
[0056] First, the multimedia automatic control system (100) can receive location data that is output from a tag (140) and transmitted through a listener (150). In one embodiment, the location data may be received in the form of a string. The multimedia automatic control system (100) can extract coordinates representing the location of a user (130) from the received location data and generate refined location data in a form that can be used for subsequent processing.
[0057] Next, the multimedia automatic control system (100) can divide the indoor space into multiple zones and determine which of the multiple zones the user (130) is located in based on generated refined location data. The multimedia automatic control system (100) according to the present disclosure does not directly correspond the location of the user (130) to the multimedia effect in a one-to-one manner, but rather divides the indoor space into multiple zones and controls the multimedia effect in correspondence with the zone where the user (130) is located. Through this, more diverse and three-dimensional effects can be realized compared to a method in which the effect changes simply according to location coordinates.
[0058] The multimedia automatic control system (100) can determine a multimedia effect corresponding to the zone where the user (130) is determined to be located, while multimedia effects are set for each of the multiple zones. In the present disclosure, the multimedia effect may refer to an effect on a plurality of multimedia components including a sound device, a video device, a lighting device, a special effects device, etc., and may include, for example, sound output through a sound device, an object displayed through a video device, a color expressed through a lighting device, etc. Subsequently, the multimedia automatic control system (100) calculates control parameters so that the determined multimedia effect is implemented in each multimedia component, and converts the calculated control parameters into a communication protocol recognizable by each multimedia component and transmits them, thereby enabling the multimedia components to be automatically controlled in real time according to the location and movement of the user (130).
[0059] A series of processes as described above can be repeated in real time as the user (130) moves. That is, as the user (130) moves within the indoor space, their location is continuously tracked, and multimedia effects corresponding to the tracked location are updated in real time and automatically controlled. Through this, the multimedia automatic control system (100) according to the present disclosure can implement a spatially responsive interaction in which the location and movement of the user (130) and multimedia effects are linked in real time.
[0060] According to the multimedia automatic control system (100) of the present disclosure, multimedia production based on the location of the user (130) can be performed automatically without relying on manual operation by a person. Accordingly, inefficiencies in manpower management, communication problems between operators, and discrepancies between visual and auditory information that occurred in the conventional method where a sound operator or lighting operator manually operated by relying on senses after familiarizing themselves with the user's (130) movement path in advance can be resolved, and stable and efficient multimedia production can be realized with minimal cost and personnel.
[0061] FIG. 2 is a schematic diagram showing a configuration in which a user terminal, a tag, a multimedia automatic control system, and an external device are connected to enable communication in order to provide multimedia automatic control according to one embodiment of the present disclosure.
[0062] Referring to FIG. 2, a multimedia automatic control system (230) according to one embodiment of the present disclosure may include systems capable of performing a service (hereinafter referred to as 'multimedia automatic control service') that automatically controls multimedia such as sound and video based on the user's location. The multimedia automatic control system (230) may correspond to the multimedia automatic control system (100) of FIG. 1.
[0063] The user terminal (210_1), wireless chip (210_2), and tag (210_3) can be connected to a multimedia automatic control system (230) capable of providing multimedia automatic control services via a network (220). Additionally, the multimedia automatic control system (230) can be linked with an external device (240) via the network (220) to collect or transmit / receive necessary data. Here, the wireless chip (210_2) and tag (210_3) may include a subject that is attached to or carried by a user who is the target of location tracking within an indoor space, and outputs location data indicating the location of the user. In one embodiment, the tag (210_3) may be attached in a form that is in close contact with the user's body, and the wireless chip (210_2) may be implemented with a built-in UWB communication function and carried by the user or mounted on the user terminal (210_1). In the present disclosure, the wireless chip (210_2) and the tag (210_3) may be the main embodiment for outputting location data, and the user terminal (210_1) may be utilized as an auxiliary component for outputting location data, setting multimedia effects, or monitoring control results.
[0064] In one embodiment, the multimedia automatic control system (230) may be any computing device capable of storing, providing, and executing computer-executable programs (e.g., software including a location data parsing engine, a zone determination module, a multimedia effect control engine, etc.) and data related to multimedia automatic control services. For example, the multimedia automatic control system (230) may include one or more server devices and / or databases, or one or more distributed computing devices and / or distributed databases based on cloud computing services. The multimedia automatic control service provided by the multimedia automatic control system (230) may be provided to a user through a wireless chip (210_2), a tag (210_3), or a user terminal (210_1). For example, the wireless chip (210_2) or the tag (210_3) may output location data calculated by transmitting and receiving wireless signals to a plurality of anchors installed in an indoor space and transmit this to the multimedia automatic control system (230) via a network (220). The multimedia automatic control system (230) can determine the area where the user is located based on the received location data, determine the multimedia effect corresponding to the area, calculate the control parameters, and then transmit the control parameters to multimedia components such as a sound device and a video device so that the multimedia can be controlled in real time.
[0065] A multimedia automatic control method according to one embodiment of the present disclosure may be performed by the operation of a wireless chip (210_2), a tag (210_3), or a user terminal (210_1), or by the operation of a multimedia automatic control system (230), or by the combined operation of a wireless chip (210_2), a tag (210_3), or a user terminal (210_1) and a multimedia automatic control system (230). For example, after the wireless chip (210_2) or the tag (210_3) outputs location data, a distributed processing method may be applied in which coordinate extraction from the location data, zone determination, determination of multimedia effects, and calculation of control parameters are performed by the multimedia automatic control system (230).
[0066] A wireless chip (210_2), a tag (210_3), a user terminal (210_1), and an external device (240) can communicate with a multimedia automatic control system (230) via a network (220). Depending on the installation environment, the network (220) may be composed of a wired network such as Ethernet, Power Line Communication, telephone line communication devices, and RS-serial communication, a mobile communication network, a Wireless LAN (WLAN), Wi-Fi, Bluetooth, Ultra-wideband (UWB), and ZigBee, or a combination thereof. The communication method is not limited thereto and may include not only communication methods utilizing communication networks that the network (220) may include (e.g., mobile communication networks, wired internet, wireless internet, broadcasting networks, satellite networks, etc.), but also API (Application Programming Interface) calls and data synchronization communication between systems. For example, the multimedia automatic control system (230) can transmit a data inquiry request to an external device (240) via a network (220), and the external device (240) can receive data corresponding to this. In FIG. 2, the external device (240) may be a concept that encompasses various servers that provide data necessary for the multimedia automatic control of the present invention. In one embodiment, the external device (240) may include a server that provides information on zone division of an indoor space or information on multimedia effects set for each zone, a server that provides content of multimedia effects, or a server that manages multimedia components such as sound devices and video devices. The multimedia automatic control system (230) can be organically linked with these external devices (240) to automatically collect data necessary for multimedia automatic control and verify consistency.
[0067] Additionally, the wireless chip (210_2) and the tag (210_3) may be any device capable of wired and / or wireless communication and capable of outputting location data, and the user terminal (210_1) may be any computing device capable of wired and / or wireless communication and capable of having an interface related to multimedia automatic control installed and executed. For example, the user terminal (210_1) may include a smartphone, computer, laptop, tablet PC, etc. Additionally, FIG. 2 is illustrated as having one user terminal, one wireless chip, one tag, and one external device communicating with the multimedia automatic control system (230) through the network (220), but is not limited thereto, and may be configured to have a different number of devices communicating with the multimedia automatic control system (230) through the network (220).
[0068] In FIG. 2, a configuration in which user location data is transmitted to a multimedia automatic control system (230) via a wireless chip (210_2) or a tag (210_3) is illustrated as an example, but is not limited thereto. Location data may be provided to the multimedia automatic control system (230) through an input device associated with the multimedia automatic control system (230) without passing through the wireless chip (210_2) or the tag (210_3), and the result of processing the location data (e.g., calculated control parameters, etc.) may be provided to the user through an output device associated with the multimedia automatic control system (230) (e.g., audio device, video device, etc.). FIG. 2 illustrates that the wireless chip (210_2) and the tag (210_3) receive information processing services related to multimedia automatic control from the multimedia automatic control system (230), but is not limited thereto. For example, without communication with the multimedia automatic control system (230), a multimedia automatic control service may be provided in a local environment through a program / application (On-premise Software) installed on the user terminal (210_1).
[0069] Although the multimedia automatic control system (230) is depicted as a single device, it is not limited thereto, and the multimedia automatic control system (230) may be composed of multiple devices such as a location data receiving server, a zone determination server, and a multimedia control server. Additionally, although the multimedia automatic control system (230) is depicted in FIG. 2 as an independent device separated from the wireless chip (210_2) and the tag (210_3), it is not limited thereto, and the multimedia automatic control system (230) may be implemented by integrating with the wireless chip (210_2) or the tag (210_3).
[0070] FIG. 3 is a block diagram schematically showing the internal configuration of a user terminal, tag, and multimedia automatic control system according to one embodiment of the present disclosure.
[0071] Referring to FIG. 3, the user terminal (210_1) may refer to any computing device capable of running a web browser, dedicated applications, etc., and capable of wired / wireless communication, and may include, for example, the user terminal (210_1) of FIG. 2. As illustrated, the user terminal (210_1) may include a memory (212), a processor (214), a communication module (216), and an input / output interface (218). The multimedia automatic control system (230) may correspond to the multimedia automatic control system (230) of FIG. 2 and may include a memory (232), a processor (234), a communication module (236), and an input / output interface (238). As illustrated, the user terminal (210_1) and the multimedia automatic control system (230) may be configured to communicate information and / or data through a network (220) using their respective communication modules (216, 236). Additionally, the input / output device (219) may be configured to input (or provide) information and / or data to the user terminal (210_1) through the input / output interface (218) or to output information and / or data generated from the user terminal (210_1). Meanwhile, the main embodiment for outputting location data in the present disclosure may be a tag (210_3) attached to the user, and the tag (210_3) may provide the calculated location data to the multimedia automatic control system (230) through the network (220) or transmit it to the multimedia automatic control system (230) through a listener by transmitting and receiving wireless signals with a plurality of anchors. The user terminal (210_1) may be configured to output the user's location data, but may also be utilized as an auxiliary configuration for setting multimedia effects or monitoring control operations.
[0072] The memory (212, 232) may include any non-temporary computer-readable recording medium and may store at least one program code for performing an operating system and the multimedia automatic control method of the present invention. In one embodiment, the memory (232) of the multimedia automatic control system (230) may store location data parsing logic according to the present invention, a zone map dividing an indoor space into a plurality of zones, information regarding multimedia effects set for each zone, calculation logic for calculating control parameters of multimedia effects, and a communication protocol conversion algorithm corresponding to each multimedia component. Additionally, the memory (232) may temporarily or permanently store location data received from a tag (210_3) or a wireless chip, refined location data generated through parsing, zone determination results, etc.
[0073] The processor (214, 234) may be configured to process commands of a computer program by performing basic arithmetic, logic, and input / output operations. For example, the processor (234) of the multimedia automatic control system (230) may be configured to perform a series of processes according to program code stored in memory (232), such as parsing received location data in the form of a string to extract the identifier and two-dimensional coordinates of a tag, applying the extracted coordinates to a zone map to determine the zone where the user is located, determining multimedia effects corresponding to the determined zone to calculate control parameters, and converting them into communication protocols corresponding to each multimedia component to generate a result. Additionally, the processor (234) may perform operations to automatically control multimedia by repeating a series of processes in real time according to the user's location and movement.
[0074] The communication module (216, 236) can provide a configuration or function for the user terminal (210_1) and the multimedia automatic control system (230), and the multimedia automatic control system (230) and an external device to communicate with each other through the network (220). In particular, the communication module (236) can perform the function of receiving location data output from a tag (210_3) or a wireless chip, or transmitting calculated control parameters to multimedia components such as an audio device and a video device.
[0075] The input / output interface (238) may be a means for interfacing with an input / output device associated with a multimedia automatic control system (230). In one embodiment, the input device may include a keyboard, a mouse, etc., for inputting commands to set a zone map or zone-specific multimedia effects, and the output device may include a display, etc., for displaying location tracking results, zone determination results, the status of control operations, etc. In another embodiment, the input / output interface (238) may be a means for interfacing with a device in which configurations or functions for performing input and output are integrated into one, such as a touchscreen.
[0076] The processor (234) of the multimedia automatic control system (230) performs a series of operations related to the multimedia automatic control described above, and outputs the state or result of the control operation through an input / output interface (238) or a communication module (236) or transmits it to a user terminal (210_1), so that the user can check the control operation or inspect the multimedia production.
[0077] FIG. 4 is a block diagram showing the internal configuration of a multimedia automatic control system according to one embodiment of the present disclosure. FIG. 5 is a flowchart for explaining a method for automatically controlling multimedia according to one embodiment of the present disclosure.
[0078] The multimedia automatic control system (100) of FIG. 4 may correspond to the multimedia automatic control system (230) of FIG. 2 and FIG. 3.
[0079] Referring to FIG. 4, the multimedia automatic control system (100) may include a receiving unit (310), a location data generating unit (320), a judgment unit (330), an effect selection unit (340), a parameter calculation unit (350), and a control unit (360). Each component illustrated in FIG. 4 is functionally distinct and does not necessarily mean a physically separated separate component; it may be implemented as a software module executed by at least one processor or distributed across multiple computing devices.
[0080] In step S100, the receiver (310) can receive location data in the form of a string representing the user's location, which is output from a plurality of anchors installed in an indoor space (e.g., a stage) and a tag attached to at least one user. In one embodiment, the tag can transmit and receive UWB wireless signals with a plurality of anchors and output its own location as location data calculated based on distance and time information with each anchor, and the receiver (310) can receive such location data through a listener. In one embodiment, the location data can be received in the form of a string including the tag's identifier, coordinate values, quality values, etc.
[0081] In step S200, the location data generation unit (320) can generate refined location data by extracting the tag identifier and 2D coordinates from the location data through regular expression-based parsing. Since the location data received by the receiving unit (310) is in the form of a string, a process of extracting and refining the desired information may be required to utilize it for subsequent zone determination and control parameter calculation. The location data generation unit (320) can generate refined location data in a form usable for interactive multimedia control by separating and processing information corresponding to the tag identifier and 2D coordinates (x, y) from the string using a regular expression.
[0082] In step S300, the determination unit (330) can determine the zone where the user is located by applying refined location data to a zone map in which the indoor space is divided into multiple zones. In one embodiment, the zone map may be data defined by dividing the indoor space into multiple zones according to a predetermined standard, for example, it may be divided into multiple zones along the width direction of the indoor space. The determination unit (330) can determine the zone where the user is currently located by determining which of the multiple zones the coordinates of the refined location data belong to.
[0083] In one embodiment, the determination unit (330) may determine the area where the user is located based on refined location data, and in addition, calculate at least one derived indicator derived from the refined location data. The derived indicator may include at least one of the following: the number of objects representing the number of users located within a specific area or specific region, the distance between the user and a reference location, the user's movement speed calculated from the change in refined location data over time, the user's movement direction, the time spent by the user within the specific area or specific region, and a clustering diagram representing the density of multiple users located within the specific region. For example, the determination unit (330) may calculate the user's movement speed by dividing the amount of coordinate change in the refined location data updated over time by the time interval, and may distinguish whether the user is in a walking state or a driving state based on the calculated movement speed. Additionally, the determination unit (330) may calculate a clustering diagram corresponding to the number of users per unit area based on the coordinate distribution of users located within the specific region.
[0084] In step S400, the effect selection unit (340) can select a multimedia effect corresponding to a determined zone from among the multimedia effects for a plurality of multimedia components, including a sound device and a video device, which are set for a plurality of zones. In the present disclosure, different multimedia effects may be set for each of the plurality of zones; for example, the type of sound output through the sound device, the object displayed through the video device, etc., may be set differently for each zone. By selecting a multimedia effect that is pre-set in correspondence with the zone determined by the determination unit (330), the effect selection unit (340) can enable different multimedia effects to be produced depending on the zone where the user is located. Thus, the multimedia automatic control system (100) according to the present disclosure can implement more diverse and three-dimensional effects by adopting a method of selecting a multimedia effect corresponding to the zone where the user is located, rather than directly mapping the user's location coordinates to a multimedia effect one-to-one.
[0085] In one embodiment, the effect selection unit (340) or the parameter calculation unit (350) may select a multimedia effect or calculate a control parameter based not only on the determined area but also on the derived indicator calculated by the determination unit (330). For example, the type or intensity of the multimedia effect corresponding to a certain area may be determined differently depending on the number of objects, clustering, or dwell time of the user located in a certain area, and the transition point or transition speed of the multimedia effect may be determined differently depending on the user's movement speed or direction of movement.
[0086] In step S500, the parameter calculation unit (350) can calculate control parameters for selected multimedia effects based on refined location data. That is, when the effect selection unit (340) selects a type of multimedia effect corresponding to the area where the user is located, the parameter calculation unit (350) can calculate control parameters indicating how the corresponding multimedia effect will be specifically applied according to the refined location data.
[0087] In one embodiment, when the selected multimedia effect includes sound output by the sound device, the parameter calculation unit (350) can calculate left-right panning values and volume attenuation amounts for the sound. Specifically, the parameter calculation unit (350) can calculate left-right panning values of the original voice of the user and left-right panning values of the modified processed sound, and in this case, the left-right panning values of the original sound and the left-right panning values of the processed sound can be calculated to face in opposite directions. For example, when the width of the indoor space is W and the x-coordinate of the refined position data is x, the left-right panning value of the original sound (P_voice) and the left-right panning value of the processed sound (P_proc) can be calculated by the following [Equation 1].
[0088] [Mathematical Formula 1]
[0089]
[0090]
[0091] Here, the panning value is a value representing the localization of the sound in the left-right direction, and in one embodiment, it may have a value from 0 to 127. According to [Equation 1], as the user moves to one side along the width direction of the indoor space, the original sound is localized in that direction and the processed sound is localized in the opposite direction, and when the user is located at the center of the width direction, both the original sound and the processed sound can be localized at the center. Through this, the sense of separation and localization of the sound can be expressed in real time according to the user's position. More specifically, according to [Equation 1], the sum of the left-right panning value of the original sound and the left-right panning value of the processed sound can be maintained at a constant value of the maximum left-right panning value. Accordingly, the sound image of the original sound and the sound image of the processed sound can be localized at positions symmetrical to each other with respect to the center of the width direction of the indoor space. In this case, since the original sound is positioned on the side where the user is located, the user's visual position and the auditory position of the original sound can coincide, whereas the processed sound is positioned on the opposite side, allowing it to be spatially separated from the original sound. Furthermore, as the user moves away from the center in the width direction, the left-right gap between the sound images of the original sound and the processed sound increases, and as the user approaches the center in the width direction, this gap decreases, allowing the two sound images to converge toward the center. Here, since the original sound and the processed sound originate from the same user's voice and the processed sound is a modified version of the original sound, if both the original sound and the processed sound are positioned on the side where the user is located, the sound images of the two sounds overlap at the same point in the left-right direction; thus, the user's left-right movement can only be expressed as a change in the parallel movement of the overlapping sound images. In contrast, when the original sound and the processed sound are positioned in opposite directions as in [Equation 1], the user's left and right movement can be simultaneously expressed as a change in which the sound image of the original sound moves along with the user, and a change in the gap between the sound image of the original sound and the sound image of the processed sound, which spread apart or converge with each other.Furthermore, even when a user is located at one edge of the indoor space, the processed sound is positioned on the opposite side; thus, the sound is not concentrated on the side where the user is located, and the entire width of the indoor space can be utilized for sound production. Additionally, since the processed sound is a modified version of the original sound, it may have a similar timbre to the original sound; however, because the two are positioned in opposite directions and spatially separated, they can be perceived as distinct.
[0092] Additionally, the parameter calculation unit (350) can calculate the sound attenuation amount to express the sense of distance according to the user's forward and backward movement. For example, when the depth of the indoor space is D, the y-coordinate of the refined position data is y, and the attenuation coefficient is α, the sound attenuation amount (G_dB) can be calculated by the following [Equation 2].
[0093] [Mathematical Formula 2]
[0094]
[0095] Here, the attenuation coefficient α may be a coefficient that determines the degree of sound attenuation. According to [Equation 2], as the user moves away to one side along the front-back direction, the sound attenuates, and the sense of distance from the user can be audibly expressed. Meanwhile, [Equation 1] and [Equation 2] are exemplary, and the relationship equations for calculating control parameters of multimedia effects can be set in various ways depending on the type of multimedia effect and the characteristics of the indoor space.
[0096] In one embodiment, the multimedia automatic control system (100) according to the present disclosure may perform at least one of a discrete switching method of control and a continuous changing method of control when controlling multimedia components based on the user's location or derived indicator.
[0097] The control of the discrete switching method may be a method of switching the state of a multimedia component from a first state to a second state when the user's location or derived indicator satisfies a preset condition. For example, the effect selection unit (340) may switch the multimedia effect applied to the multimedia component from a first effect to a second effect when conditions are met, such as when the user enters a specific area, when the number of objects in a certain area exceeds a preset number, or when the user's dwell time exceeds a preset time. Through this control of the discrete switching method, a cue-based automatic production in which preset productions are sequentially switched according to the satisfaction of specific conditions can be implemented.
[0098] The control method of continuous change may be a method of continuously changing control parameters for multimedia components according to changes in the user's location or derived indicators. For example, the parameter calculation unit (350) may calculate control parameters such that as the user moves in one direction, the panning value of the sound corresponding to that direction gradually increases, and accordingly, multimedia production linked to the user's location change may be implemented.
[0099] The multimedia automatic control system (100) according to the present disclosure can perform control of a discrete switching method and control of a continuous change method together within a single system, thereby enabling simultaneous implementation of a transition of the performance and a gradual change of the performance linked to a change in position according to the fulfillment of conditions.
[0100] In step S600, the control unit (360) can automatically control multiple multimedia components in real time according to the determined area by converting the calculated control parameters into a communication protocol corresponding to each multimedia component and transmitting them. In one embodiment, since multimedia components such as an audio device and a video device can be controlled by different communication protocols, the control unit (360) can convert the calculated control parameters into a communication protocol that each multimedia component can recognize and transmit them. For example, the control unit (360) can convert the control parameters for the audio device into a protocol that the audio device can recognize and transmit them, and convert the control parameters for the video device into a protocol that the video device can recognize and transmit them.
[0101] The steps S100 to S600 described above can be repeatedly performed in real time according to the user's location and movement. That is, as the user moves within the indoor space, their location is continuously tracked, and multimedia effects and control parameters corresponding to the tracked location are updated in real time so that the multimedia components can be automatically controlled. Through this, the multimedia automatic control system (100) according to the present disclosure can implement a spatially responsive interaction in which the user's location and movement are linked with multimedia production in real time.
[0102] FIG. 6 is a diagram illustrating a multimedia control operation according to one embodiment of the present disclosure.
[0103] Referring to FIG. 6, a multimedia automatic control system (100) according to one embodiment of the present disclosure may include a first computing device (160) and a second computing device (170). In one embodiment, a receiving unit (310) and a location data generating unit (320) may be provided in the first computing device (160), and a judgment unit (330), an effect selection unit (340), a parameter calculation unit (350), and a control unit (360) may be provided in the second computing device (170). Distributing the configuration of the multimedia automatic control system (100) between the first computing device (160) and the second computing device (170) in this manner may be particularly advantageous when there is a physical distance between the location where the location data is output and the location where the multimedia component is controlled.
[0104] Referring to FIG. 6, the UWB module (400) may include an anchor (120), a tag (140), and a listener (150). In one embodiment, the anchor (120), the tag (140), and the listener (150) may each have their roles individually assigned to the same type of UWB module, and accordingly, a configuration for location tracking can be provided without having separate different hardware. Location data (LD) output from the tag (140) may be transmitted to a first computing device (160) through the listener (150). In one embodiment, the listener (150) and the first computing device (160) may be connected via USB serial communication, and the listener (150) may transmit the location data (LD) received from the tag (140) to a receiver (310) of the first computing device (160) via USB serial communication.
[0105] A receiving unit (310) provided in the first computing device (160) receives location data (LD), and a location data generating unit (320) can generate refined location data (RLD) from the location data (LD) through regular expression-based parsing. The generated refined location data (RLD) can be transmitted from the first computing device (160) to the second computing device (170).
[0106] In one embodiment, the first computing device (160) can transmit refined location data (RLD) to the second computing device (170) wirelessly via User Datagram Protocol (UDP) communication. By transmitting the refined location data (RLD) wirelessly in this manner, data can be transmitted without the constraints of a wired connection even when there is a physical distance between the first computing device (160) and the second computing device (170). Additionally, UDP communication is a method that prioritizes transmission speed over communication reliability and is suitable for one-to-many communication, thereby reducing the temporal difference in representation that may occur between the user's actual location and multimedia effects.
[0107] In one embodiment, the first computing device (160) may be positioned adjacent to a location where location data (LD) is output, and the second computing device (170) may be positioned adjacent to a location controlling multimedia components. For example, if the indoor space is a performance hall, the first computing device (160) may be positioned on the stage side and connected to the listener (150) via USB serial communication, and the second computing device (170) may be positioned on the operator side that controls multimedia components. Since the data transmission speed of the USB serial communication connecting the listener (150) and the first computing device (160) may decrease as the connection length increases, the efficiency and stability of data transmission can be ensured by positioning the first computing device (160) adjacent to the listener (150) and connecting the long distance between the first computing device (160) and the second computing device (170) via wireless communication.
[0108] A determination unit (330) equipped in the second computing device (170) determines the area where the user is located based on refined location data (RLD), an effect selection unit (340) selects a multimedia effect corresponding to the determined area, and a parameter calculation unit (350) can calculate control parameters of the selected multimedia effect. Subsequently, a control unit (360) can convert the calculated control parameters into a communication protocol corresponding to each multimedia component and transmit them.
[0109] Referring to FIG. 6, the control unit (360) can calculate and transmit a first control parameter (PRM1) for the sound device (410), a second control parameter (PRM2) for the image device (420), and a third control parameter (PRM3) for the lighting device (430), respectively. In one embodiment, since the sound device (410), the image device (420), and the lighting device (430) can each be controlled by different communication protocols, the control unit (360) can convert the first control parameter (PRM1) into a communication protocol recognizable by the sound device (410) and transmit it, convert the second control parameter (PRM2) into a communication protocol recognizable by the image device (420) and transmit it, and convert the third control parameter (PRM3) into a communication protocol recognizable by the lighting device (430) and transmit it. Through this, multiple multimedia components operating by different communication protocols can be integrally controlled by a single multimedia automatic control system (100).
[0110] As described above, a multimedia automatic control system (100) according to one embodiment of the present disclosure can eliminate limitations based on the physical distance between the location where location tracking is performed and the location where the multimedia component is controlled by distributing a first computing device (160) responsible for receiving location data (LD) and generating refined location data (RLD), and a second computing device (170) responsible for area determination and multimedia control, and by linking both devices via wireless communication.
[0111] FIG. 7 is a diagram illustrating a multimedia control operation based on the number of users per zone according to one embodiment of the present disclosure.
[0112] In FIG. 7, a lighting device is illustrated as an exemplary multimedia component, but this is for convenience of explanation and can be applied in the same way to various multimedia components such as sound devices and video devices.
[0113] Referring to FIG. 7, the indoor space (110) may be divided into multiple zones, for example, a first zone (111), a second zone (112), and a third zone (113). Additionally, at least one multimedia component may be assigned to each zone, for example, a first lighting device (430_1) may be assigned to the first zone (111), a second lighting device (430_2) to the second zone (112), and a third lighting device (430_3) to the third zone (113).
[0114] In one embodiment, tags may be attached to each of a plurality of users and provided in multiple numbers. In this case, the determination unit (330) can determine the area where the user is located for each tag and calculate the number of users in each area. That is, the determination unit (330) can individually determine the area where each user is located by inputting refined location data generated from each of the multiple tags into an area map, and calculate the number of users located in each area by aggregating the results. For example, as shown in FIG. 7, the determination unit (330) can calculate the number of users located in the first area (111) as 3, the number of users located in the second area (112) as 2, and the number of users located in the third area (113) as 5.
[0115] The parameter calculation unit (350) can change the effect of a multimedia component corresponding to a zone according to the number of users in a plurality of zones. Specifically, the parameter calculation unit (350) can calculate a control parameter that changes at least one of the volume of the audio device and the display form of the object of the video device corresponding to a zone according to the number of users located in a zone. In one embodiment, the parameter calculation unit (350) can calculate a control parameter such that the volume of the audio device corresponding to a zone increases as the number of users located in a zone increases, or calculate a control parameter such that the object of the video device corresponding to a zone is displayed with greater emphasis.
[0116] For example, the third lighting device (430_3) corresponding to the third zone (113) with the highest number of users can be controlled to the highest brightness or the most emphasized color, and the second lighting device (430_2) corresponding to the second zone (112) with a relatively low number of users can be controlled to a relatively low brightness. That is, by controlling the effects of the multimedia components corresponding to each zone differently according to the number of users in that zone, the distribution of users can be reflected in the multimedia production in real time.
[0117] Additionally, in one embodiment, the parameter calculation unit (350) may change multimedia effects based on the clustering of users within a zone or the duration of stay of users in that zone, in addition to the number of users per zone. For example, control parameters may be calculated so that the higher the clustering of a zone, the more the multimedia effect corresponding to that zone is emphasized, and control parameters may be calculated so that the multimedia effect corresponding to that zone gradually changes when users stay in a zone for a preset duration or longer.
[0118] As such, the multimedia automatic control system (100) according to one embodiment of the present disclosure can recognize each user individually and group them by zone even when multiple users are present in an indoor space, and can automatically adjust the multimedia effects of the corresponding zone according to the number of users in each zone. Through this, dynamic multimedia effects that respond to the location distribution of users can be implemented, and automated effects can be performed without separate manual operation even in situations where multiple users are present. In addition, the utilization of the indoor space can be improved by providing differentiated multimedia effects according to the number of users in each zone even when they are not visually distinguishable.
[0119] FIG. 8 is a diagram illustrating a multimedia control operation based on zone location according to one embodiment of the present disclosure.
[0120] Although multimedia components are not separately illustrated in FIG. 8 for the convenience of explanation, the same can be applied to various multimedia components such as lighting devices, sound devices, and video devices.
[0121] Referring to FIG. 8, a plurality of zones of the indoor space (110) may include a zone corresponding to the stage area (114) and a zone located outside the stage area (114) corresponding to the user's entry / exit path (115). In one embodiment, a plurality of zones of the indoor space (110) may include a first edge area (116) located on one side of the stage area (114) and a second edge area (117) located on the other side. That is, the zones according to the present disclosure may be configured to include not only the stage area (114) where multimedia effects are provided to the user, but also the entry / exit path (115) through which the user enters the stage area (114) or leaves the stage area (114), and edge areas (116, 117) located on the side of the stage area (114).
[0122] According to one embodiment of the present disclosure, the determination unit (330) can determine whether the user is in an entry state or an exit state based on changes in the refinement location data over time. As previously described, since the refinement location data can be updated in real time according to the user's location and movement, the determination unit (330) can determine the direction of movement of the user from the refinement location data that is updated over time. Specifically, based on the user's movement path, the determination unit (330) can determine the user's state as an entry state when the user enters the stage area (114) from the entry / exit path (115), and can determine the user's state as an exit state when the user leaves the stage area (114) to the entry / exit path (115).
[0123] The effect selection unit (340) can select different multimedia effects depending on the determined state. Specifically, the effect selection unit (340) can select a pre-set first multimedia effect when the user is in an entry state, and select a pre-set second multimedia effect that is different from the first multimedia effect when the user is in an exit state. For example, when the user is in an entry state, a lighting effect or sound effect that highlights the user's appearance may be selected, and when the user is in an exit state, a lighting effect or sound effect suitable for the user's exit may be selected.
[0124] Additionally, in one embodiment, the judgment unit (330) may further calculate the user's movement speed from the change over time of the refined position data, and the effect selection unit (340) may select different multimedia effects depending on the user's movement speed. For example, different multimedia effects may be selected for cases where the user enters the stage area (114) at a speed greater than a preset speed and cases where the user enters at a speed less than a preset speed, and accordingly, a differentiated performance may be implemented depending on whether the user appears by running or by walking.
[0125] Additionally, in one embodiment, the first edge area (116) and the second edge area (117) can be utilized as areas located on the side of the stage area (114) to which separate multimedia effects are applied. In one embodiment, when the effect selection unit (340) determines that the user is located in the first edge area (116) or the second edge area (117), it can select a multimedia effect pre-set in the corresponding edge area that is different from the multimedia effect corresponding to the stage area (114). In addition, different multimedia effects can be set in the first edge area (116) and the second edge area (117), respectively, and accordingly, a differentiated multimedia performance can be achieved depending on which side of the stage area (114) the user is located. Through this, the side space of the stage area (114) can also be utilized as an area for multimedia performance, thereby further enhancing the utilization of the indoor space.
[0126] In this way, the multimedia automatic control system (100) according to one embodiment of the present disclosure can identify the entry or exit of a user based not only on which area the user is located in, but also on which area the user moves from to which area, and can automatically apply different multimedia effects corresponding thereto. Through this, multimedia performance linked to the user's entry and exit from the stage can be automatically implemented without separate manual operation, and the utilization of the indoor space can be improved by setting and utilizing the edge area located outside the stage area (114) as an area.
[0127] The method described above may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or multiple hardware components combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Furthermore, other examples of media may include recording or storage media managed by app stores that distribute applications or sites and servers that supply or distribute various other software.
[0128] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in connection with the present disclosure may be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate such interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether such functions are implemented in hardware or in software depends on the design requirements imposed on the specific application and the overall system. Those skilled in the art may implement the described functions in various ways for each specific application, but such implementations should not be construed as departing from the scope of the present disclosure.
[0129] In a hardware implementation, the processing units used to perform the techniques may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or a combination thereof.
[0130] Accordingly, the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors coupled with a DSP core, or any other combination of configurations.
[0131] In firmware and / or software implementations, techniques may be implemented as instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and may cause the processor(s) to perform specific aspects of the functions described in this disclosure.
[0132] When implemented in software, the techniques described above may be stored on a computer-readable medium as one or more instructions or code, or transmitted through a computer-readable medium. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transmission of a computer program from one place to another. Storage media may be any available media accessible by a computer. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium accessible by a computer that can be used to transfer or store desired program code in the form of instructions or data structures. Additionally, any connection is appropriately referred to as a computer-readable medium.
[0133] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, coaxial cable, fiber optic cable, twisted pair cable, digital subscriber line, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disks and discs include CDs, laser discs, optical discs, DVDs (digital versatile discs), floppy disks, and Blu-ray discs, wherein disks usually play data magnetically, whereas discs play data optically using a laser. The above combinations should also be included within the scope of computer-readable media.
[0134] The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of storage medium. An exemplary storage medium may be connected to a processor so that the processor can read information from the storage medium or write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may exist within an ASIC. The ASIC may exist within a user terminal. Alternatively, the processor and the storage medium may exist as separate components within the user terminal.
[0135] Although the embodiments described above have been described as utilizing aspects of the subject matter disclosed herein in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across a plurality of devices. Such devices may include PCs, network servers, and portable devices.
[0136] Although the present disclosure has been described in relation to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure as understood by a person skilled in the art to which the invention of the present disclosure pertains. Furthermore, such modifications and changes should be considered to fall within the scope of the claims appended to this specification. Explanation of the symbols
[0138] 100: Multimedia Automatic Control System 110: Stage 120: Anchor 130: User 140: Tag 150: Listener 160: First computing device 170: Second computing device
Claims
Claim 1 A receiving unit configured to receive location data in the form of a string representing the location of a user, output from a plurality of anchors installed in an indoor space and a tag attached to at least one user; a generating unit configured to generate refined location data by extracting the identifier and 2D coordinates of the tag through regular expression-based parsing from the location data; a determining unit configured to determine the zone where the user is located by applying the refined location data to a zone map that divides the indoor space into a plurality of zones along the width direction; a selection unit configured to select a multimedia effect corresponding to the determined zone from among multimedia effects that are set differently for each of the plurality of zones and include a type of sound processing applied to the sound output by the audio device as a multimedia effect for a plurality of multimedia components including an audio device and a video device; and a parameter calculation unit configured to calculate control parameters of the selected multimedia effect based on the refined location data. and includes a control unit configured to automatically control the plurality of multimedia components in real time according to the determined zone by converting the control parameters into a communication protocol corresponding to each multimedia component and transmitting them, and the sound device is configured to output together the original voice of the user and a processed sound in which the original sound is modified according to the type of sound processing included in the selected multimedia effect, and the parameter calculation unit is further configured to calculate the left-right panning value of the original sound and the left-right panning value of the processed sound based on the following [Equation 1] and to calculate the volume attenuation amount of the sound based on the following [Equation 2], [Equation 1] [Mathematical Formula 2] (Here, P_voice is the left-right panning value of the original sound, P_proc is the left-right panning value of the processed sound, x is the x-coordinate of the refined position data, W is the width of the indoor space, G_dB is the sound attenuation amount, y is the y-coordinate of the refined position data, D is the depth of the indoor space, and α is the attenuation coefficient) The left-right panning value of the original sound is calculated so that the original sound is positioned toward the side where the user is located, and the left-right panning value of the processed sound is calculated so that the sum with the left-right panning value of the original sound is maintained constant at the maximum value of the left-right panning value, so that the processed sound is positioned toward the side symmetrical to the original sound with respect to the center in the width direction of the indoor space, and accordingly, as the user moves away from the center in the width direction, the left-right distance between the sound image of the original sound and the sound image of the processed sound increases, and as the user approaches the center in the width direction, the distance decreases, and the control unit is configured such that the type of sound processing selected by the selection unit based on the user moving from one of the plurality of zones to another zone A multimedia automatic control system based on Ultra-wideband (UWB) location data, further configured to perform discrete control that switches and continuous control in which the left-right panning value of the original sound and the left-right panning value of the processed sound change continuously based on the change in the refined location data, wherein the receiving unit and the generating unit are provided in a first computing device, and the judgment unit, the selection unit, the parameter calculation unit, and the control unit are provided in a second computing device, and wherein the first computing device is configured to receive the location data via USB serial communication from a listener that transmits the location data output from the tag, and to wirelessly transmit the refined location data to the second computing device via User Datagram Protocol (UDP) communication. Claim 2 delete Claim 3 A multimedia automatic control system based on UWB location data, wherein, in claim 1, the tags are provided in multiple numbers, each attached to a plurality of users, and the determination unit is further configured to determine the area where the user is located for each tag and to calculate the number of users in each of the plurality of areas, and the parameter calculation unit is further configured to change at least one of the volume of the sound device corresponding to the area and the display form of the object of the video device according to the number of users in each of the plurality of areas. Claim 4 A multimedia automatic control system based on UWB location data, wherein the plurality of zones includes a first zone corresponding to a stage area of the indoor space and a second zone located outside the stage area corresponding to the user's entry or exit path, and the determination unit is further configured to determine an entry state in which the user moves from the second zone to the first zone and an exit state in which the user moves from the first zone to the second zone based on changes over time in the refined location data, and the selection unit is further configured to select a preset first multimedia effect when the user is in the entry state and to select a preset second multimedia effect different from the first multimedia effect when the user is in the exit state. Claim 5 delete
Citation Information
Patent Citations
Stage direction apparatus
KR1020150103791A
Integrated performance system for stage equipment
KR102504276B1
