A theater light, sound, and video intelligent management method based on an internet of things

Abstract

The application discloses a theatre light sound effect video intelligent management method based on Internet of Things and relates to the field of intelligent management. The method comprises the following steps: deploying an Internet of Things sensing layer in a theatre auditorium area; constructing an audience infrasound-light coupling fatigue accumulation model according to the time accumulation of the infrasound frequency band and the time change rate of the light frequency flicker index, and calculating a state base value; collecting video images of the auditorium area in real time through the video acquisition equipment, extracting the features of human postures and head movements in the images, and calculating a crowd activity factor. The application can accurately estimate the real-time state of the audience by constructing the audience infrasound-light coupling fatigue accumulation model to calculate the state base value, extracting the crowd activity factor in combination with the features of human postures and head movements, and generating a comprehensive sensory state index through a fuzzy neural coupling network.

Description

Technical Field

[0001] This invention relates to the field of intelligent management, specifically to an intelligent management method for theater lighting, sound effects, and video based on the Internet of Things. Background Technology

[0002] In modern theatrical performances, lighting, sound effects, and video systems are core means of creating an immersive viewing experience. Sound systems often include infrasound frequencies (typically below 20Hz) to enhance the tactile experience, while lighting systems widely employ PWM dimming technology to achieve brightness adjustment and flicker effects. Research shows that prolonged exposure to infrasound can lead to physiological reactions such as fatigue and decreased attention span, while the light flicker generated by PWM dimming can also have a certain impact on the nervous system. Meanwhile, the collective active behaviors of the audience (such as body movements and head movements) can alleviate fatigue to some extent and enhance the interactive atmosphere of the performance. Therefore, how to comprehensively perceive the physiological and behavioral states of the audience and adjust the performance equipment accordingly in real time has become a key issue in improving the viewing experience.

[0003] Currently, some intelligent theater management systems exist, typically using sensors to collect environmental parameters or simple behavioral data from the audience seating area. For example, cameras detect whether audience members are leaving or microphones capture ambient volume. Some systems can also control lighting and sound according to a pre-set performance script. Furthermore, some research has attempted to monitor individual audience members' conditions using physiological sensors such as heart rate and skin conductance sensors; however, due to high cost, complex deployment, and privacy concerns, these methods are difficult to implement in large theaters. Overall, existing systems mostly employ open-loop control strategies, lacking the ability to jointly perceive audience fatigue and active behavior, and are unable to achieve adaptive adjustment based on real-time feedback.

[0004] However, the aforementioned existing technologies have significant shortcomings in practical applications. Traditional methods fail to effectively distinguish between infrasound cumulative fatigue and optical flicker coupling fatigue, leading to significant errors in estimating the audience's actual fatigue state. Existing systems generally ignore the characteristic that the active behaviors of the audience (such as head turning, shoulder swaying, etc.) can serve as natural counteracting signals of physiological fatigue, thus missing the opportunity to alleviate fatigue through collective interaction. The lack of an adaptive compensation mechanism that compares the audience's state with the expected goals of the performance script in a closed loop often results in equipment adjustments being lagging or excessive, making it difficult to achieve precise immersion management. Therefore, how to construct an intelligent management method that can perceive the audience's physiological-behavioral coupling state in real time and dynamically adjust the lighting, sound, and video systems accordingly is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] Based on this, the purpose of this invention is to provide an intelligent management method for theater lighting, sound effects, and video based on the Internet of Things, in order to solve the technical problems in the prior art that cannot simultaneously perceive infrasound cumulative fatigue and optical flicker coupling fatigue, ignore the natural offsetting effect of the active behavior of the audience on fatigue, and lack adaptive compensation adjustment based on closed-loop comparison of real-time audience status and expected goals of the performance script.

[0006] To achieve the above objectives, the present invention provides the following technical solution: An Internet of Things (IoT) sensing layer is deployed in the theater audience area. This sensing layer includes: a long-time integral microphone array for collecting infrasound frequency band data from sound equipment; a photoelectric sensor array for collecting the optical flicker index from lighting equipment; and a video acquisition device deployed above the audience seating area. The sound equipment is equipped with a real-time adjustable infrasound frequency band filter module, and the lighting equipment is equipped with a programmable PWM dimming frequency generator. Based on the time accumulation of the infrasound frequency band data and the time change rate of the optical flicker index, an audience infrasound-optical coupling fatigue accumulation model is constructed, and the state base value is calculated. The video capture device acquires video images of the audience seating area in real time, extracts features from human postures and head movements in the images, and calculates the crowd activity factor. ; the state base value With crowd activity factor The input is fed into a fuzzy neural coupling network to construct a fatigue-activity nonlinear mapping model, and outputs a comprehensive sensory state index. ; Obtain the expected sensory impact sequence in the current performance script, wherein the expected sensory impact sequence is a sequence of target values ​​for the comprehensive sensory state index that the audience is expected to achieve at different stages of the performance, pre-set according to the performance script timeline. The comprehensive sensory state index calculated in real time The expected target value corresponding to the time in the expected sensory impact sequence The difference between the two values ​​is used as a control deviation input to the adaptive compensation controller, which generates a device adjustment vector based on the control deviation. The expected sensory impact timing sequence is a pre-set sequence of comprehensive sensory state target values ​​that the audience is expected to achieve at different stages of the performance, based on the timeline of the performance script. The current performance script is a pre-stored structured data file describing the timeline of the performance content. The device adjustment vector... The parameter settings are applied synchronously to the subsonic frequency band dynamic notch filter of the audio equipment and the PWM dimming frequency generator of the lighting equipment.

[0007] In summary, the present invention has the following main beneficial effects:

[0008] This invention calculates the baseline state value by constructing a fatigue accumulation model of audience infrasound-light coupling, and extracts crowd activity factors by combining human posture and head movement characteristics. Then, it generates a comprehensive sensory state index through a fuzzy neural coupling network, which can accurately estimate the real-time state of the audience. On this basis, the deviation between the current state and the expected target value in the performance script is input into an adaptive compensation controller, which dynamically generates adjustment vectors that act on the dynamic notch filter of the infrasound band of the sound equipment and the PWM dimming frequency generator of the lighting equipment. This not only realizes infrasound phase cancellation and PWM frequency optimization to alleviate involuntary fatigue, but also amplifies the collective active behavior of the audience by using closed-loop rhythm interaction during the climax of the performance, forming a positive feedback from the audience's rhythm to the infrasound response of the lights. This significantly improves the immersion of the performance, prolongs the audience's attention span, and provides a self-optimizing intelligent management solution for different types of performances. Attached Figure Description

[0009] Figure 1 This is a flowchart of the intelligent management method for theater lighting, sound effects, and video according to the present invention. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0011] It should be noted in advance that the sound pressure level in the infrasound band (usually referring to below 20Hz), PWM dimming frequency, human posture characteristics, etc. involved in the method of the present invention are all physical quantities that can be obtained directly or indirectly through existing sensor technology. The acquisition method, physical meaning, unit and preferred value range of each parameter will be explained one by one below, and illustrated with specific numerical examples, so as to ensure that those skilled in the art can reproduce the technical solution described in the present invention without creative effort.

[0012] A smart management method for theater lighting, sound effects, and video based on the Internet of Things, as shown in the figure, includes:

[0013] I. System Overall Architecture

[0014] This invention provides an IoT-based intelligent management method for theater lighting, sound effects, and video. First, an IoT sensing layer is deployed in the theater's audience seating area. This sensing layer includes: a long-time integrating microphone array for collecting infrasound frequency band data from the sound equipment; a photoelectric sensor array for collecting the light flicker index from the lighting equipment; and video acquisition equipment deployed above the audience seating area. The sound equipment is equipped with a real-time adjustable infrasound frequency band filter module, and the lighting equipment is equipped with a programmable PWM dimming frequency generator. The microphone array is recommended to use a multi-channel capacitive infrasound sensor (e.g., InfraSound M201), with a frequency response range covering 0.5Hz to 20Hz. One microphone is deployed per 50 square meters in the audience seating area. The sensor output signal is pre-amplified and then converted from analog to digital by a 24-bit data acquisition card, with a sampling rate of no less than 100Hz and an integration time constant of 10 seconds to obtain long-term stable sound pressure level data. A photoelectric sensor array is used to detect the PWM dimming frequency of the lights. Photodiodes with a rise time better than 0.1 microseconds (such as the Hamamatsu S6968) are recommended. At least two photodiodes should be installed in each area (front, middle, and rear) of the audience seating area to directly measure the fundamental frequency of the PWM waveform of the lights, in kHz. The video acquisition equipment uses an infrared-enhanced wide-angle camera with a resolution of 1920×1080 and a frame rate of 30fps. It should be installed 8-12 meters above the audience seating area with a downward angle of 30-45 degrees to ensure full coverage of the audience. A dynamic notch filter for the infrasound band is a filter that can adjust the center frequency and notch depth in real time to suppress or enhance specific infrasound frequency components to mitigate or utilize the impact of infrasound on the audience. A PWM dimming frequency generator is a pulse width modulation signal generator that controls the dimming of lighting equipment. It adjusts the brightness and flicker effect of the lights by changing the frequency and duty cycle. A long-time integrating microphone array is an array of multiple microphones that can integrate and acquire infrasound signals for a long time to improve measurement stability under low signal-to-noise ratio conditions.

[0015] II. Calculation of State Base Values

[0016] Based on the time accumulation of the infrasound frequency band and the time change rate of the optical flicker index, a fatigue accumulation model of audience infrasound-optical coupling is constructed, and the state base value is calculated. :

[0017] ;

[0018] in, Current time, in seconds (s). The variable is the integral variable, and the unit is seconds (s).

[0019] For a moment The infrasound weighted sound pressure level, measured in decibels (dB). This value is measured using a long-time integrating microphone array at a center frequency of 10Hz (1 / 3 octave band) and corrected for A-weighting according to ISO 7196:1995 to simulate the actual level of human perception of infrasound. Example value: If the measured value during the climax of a performance is 92dB, then substitute 92.

[0020] Lighting equipment at all times The PWM dimming frequency, in kilohertz (kHz). It is measured using a photoelectric sensor array, typically ranging from 0.5 kHz to 4 kHz. Example: If the measured value at a certain moment is 1.2 kHz, then substitute 1.2.

[0021] This is the infrasound cumulative gain coefficient, which is dimensionless. This coefficient is obtained through feedforward calibration and is typically taken as 0.05 to 0.2. In this embodiment, it is taken as 0.12.

[0022] To preset the fatigue time decay constant, in units ,Pick The corresponding fatigue half-life is approximately 277 seconds, which means it decays to half its original value in about 4.6 minutes.

[0023] This is the optical flicker coupling coefficient, measured in seconds per kHz (i.e., seconds per kilohertz change rate). The value ranges from 2 to 8 s / kHz; in this embodiment, it is set to 5 s / kHz.

[0024] For a moment The absolute value of the PWM dimming frequency change rate, in kHz / s. It is calculated by dividing the difference between two consecutive sampling points by the sampling interval. For example, if the frequency was 1.2kHz one second ago and 1.5kHz currently, the change rate is 0.3kHz / s, and the absolute value is 0.3.

[0025] Substitute the example values: Assume the integration time is 10 seconds. , , , , The integral is approximately calculated as follows: , ,final The unit is consistent with the decibel time integral, but in this invention it is regarded as a dimensionless fatigue baseline index.

[0026] III. Factors that increase crowd activity Calculation

[0027] The video capture device acquires real-time video images of the audience seating area, extracts features from human postures and head movements in the images, and calculates the crowd activity factor. :

[0028] ;

[0029] in, This represents the dimensionless number of viewers that the video capture device can identify. It is calculated in real-time using a human pose estimation model (such as OpenPose) to determine the number of valid skeletons. Example: 150 people were identified.

[0030] For the first The head tilt angle of an audience member refers to the horizontal tilt angle of the head relative to the direction facing the stage, calculated from the angle between the head key point in the video image and the line connecting the center of the shoulder. The value ranges from 0° to 90°. Example: An audience member tilts their head 45°.

[0031] This is the normalization factor for the head deflection angle, with a value of 90°. Unit: degrees (°).

[0032] For the first The shoulder sway amplitude of an audience member, in centimeters (cm). Calculated by the lateral displacement of the left and right shoulder joint key points within the video plane and converted to actual distance. Example: An audience member's shoulder sway is 8cm.

[0033] The normalization factor for shoulder swing amplitude is 10cm. Unit: centimeters (cm).

[0034] For the first The head oscillation frequency of a spectator, in Hz (times / second). Obtained through zero-crossing rate or spectral analysis of the head's key point motion trajectory. Example: A spectator's head oscillation frequency is 0.6 Hz.

[0035] This is the normalization factor for the head oscillation frequency, which is set to 1.2Hz in this embodiment. Unit: Hz.

[0036] , , As a preset weighting factor, and In this embodiment, , , .

[0037] Example calculation: Take one audience member, , ,but , cm, cm, then , Hz, Hz, then The sum of the three terms is 0.62. If the average value of all viewers is 0.6, then .

[0038] IV. Calculation of the Comprehensive Sensory State Index

[0039] The state baseline values ​​and the crowd activity factor are input into a fuzzy neural coupling network to construct a fatigue-activity nonlinear mapping model, and output a comprehensive sensory state index.

[0040] ;

[0041] in, The comprehensive sensory state index is dimensionless. Positive values ​​indicate a state of excitement, while negative values ​​indicate a state of fatigue.

[0042] , , , To ensure that the preset adjustable parameters are used, this embodiment takes... (Compress the fatigue baseline value). , , , The unit is seconds.

[0043] The rate of change of the population activity factor, in units of: It is calculated by numerical difference, for example, the difference between two consecutive frames (0.2 seconds apart) divided by 0.2 seconds.

[0044] Substituting the example, if ,but , ,but Round to two decimal places. If we take 1.03, then ,like It took 2 seconds for the value to rise from 0.5 to 0.6, with a rate of change of 0.05. but 0.05 = 0.025 This indicates that the current audience is generally excited.

[0045] The physical meaning of this formula is: Positive contribution However, a higher fatigue value indicates a greater need for compensation; and and its rate of change Negative inhibition, namely the head movements and body rhythms of the audience, can promote the self-resetting of the vestibular system, thereby relieving infrasound fatigue.

[0046] V. Adaptive Compensation Controller and Equipment Adjustment Vector

[0047] 1. Calculation of control targets and deviations

[0048] The control objective of the adaptive compensation controller is to make the overall sensory state index as close as possible to a preset target value. The control deviation is defined. :

[0049] ;

[0050] in:

[0051] Dynamically set according to the performance script timeline, units and... Consistent, dimensionless. During the less dramatic phases of a performance (such as narrative dialogue segments). Set to a low value range [1.0, 2.0]; during the climax interaction phase (such as the music climax or audience interaction segment). Set the range to high values ​​[5.0, 7.0]; during the transition phase, The changes are smoothed by linear interpolation.

[0052] when This indicates that the current sensory state of the audience is lower than expected (too tired or not excited enough), and it is necessary to enhance the sense of immersion (i.e., increase the level of excitement).

[0053] when This indicates that the current sensory state of the audience is higher than expected (possibly due to over-excitement or tension), and it is necessary to counteract fatigue or overstimulation (i.e., appropriately reduce the intensity of stimulation).

[0054] 2. Equipment adjustment vector Composition

[0055] The device adjustment vector is a multi-dimensional control command set that simultaneously acts on the infrasound band dynamic notch filter of the audio equipment and the PWM dimming frequency generator of the lighting equipment. It is defined as:

[0056] ;

[0057] in, This is the gain adjustment for the infrasound channel, measured in decibels (dB). Positive values ​​indicate gain increase (enhancing the sense of impact), while negative values ​​indicate gain decrease (reducing fatigue).

[0058] This represents the PWM dimming frequency adjustment, measured in kilohertz (kHz). A positive value indicates an increased frequency (reducing visible flicker), while a negative value indicates a decreased frequency (increasing the rhythm of the flicker).

[0059] This is the stroboscopic depth modulation coefficient, dimensionless, with a value range of [0,1]. 0 indicates no stroboscopic effect, and 1 indicates maximum depth stroboscopic effect.

[0060] This represents the amplitude of the infrasound phase-out cancellation command, measured in decibels (dB). It is non-zero only when active infrasound cancellation is required; typically, a negative value indicates the intensity of the phase-out cancellation.

[0061] 3. Pattern generation rules based on deviation and state

[0062] The controller is based on the current , and its rate of change A hybrid strategy combining piecewise PID and fuzzy rules is used to generate... Specifically, it is divided into the following three modes:

[0063] Mode 1: Fatigue Offset Mode

[0064] Triggering conditions: Higher than the upper limit of the preset target value (upper limit deviation threshold) ), and at the same time, the activity factor of the crowd Below the low activity threshold This indicates that although the audience's sensory index was high, their physical activity was low, possibly due to passive tension or involuntary fatigue caused by infrasound.

[0065] Control objectives: To counteract harmful infrasound, reduce light flicker stimulation, and actively reduce fatigue accumulation.

[0066] Generation rules:

[0067] ;

[0068] in, : Attenuation ratio of the infrasound channel gain in Mode 1, unit: decibels per unit deviation (dB / dimensionless deviation). Used to control deviation. This is converted to a reduction in infrasound gain. The value ranges from 0.2 to 1.0 dB; in this embodiment, 0.5 dB is used. This coefficient is obtained through offline simulation or on-site tuning: in typical performance segments, it is gradually increased... This continues until the audience's subjective fatigue rating decreases by 30%.

[0069] : The PWM frequency increase ratio in Mode 1, in kilohertz per unit deviation (kHz / dimensionless deviation). Used to convert control deviation into an increase in PWM frequency. The value ranges from 0.5 to 1.2 kHz; in this embodiment, it is 0.8 kHz. This coefficient is determined by the flicker visibility threshold of the lighting equipment, ensuring that the optical flicker index decreases to a level imperceptible to the human eye after the frequency is increased.

[0070] : The proportional coefficient of the infrasound phase cancellation command in Mode 1, dimensionless. Used to map the state base value to the amplitude (in dB) of the phase cancellation signal. The value ranges from 0.01 to 0.05; in this embodiment, it is 0.02. This coefficient is determined by the maximum linear output capability of the infrasound loudspeaker system to avoid overdrive.

[0071] : The upper bound deviation threshold for mode 1 triggering, dimensionless. When The system enters fatigue mitigation mode. In this embodiment, the threshold is set to 1.0. This threshold is based on the following: when the sensory state index exceeds the target value by 1 unit and the duration exceeds 2 seconds, the audience begins to show early signs of overstimulation.

[0072] The threshold for determining inactivity in Mode 1 is dimensionless. When... The fatigue mitigation mode is triggered in time. In this embodiment, a threshold of 0.3 is used. This threshold is obtained by statistically analyzing the average activity factor of the audience before they leave multiple performances.

[0073] Example: If , ,but dB kHz, dB.

[0074] Mode 2: Enhanced Immersion Mode

[0075] Triggering conditions: Below the preset target value lower bound (lower bound deviation threshold) ), and the current performance script is marked as a "climax node" (such as a pre-marked timecode).

[0076] Control objective: Enhance the infrasound impact, synchronize the PWM frequency with the music rhythm, and increase excitement.

[0077] Generation rules:

[0078] ;

[0079] in, : The gain boost ratio for the infrasound channel in Mode 2, in decibels per unit deviation (dB / dimensionless deviation). The value ranges from 0.8 to 1.5 dB; in this embodiment, it is 1.2 dB. This coefficient is determined through subjective listening tests: the minimum rate of gain change required to enhance the infrasound perception from "perceptible" to "significantly impactful" without distortion.

[0080] : The proportional coefficient of the flicker depth modulation coefficient in Mode 2, dimensionless. Used to convert the control deviation into the flicker depth modulation amount (maximum 1). The value range is 0.2~0.5, and in this embodiment, it is 0.3. This coefficient is calibrated from the flicker depth linear response curve of the lighting equipment.

[0081] : The fundamental frequency of the current music rhythm in Mode 2, in kHz. It is obtained through a real-time beat tracking algorithm of the audio program source (such as fundamental frequency estimation based on autocorrelation). For example, the fundamental frequency of the drum beats in electronic dance music is usually 2.0~3.0kHz, which serves as the synchronization target for the PWM frequency.

[0082] : The lower bound deviation threshold triggered by mode two, dimensionless. When The audience then enters an enhanced immersion mode. In this example, a threshold of 1.0 is used. This threshold is also determined based on on-site psychophysical experiments: when the deviation exceeds 1.0, 70% of the audience members reported feeling that the performance was not engaging enough.

[0083] : The real-time PWM dimming frequency of the lighting equipment at the current moment.

[0084] Example: If , dB If the current PWM frequency is 1.2kHz and the music base frequency is 2.5kHz, then kHz.

[0085] Mode 3: Closed-loop rhythm interaction mode

[0086] Triggering condition: The rate of change in the crowd activity factor exceeds a positive threshold. This indicates that the audience suddenly becomes active (such as clapping or moving rhythmically).

[0087] Control objective: Quickly follow and amplify the collective active behavior of the audience to form a rhythmic closed loop.

[0088] Generation rules:

[0089] ;

[0090] in, In Mode 3 (Closed-Loop Rhythm Interaction Mode), the strobe depth is a proportional coefficient to the crowd activity factor after high-pass filtering; it is dimensionless. The value ranges from 0.5 to 1.0, and in this embodiment, it is set to 0.8. The larger this coefficient, the stronger the amplification of subtle changes in audience activity. It is set through on-site adjustments to a value that neither causes light sensitivity discomfort nor hinders clear perception of rhythmic following.

[0091] : The scaling factor for the infrasound envelope follower gain in Mode 3, in decibels per unit high-pass output (dB / dimensionless). It is used to convert the high-pass filtered active factor into an additional envelope gain for the infrasound channel. The value ranges from 0.2 to 0.8 dB; in this embodiment, it is 0.5 dB. This factor is determined by the dynamic headroom of the audio system to ensure that envelope follower does not affect the stability of the main gain.

[0092] : The rate of change threshold for triggering rhythm interaction mode in Mode 3, unit: .when The system then enters a fast follow mode. In this embodiment, the threshold is 0.15s⁻¹. This threshold corresponds to the typical rate of change in audience activity from quiet to sudden applause, as measured in multiple performances. The 85th quantile of the rate of change distribution was determined.

[0093] It is a first-order high-pass filter with a cutoff frequency of 0.1Hz, used to extract the rapidly changing components of the active factor.

[0094] Example: If If the voltage jumps from 0.5 to 0.7 within 0.2 seconds, the Qualcomm output is approximately 0.2. , dB (as additional gain in the infrasound envelope).

[0095] 4. Vector composition and output

[0096] The final output device adjustment vector Synthesizing is performed according to the trigger priority of the above modes. Priority, from highest to lowest, is: Mode 3 (Rhythmic Interaction) > Mode 1 (Fatigue Cancellation) > Mode 2 (Immersion Enhancement). Only one mode is the primary mode at any given time, but Mode 3 can be superimposed on the low-frequency components of Mode 1 or Mode 2. The synthesized mode... Transmitted to the audio digital signal processor (DSP) and lighting control panel at a frequency of at least 10Hz via IoT communication protocols (such as MQTT). For example, the audio DSP... Adjust the gain coefficient of the IIR filter in the infrasound channel, and the lighting control panel will adjust accordingly. Rewrite the PWM register value of the LED driver, and according to Real-time modulation of stroboscopic depth.

[0097] VI. Video Data Privacy Protection

[0098] The video capture device only captures the outline and skeletal information of the audience in the seating area, excluding facial features that can identify individuals. The captured video data is processed in real-time on an edge computing node (such as an NVIDIA Jetson TX2), discarding the original images and retaining only the crowd activity factor. The time-series data is uploaded to the IoT cloud platform (claim 8). The edge computing node uploads only about 3.6MB of data per hour (1 floating-point number per second, 8 bytes of double precision, 3600 seconds), greatly reducing bandwidth and privacy risks.

[0099] VII. Feedforward Calibration

[0100] Before the performance, a frequency sweep test signal covering a preset frequency range (e.g., a linear sweep from 0.5Hz to 20Hz) is released to the audience. Simultaneously, the micro-tremor response of the audience's heads (rapid, minute vibrations of the head less than 0.5cm in the vertical direction) is recorded using video capture equipment. The infrasound sensitivity coefficient of each seating area is calculated, and this coefficient is used as a weighting factor for the area. Substitute the state base value In the calculation. For example: the sensitivity coefficient for the front row is 1.2, for the middle row 1.0, and for the back row 0.8, then in the calculation... When Replace with This enables differentiated regulation based on regional differences.

[0101] VIII. Online Learning Mechanism of Fuzzy Neural Coupled Networks

[0102] The fuzzy neural coupling network employs an online learning mechanism, based on historical data... , The parameters are dynamically adjusted based on their correlation with subsequent audience departure behavior (e.g., the percentage of attendees leaving in the last 10 minutes of the performance). , , , To optimize fatigue prediction accuracy for different performance types. Specific implementation: After each performance, the percentage of audience members leaving within the last 20 minutes is calculated. If the departure percentage is higher than 15%, the current parameter combination is deemed unsuitable. The four parameters are then adjusted using a particle swarm optimization algorithm, with a step size limited to ±10%. The updated parameters are stored in the theater parameter library for use in the next similar performance (e.g., symphony orchestras, plays, and rock concerts each maintain independent parameter sets).

[0103] This invention deploys an IoT sensing layer to collect infrasound frequency bands, optical flicker indexes, and audience video images in real time. It constructs an infrasound-optical coupled fatigue accumulation model to calculate the state baseline value and extracts features such as head deflection and shoulder swaying to calculate crowd activity factors. Then, it uses a fuzzy neural coupling network to output a comprehensive sensory state index. The deviation between this index and the expected target value in the performance script is then input into an adaptive compensation controller to generate a device adjustment vector that simultaneously acts on the infrasound frequency band dynamic notch filter of the sound equipment and the PWM dimming frequency generator of the lighting equipment. This achieves intelligent adjustment of three modes: fatigue cancellation, immersion enhancement, and closed-loop rhythmic interaction. This effectively solves the technical problems in existing technologies, such as the inability to jointly perceive infrasound and optical flicker coupled fatigue, the neglect of the fatigue cancellation effect of crowd activity behavior, and the lack of closed-loop adaptive compensation based on real-time audience status and the expected target of the performance script.

[0104] The steps of the methods or algorithms described in this application can be directly embedded in hardware, software units executed by a processor, or a combination of both. Exemplarily, a storage medium can be connected to a processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. Optionally, the processor and the storage medium can also be located in different components within a terminal. These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely illustrative examples of this application and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Thus, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A smart management method for theater lighting, sound effects, and video based on the Internet of Things, characterized in that: An Internet of Things (IoT) sensing layer is deployed in the theater's audience seating area. The sensing layer includes: a long-time integral microphone array for collecting infrasound frequency band data from the sound equipment; a photoelectric sensor array for collecting the optical flicker index of the lighting equipment; and a video acquisition device deployed above the audience seating area. The sound equipment is equipped with a real-time adjustable infrasound frequency band filter module, and the lighting equipment is equipped with a programmable PWM dimming frequency generator. Based on the time accumulation of the infrasound frequency band and the time change rate of the optical flicker index, a fatigue accumulation model of audience infrasound-optical coupling is constructed, and the state base value is calculated. ; The video capture device acquires real-time video images of the audience seating area, extracts features from human postures and head movements in the images, and calculates the crowd activity factor. ; The state base value With crowd activity factor The input is fed into a fuzzy neural coupling network to construct a fatigue-activity nonlinear mapping model, and outputs a comprehensive sensory state index. ; Obtain the expected sensory impact sequence in the current performance script. The expected sensory impact sequence is a pre-set sequence of target values ​​for the comprehensive sensory state index that the audience is expected to achieve at different stages of the performance, based on the timeline of the performance script. The comprehensive sensory state index calculated in real time The expected target value corresponding to the time in the expected sensory impact sequence The difference between the two values ​​is used as a control deviation input to the adaptive compensation controller, which generates a device adjustment vector based on the control deviation. The expected sensory impact sequence is a pre-set sequence of comprehensive sensory state target values ​​that the audience should achieve at different stages of the performance, based on the timeline of the performance script. The current performance script is a pre-stored structured data file that describes the timeline of the performance content. The device adjustment vector The parameter settings are applied synchronously to the subsonic frequency band dynamic notch filter of the audio equipment and the PWM dimming frequency generator of the lighting equipment.

2. The method for intelligent management of theater lighting, sound effects, and video based on the Internet of Things according to claim 1, characterized in that: The calculation of the state base value The formula: ； in, For a moment The infrasound band weighted sound pressure level; Lighting equipment at all times The PWM dimming frequency; Infrasound cumulative gain coefficient; The preset fatigue time decay constant; is the optical stroboscopic coupling coefficient.

3. The intelligent management method for theater lighting, sound effects, and video based on the Internet of Things as described in claim 1, characterized in that: The calculation of crowd activity factor : ； in, The number of viewers that the video capture device can identify; For the first The head tilt angle of an audience member refers to the horizontal tilt angle of the head relative to the direction facing the stage. This is the normalization factor for head deflection angle; For the first The range of shoulder movement of each audience member; This is the normalization factor for shoulder swing amplitude; For the first The frequency of head movements of the audience members; This is a normalization factor for the head oscillation frequency. , , This is a preset weighting factor.

4. The intelligent management method for theater lighting, sound effects, and video based on the Internet of Things as described in claim 1, characterized in that: The specific formula for constructing the fatigue-activity nonlinear mapping model is as follows: ； in, It is a comprehensive sensory state index; , , , For preset adjustable parameters; in the formula, Positive contribution ,and and its rate of change The generation of negative inhibition means that the active behavior of the audience can serve as a counteracting signal of physiological fatigue.

5. The intelligent management method for theater lighting, sound effects, and video based on the Internet of Things according to claim 4, characterized in that: The control objective of the adaptive compensation controller is to make... Approaching the preset target value The The performance changes dynamically according to the timeline of the script: It is pre-set to have a calm phase and a climactic interactive phase. During the calm phase... Set in the preset low value range; During the climax of the interaction Set in the preset high value range.

6. The intelligent management method for theater lighting, sound effects, and video based on the Internet of Things according to claim 1, characterized in that: The generation device adjustment vector Includes the following modes: when Higher than the upper limit of the preset target value and When the activity level is below the preset activity threshold, the adjustment vector includes an infrasound phase cancellation instruction and a PWM frequency boost instruction. when When the value is below the preset target lower bound and the performance is at its climax, the adjustment vector includes an infrasound gain boost instruction and a PWM frequency synchronization modulation instruction. when rate of change When the preset positive threshold is exceeded, the adjustment vector triggers the rapid response of the light flashing and the low-frequency sound, amplifying the collective active behavior of the audience.

7. The intelligent management method for theater lighting, sound effects, and video based on the Internet of Things as described in claim 6, characterized in that: The audience response follow mode in the aforementioned mode specifically refers to: [the following text appears to be incomplete and likely refers to a specific method or mechanism:] [The following text appears to be incomplete and likely refers to a specific method or mechanism:] [The following text appears to be incomplete and likely refers to a specific method or mechanism:] [The audience response follow mode specifically ... The real-time value is high-pass filtered and mapped to the strobe depth modulation signal of the lighting control channel and the infrasound envelope follow signal of the sound control channel, forming a closed-loop rhythmic interaction from the audience's rhythm to the infrasound response of the lights.

8. The intelligent management method for theater lighting, sound effects, and video based on the Internet of Things according to claim 1, characterized in that: The video capture device only captures the outline and skeletal information of the audience in the seating area, and does not capture facial features that can identify individuals. The captured video data is processed in real time at the edge computing node, and the original image is discarded, retaining only the crowd activity factor. The time series data is uploaded to the IoT cloud platform.

9. The intelligent management method for theater lighting, sound effects, and video based on the Internet of Things according to claim 1, characterized in that: It also includes feedforward calibration: before the performance, a frequency sweep test signal covering a preset frequency is released to the audience seats, and the micro-tremor response of the audience's heads is recorded synchronously through video acquisition equipment. The infrasound sensitivity coefficient of each seat area is calculated, and this coefficient is used as the area weighting factor. Substitute the state base value In the calculation.

10. A method for intelligent management of theater lighting, sound effects, and video based on the Internet of Things according to claim 1, characterized in that: The fuzzy neural coupling network employs an online learning mechanism, based on historical data... , Parameters are dynamically adjusted based on their correlation with subsequent audience departure behavior. , , , To optimize the accuracy of fatigue prediction under different performance types.

Images