Stage lighting control method, device and storage medium based on somatosensory interaction
By analyzing stage and audience video data in real time, and generating regulation instructions to adjust lighting effects, the problem of lack of real-time interaction in the existing technology of lighting control is solved, and emotionally synchronized lighting control is realized, which improves the immersion and ornamentality of the stage effect. .
Patent Information
- Application Number
- CN202410936371.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-07-12
AI Technical Summary
The existing stage lighting control methods lack real-time interaction and cannot adjust the lighting effects based on real-time feedback from actors or audiences, resulting in insufficient immersion in the stage effect.
Using somatosensory interaction-based method, the server is connected to the stage and audience video sensors, and the video and music data are obtained in real time. The neural network and emotion recognition algorithm are used to analyze the emotions of actors and audiences, and regulatory instructions are generated to adjust the brightness and color of the light to realize emotions. Synchronous lighting control.
It realizes accurate capture of the emotions of stage actors and audiences, provides real-time interactive and coherent lighting control, enhances the artistry and viewing of stage performances, and enhances the audience's audio-visual experience.
Smart Images

Figure CN118474958B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of lighting control, and in particular to a stage lighting control method, device and storable medium based on somatosensory interaction. Background Art
[0002] With the continuous development and improvement of modern stage performances, the audience's requirements for performance effects are getting higher and higher. They hope to see more precise and artistic performances. As an important part of stage technology, lighting control has a great impact on stage effects, aiming to enhance the artistry, appreciation and interactivity of the performance. Therefore, stage lighting control is very necessary.
[0003] At present, stage lighting control is usually a preset plan designed in advance by lighting designers. This fixed control method is single and lacks real-time interactivity, which means that adjustments cannot be made according to real-time feedback from actors or audiences during the performance, resulting in insufficient immersion in the stage effects presented. Summary of the invention
[0004] The present invention provides a stage lighting control method, device and storable medium based on somatosensory interaction, which are used to solve the above-mentioned technical problems.
[0005] A first aspect of the present invention provides a stage lighting control method based on somatosensory interaction, comprising the following steps:
[0006] T1: The server communicates with various sensors on the stage to obtain real-time stage video and audience video; in addition, background music is pre-stored in the server;
[0007] T2: Perform rhythm analysis based on background music to obtain the rhythm value corresponding to each music clip; at the same time, analyze the emotions of the actors and the audience based on the dance video and the audience video to obtain the performance emotion value and the audience emotion value corresponding to each acquisition moment; the performance emotion value VDj, the audience emotion value SBj and the background emotion value ZPj are calculated by the set formula Calculate and obtain the emotion evaluation index VSj at each collection moment, where d4, d5, and d6 are respectively set proportional coefficients, where j=1, 2, 3...J, J is a positive integer, J represents the total number of collection moments, and j represents the sequence number of any collection moment; then perform emotion fluctuation analysis based on the emotion evaluation index at each collection moment to generate a control instruction, and send the control instruction to T3;
[0008] T3: Control the lighting according to the received control command. The specific control steps are as follows:
[0009] Step 1: Analyze the control instructions to obtain the emotional fluctuation index; use the emotion recognition algorithm to perform emotion recognition on the video and background music to obtain the emotion type corresponding to the current moment and the emotion intensity corresponding to the emotion type;
[0010] Step 2: Compare the emotion type with all the set emotion types to match the corresponding light brightness range and record it as the target light brightness range; multiply the emotion intensity by the set brightness conversion coefficient to obtain the target light brightness;
[0011] Step 3: Compare the emotion type with all the set emotion types to match the corresponding light color set; set a color value for each color in the light color set, multiply the emotion intensity by the set color conversion coefficient to get the target color value, and compare the target color value with the color values corresponding to each color in the light color set to match the target color; thus, the control parameters of this light control can be obtained, where the control parameters include the target light brightness, recorded as U1, the target color, and the color value corresponding to the target color, recorded as U2;
[0012] Step 4: Obtain the control parameters corresponding to the most recent control instruction at the current moment, and record them as U1' and U2' respectively; Calculate to obtain the brightness control speed H1 and the color control speed H2, where g1, g2, g3, and g4 are respectively the set proportional coefficients, and α and β are respectively the correction coefficients of the brightness control speed and the color control speed; perform formula calculation and analysis to obtain the brightness control speed and the color control speed, and according to the brightness control speed and the color control speed, transition the control parameters corresponding to the previous control instruction to the control parameters of this light control;
[0013] T4: After each control is completed, steps T2-T3 are repeated until no control instructions are generated.
[0014] Optionally, the specific process of rhythm analysis based on background music is as follows:
[0015] 201: Segment the background music into a number of music clips, and use a neural network model to identify each music clip to obtain the number of beats of each music clip and the intensity of each beat;
[0016] 202: Arrange the beats in order according to their order in the music clip, calculate the interval between two adjacent beats, and compare and analyze them to obtain the background emotion value; similarly, compare and analyze the beat intensity to obtain the intensity value;
[0017] 203: Substitute the rhythm value Za and the intensity value Pa into the set formula The background emotion value ZP of the music clip is obtained by calculation, wherein a7 and a8 are respectively set proportional coefficients.
[0018] Optionally, the specific analysis process for generating the background emotion value and the intensity value is:
[0019] Compare and analyze the interval duration with the set duration interval to divide the interval duration into fast rhythm, medium rhythm and slow rhythm; count the cumulative times of fast rhythm, medium rhythm and slow rhythm respectively, and record them as Z1, Z2, Z3 respectively; sum the interval durations corresponding to fast rhythm, medium rhythm and slow rhythm to obtain the total value of fast rhythm, the total value of medium rhythm and the total value of slow rhythm, and record them as Z4, Z5, Z6 respectively; substitute Z1, Z2, Z3, Z4, Z5, Z6 into the set formula Calculate and obtain the rhythm value Za, where a1, a2, and a3 are respectively the set proportional coefficients, and a1>a2>a3>0;
[0020] Similarly, the beat intensity is compared and analyzed with the set beat interval to divide the beat intensity into first-level intensity beat, second-level intensity beat and third-level intensity beat, and the cumulative number of first-level intensity beat, second-level intensity beat and third-level intensity beat are counted respectively, and recorded as P1, P2, and P3 respectively; the beat intensities corresponding to the first-level intensity beat, second-level intensity beat and third-level intensity beat are summed up to obtain the first-level intensity value, the second-level intensity value and the third-level intensity value, and recorded as P4, P5, and P6 respectively; P1, P2, P3, P4, P5, and P6 are substituted into the set formula The intensity value Pa is obtained by calculation, wherein a4, a5, and a6 are respectively set proportional coefficients, and a4>a5>a6>0.
[0021] Optionally, the specific process of motion analysis of dance videos is as follows:
[0022] The neural network model is used to identify dance movements in stage videos, and the skeleton information is extracted from each dance movement using computer vision technology and deep learning models. The skeleton information includes movement speed, step frequency, and step strength, which are recorded as V, L, and D respectively. The performance emotion value VD of each dance action is calculated, wherein b1, b2, and b3 are respectively set proportional coefficients; thereby, the dance action corresponding to each acquisition moment and the performance emotion value VDj corresponding to the dance action can be identified.
[0023] Optionally, the specific process of performing action analysis based on the audience video is as follows:
[0024] The neural network model and computer vision technology are used to identify the viewing actions of the audience video. Several preset actions are set, so that the actions of each audience in the audience seats at each acquisition time can be identified and compared with the preset actions. When the audience action is consistent with the preset action, the audience action is recorded as a valid action and the maintenance duration of the valid action is obtained; otherwise, it is recorded as an invalid action; each valid action is compared with all the preset actions to match the corresponding expression base value; the expression base value B1 and the corresponding maintenance duration B2 of the valid action are substituted into the set formula Calculate and obtain the expression value Bb of each action, where b4 and b5 are the set proportional coefficients respectively;
[0025] The number of audience members in the auditorium at the collection time is counted, and the number of valid actions corresponding to each audience member in the auditorium at the collection time is counted, thereby obtaining the expression value corresponding to each valid action; each expression value is compared and analyzed with the set value range to divide the expression value into high expression, medium expression and low expression; the number of high expression, medium expression and low expression is counted respectively, and the expression values corresponding to high expression, medium expression and low expression are summed up to obtain the total value of high expression, the total value of medium expression and the total value of low expression;
[0026] Substitute the number of viewers S1, the number of effective actions S2, the number of high expressions B1, the number of medium expressions B2, the number of low expressions B3, the total value of high expressions B4, the total value of medium expressions B5 and the total value of low expressions B6 into the set formula The audience emotion value SB corresponding to the acquisition moment is calculated, wherein d1, d2, and d3 are respectively set proportional coefficients, and d1>d2>d3>0; thus, the audience emotion value corresponding to each acquisition moment is obtained and recorded as SBj.
[0027] Optionally, the specific process of performing emotion fluctuation analysis based on the emotion evaluation index is as follows:
[0028] With time as the horizontal axis and the emotion evaluation index as the vertical axis, a curve relationship diagram of the emotion evaluation index changing with time is plotted; a tangent line is drawn at the evaluation point, and the tangent expression is obtained by data fitting. The evaluation derivative corresponding to each evaluation point is obtained by derivative calculation and recorded as Wj, where j=1,2,3...J, J is a positive integer, J represents the total number of acquisition moments, and j represents the sequence number of any acquisition moment; the evaluation derivatives greater than zero are summed up to obtain the upper emotional volatility and recorded as A1, and the evaluation derivatives less than zero are summed up and the absolute value is taken to calculate the lower emotional volatility and recorded as A2; using the set formula The emotional fluctuation index Af is calculated, where f1 and f2 are respectively set proportional coefficients; when the emotional fluctuation index is greater than the set fluctuation threshold, a control instruction is generated.
[0029] A second aspect of the present invention provides a stage lighting control device based on somatosensory interaction, the device comprising:
[0030] The server acquires the stage video and the audience video in real time by communicating with the sensors on the stage; in addition, the server stores background music in advance;
[0031] The somatosensory interaction module performs rhythm analysis based on the background music to obtain the background emotion value corresponding to each music clip; at the same time, the movements of the actors and the audience are analyzed based on the dance video and the audience video to obtain the performance emotion value and the audience emotion value corresponding to each acquisition moment; a comprehensive analysis is performed based on the background emotion value, the performance emotion value and the audience emotion value to obtain the emotion evaluation index at each acquisition moment, and the emotion fluctuation analysis is performed based on this to generate control instructions, and the control instructions are sent to the intelligent control module;
[0032] The intelligent control module controls the lighting according to the control instructions received, specifically:
[0033] Step 1: Analyze the control instructions to obtain the emotional fluctuation index; use the emotion recognition algorithm to perform emotion recognition on the video and background music to obtain the emotion type corresponding to the current moment and the emotion intensity corresponding to the emotion type;
[0034] Step 2: Compare the emotion type with all the set emotion types to match the corresponding light brightness range and record it as the target light brightness range; multiply the emotion intensity by the set brightness conversion coefficient to obtain the target light brightness;
[0035] Step 3: Compare the emotion type with all the set emotion types to match the corresponding light color set; set a color value for each color in the light color set, multiply the emotion intensity by the set color conversion coefficient to obtain the target color value, and compare the target color value with the color values corresponding to each color in the light color set to match the target color; thereby, the control parameters of this light control can be obtained, wherein the control parameters include the target light brightness, the target color, and the color value corresponding to the target color;
[0036] Step 4: Obtain the control parameters corresponding to the most recent control instruction at the current moment, and perform formulaic calculation and analysis on the control parameters of the previous light control and the mood fluctuation index to obtain the brightness control speed and the color control speed. Based on the brightness control speed and the color control speed, the control parameters corresponding to the previous control instruction are transferred to the control parameters of this light control.
[0037] Step 5: After each control is completed, the intelligent control module sends a re-analysis instruction to the body sensing interaction module for emotion fluctuation analysis until no control instruction is generated.
[0038] A third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned stage lighting control method based on somatosensory interaction.
[0039] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:
[0040] 1. Through the comprehensive use of music analysis, video analysis and audience feedback analysis, it can capture the performance emotions of stage actors and the feedback emotions of the audience in real time, and generate an emotional evaluation index based on this; then establish a curve relationship diagram of the evaluation index over time, and analyze the dynamic changes of emotions based on this to generate control instructions; it realizes the accurate capture of the performance emotions of stage actors and the feedback emotions of the audience, which helps to better reflect the emotional experience of stage performances and audience interactions, and provides effective technical support for the intelligent control of stage lighting;
[0041] 2. The emotional state of the audience is analyzed in real time through the emotional fluctuation index and emotion recognition algorithm, so as to adjust the lighting effect according to the changes in the audience's emotions, so that the stage lighting can be consistent with the emotional state of the audience; at the same time, the brightness control speed and color control speed are used to achieve a smooth transition of lighting parameters, avoiding the discomfort caused by sudden lighting changes to the audience and maintaining the continuity of the stage atmosphere; in addition, by continuously sending re-analysis instructions to the somatosensory interaction module for emotional fluctuation analysis, the intelligent control module can continuously make adjustments according to the emotional feedback of the audience and actors during the performance, realizing the real-time interactivity of stage lighting control;
[0042] In summary, based on the intelligent control method of somatosensory interaction, stage lighting can be more closely integrated with the dancers' performances and the audience's feedback, bringing the audience a more shocking and emotionally resonant audio-visual experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. The following drawings are not intentionally scaled to the actual size, and the focus is on illustrating the main purpose of the present application.
[0044] Figure 1 It is a principle block diagram of the present invention;
[0045] Figure 2 It is a structural schematic diagram of the stage lighting control device of the present invention. DETAILED DESCRIPTION
[0046] Embodiments of the present invention provide an energy scheduling management method, apparatus, device and storage medium. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0047] It should be noted that the relevant information (including but not limited to stage videos, audience videos and background music, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.
[0048] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1-2 In the embodiment of the present invention, the stage lighting control method based on somatosensory interaction is implemented by applying to a stage lighting control device, wherein the stage lighting control device includes a server 111, a somatosensory interaction module 112 and an intelligent control module 113; a stage play performed by dancers is used as a specific application scenario for detailed description, and the specific method includes the following steps:
[0049] T1: The server 111 acquires the stage video and the audience video in real time by communicating with the sensors on the stage; the stage video refers to the stage performance video, and the audience video refers to the viewing video of the audience; in addition, the server stores background music in advance;
[0050] T2: The somatosensory interaction module 112 analyzes the rhythm of the background music to obtain the corresponding music emotion changes, and analyzes the movements based on the stage video and the audience video to identify the performance and the audience emotion changes, and generates control instructions; specifically:
[0051] The rhythm analysis of background music is as follows:
[0052] The background music is divided into several music clips, and each music clip is identified by a neural network model to obtain the beat number of each music clip (the beat number here is BPM, and a fast BPM usually indicates active and excited emotions, while a slower BPM means calm and gentle emotions) and the intensity of each beat (the beat intensity here refers to the relative intensity or importance of each beat in the music, usually used to describe the relationship between the volume or intensity of each beat and the overall music rhythm). Randomly select one of the music clips, sort the beats in the order of their order in the music clip, and calculate the interval between two adjacent beats. It should be noted that the larger the interval between the beats, the stronger the intensity. The slower the rhythm, the calmer and gentler the mood; compare and analyze the interval duration with the set duration interval. When the interval duration is greater than the maximum value in the set duration interval, a slow rhythm is accumulated; when the interval duration is within the set interval duration interval, a medium rhythm is accumulated; when the interval duration is less than the minimum value in the set duration interval, a fast rhythm is accumulated; count the cumulative times of fast rhythm, medium rhythm and slow rhythm respectively, and record them as Z1, Z2, and Z3 respectively; sum the interval durations corresponding to fast rhythm, medium rhythm and slow rhythm to obtain the total value of fast rhythm, the total value of medium rhythm and the total value of slow rhythm, and record them as Z4, Z5, and Z6 respectively; use the set formula Calculate and obtain the rhythm value Za, where a1, a2, and a3 are respectively the set proportional coefficients, and a1>a2>a3>0;
[0053] Compare and analyze the beat intensity with the set beat interval. When the beat intensity is greater than the maximum value in the set intensity interval, a first-level intensity beat is accumulated; when the beat intensity is within the set intensity interval, a second-level intensity beat is accumulated; when the beat intensity is less than the minimum value in the set intensity interval, a third-level intensity beat is accumulated; the cumulative number of first-level intensity beats, second-level intensity beats and third-level intensity beats are counted respectively, and recorded as P1, P2, and P3 respectively; the beat intensities corresponding to the first-level intensity beats, second-level intensity beats and third-level intensity beats are summed to obtain the first-level intensity value, the second-level intensity value and the third-level intensity value, and recorded as P4, P5, and P6 respectively; use the set formula The intensity value Pa is calculated, where a4, a5, and a6 are respectively set proportional coefficients, and a4>a5>a6>0;
[0054] The rhythm value Za and the intensity value Pa are calculated by the formula The background emotion value ZP of the music clip is calculated, where a7 and a8 are respectively the set proportional coefficients. It can be seen from the formula that when the rhythm value is larger, it means a more excited emotion, and the background emotion value is larger; when the intensity value is larger, it means a more excited emotion; thus, the background emotion value corresponding to each music clip in the background music can be obtained;
[0055] The real-time action analysis of the stage video is as follows:
[0056] A neural network model is used to identify dance movements in stage videos (it should be noted that the dance movements here are continuous, dynamic sequences of movements within a very short period of time, rather than static postures). It should be noted that the neural network model learns to identify different dance movements by training dance videos. Each dance movement is encoded as a feature vector, and the neural network outputs the probability distribution of these feature vectors, indicating the possibility of each dance movement appearing in the video, thereby judging and identifying the dance movements; computer vision technology and deep learning models are used to extract skeleton information from each dance movement, where the skeleton information includes movement speed, step frequency and step strength, and they are recorded as V, L and D respectively; using the set formula The performance emotion value VD of each dance action is calculated, where b1, b2, and b3 are respectively the set proportional coefficients; it can be known from the formula that when the action speed is faster, the step frequency is higher, and the dance step strength is greater, it means that the dancer's emotions are stronger at this time, and the need for lighting changes is greater at this time; thus, the dance action corresponding to each collection moment and the performance emotion value corresponding to the dance action can be identified and recorded as VDj, where j=1,2,3...J, J is a positive integer, J represents the total number of collection moments, and j represents the sequence number of any collection moment;
[0057] The real-time action analysis of audience video is as follows:
[0058] The neural network model and computer vision technology are used to identify the viewing actions of the audience video. Similarly, the audience action here is a continuous and dynamic action sequence in a very short period of time, rather than a static posture; there are several preset actions, which specifically include clapping, crying, cheering, smiling, etc.; thereby, the actions of each audience in the auditorium at each collection moment can be identified and compared with the preset actions. When the audience action is consistent with the preset action, the audience action is recorded as a valid action, and the maintenance duration of the valid action is obtained; otherwise it is recorded as an invalid action; it should be noted that the valid action and the maintenance duration corresponding to each collection moment here refer to the duration of the effective action from the previous collection moment to the current collection moment; for example, the system is set to collect once every 3 minutes, and the interval between the N-1th collection moment and the Nth collection moment is 3 minutes. The duration of the effective action within the 3 minutes represents the effective action corresponding to the Nth collection moment and the maintenance duration of the effective action;
[0059] Each preset action is set to correspond to an expression base value. It should be noted that different preset actions correspond to different expression values. The expression value represents the magnitude of the audience's emotional expression. For example, the emotion expressed by tears is deeper than that expressed by smiles, so the corresponding expression value is larger. The emotion expressed by cheers is stronger than that expressed by applause. Each valid action is compared with all preset actions to match the corresponding expression base value, which is recorded as B1. The expression base value B1 of the valid action and the corresponding maintenance duration B2 are calculated using the set formula Calculate and obtain the expression value Bb of each action, where b4 and b5 are the set proportional coefficients respectively; take any collection time, count the number of spectators in the auditorium at the collection time and record it as S1; and count the number of valid actions corresponding to each spectator in the auditorium at the collection time S2, thereby obtaining the expression value corresponding to each valid action; compare and analyze each expression value with the set value interval, when the expression value is greater than the maximum value in the set value interval, a high expression is accumulated; when the expression value is within the set value interval, a medium expression is accumulated; when the expression value is less than the minimum value in the set value interval, a low expression is accumulated; count the number of high expression, medium expression and low expression respectively, and record them as B3, B4, B5 respectively; sum the expression values corresponding to high expression, medium expression and low expression respectively to obtain the total value of high expression, the total value of medium expression and the total value of low expression, and record them as B6, B7, B8 respectively; use the set formula Calculate the audience emotion value SB corresponding to the acquisition moment, where d1, d2, and d3 are respectively set proportional coefficients, and d1>d2>d3>0; thus, the audience emotion value corresponding to each acquisition moment can be obtained and recorded as SBj;
[0060] Identify the background music at each acquisition moment and determine the music segment to which it belongs, so that the background emotion value corresponding to each acquisition moment can be obtained and recorded as ZPj; the performance emotion value VDj, the audience emotion value SBj and the background emotion value ZPj are calculated by the set formula Calculate and obtain the emotion evaluation index VSj at each collection moment, where d4, d5, and d6 are respectively the set proportional coefficients; construct a two-dimensional rectangular coordinate system with time as the horizontal coordinate and the emotion evaluation index as the vertical coordinate, input the emotion evaluation index into the coordinate system according to its corresponding collection moment, and record the position of the emotion evaluation index in the coordinate system as the evaluation point, and use a smooth curve to connect them in sequence to obtain a curve relationship diagram of the emotion evaluation index changing with time; draw a tangent line at the evaluation point, use data fitting to obtain a tangent expression, and calculate the derivative of the tangent expression to obtain the evaluation derivative corresponding to each evaluation point, which is recorded as Wj. It should be noted that the evaluation derivative refers to the change trend of the emotion evaluation index at the evaluation point. When the evaluation derivative is greater than zero, it means that the emotion evaluation index at the evaluation point shows an upward trend; when the evaluation derivative is less than zero, it means that the emotion evaluation index at the evaluation point shows a downward trend; if the emotion evaluation index changes significantly, it means that the stage lighting needs to be adjusted;
[0061] The evaluation derivatives greater than zero are summed up to get the upper emotional wave, recorded as A1, and the evaluation derivatives less than zero are summed up and taken to get the lower emotional wave, recorded as A2; using the set formula The emotional fluctuation index Af is calculated, where f1 and f2 are respectively set proportional coefficients; it can be seen from the formula that the greater the emotional fluctuation, the greater the possibility of needing lighting control; when the emotional fluctuation index is greater than the set fluctuation threshold, a control instruction is generated to the intelligent control module 113;
[0062] By comprehensively utilizing music analysis, video analysis, and audience feedback analysis, it is possible to capture the performance emotions of stage actors and the audience's feedback in real time, and generate an emotional evaluation index based on this; then establish a curve relationship graph of the evaluation index changing over time, and analyze the dynamic changes of emotions based on this to generate control instructions; it achieves accurate capture of the performance emotions of stage actors and the audience's feedback emotions, helps to better reflect the emotional experience of stage performances and audience interaction, and provides effective technical support for the intelligent control of stage lighting.
[0063] T3: The intelligent control module 113 controls the lighting according to the received control instruction, and the control steps are as follows:
[0064] Step 1: Analyze the control instructions to obtain the emotion fluctuation index Af; use the existing emotion recognition algorithm to perform emotion recognition on the video and background music to obtain the emotion type corresponding to the current moment and the emotion intensity corresponding to the emotion type; the specific emotion types include happiness, sadness, tension, etc.;
[0065] Step 2: Set a light brightness range corresponding to each emotion type. Specifically, different emotion types correspond to different light brightness ranges. For example, the light brightness range corresponding to a happy emotion is greater than the light brightness range corresponding to a sad emotion. Compare the emotion type with all the set emotion types to match the corresponding light brightness range as the target light brightness range. Multiply the emotion intensity by the set brightness conversion coefficient to obtain the target light brightness.
[0066] Step 3: Set each emotion type to correspond to a light color set. Specifically, different emotion types correspond to different light color sets. For example, the light color set corresponding to happy emotion is warm, and the light color set corresponding to sad emotion is cold; the light color set corresponding to nervous emotion is dark, such as dark red or dark purple; compare the emotion type with all the set emotion types to match the corresponding light color set; set each color in the light color set to correspond to a color value, multiply the emotion intensity by the set color conversion coefficient to obtain the target color value, and compare the target color value with the color value corresponding to each color in the light color set to match the target color; thus, the control parameters of this light control can be obtained, wherein the control parameters include the target light brightness, the target color, and the color value corresponding to the target color, and they are respectively recorded as U1 (target light brightness) and U2 (color value corresponding to the target color);
[0067] Step 4: Get the control parameters corresponding to the most recent control instruction at the current moment, and record them as U1' and U2' respectively. The brightness control speed H1 and the color control speed H2 are calculated, where g1, g2, g3, and g4 are the set proportional coefficients, and α and β are the correction coefficients of the brightness control speed and the color control speed. It can be seen from the formula that the larger the emotional fluctuation value, the more obvious the emotional change, and the corresponding brightness control speed and color control speed are larger.
[0068] According to the brightness control speed and the color control speed, the control parameters corresponding to the previous control instruction are transferred to the control parameters of this lighting control; specifically, the target light brightness corresponding to the previous control instruction is transferred to the target light brightness of this control according to the brightness control speed H1, and at the same time, the target light color corresponding to the previous control instruction is transferred to the target light color of this control according to the color control speed H2, so as to avoid sudden changes causing discomfort to the audience, making the lighting control smoother and more natural, helping to maintain the continuity of the stage atmosphere, and improving the audience's comfort and viewing experience;
[0069] The emotional state of the audience is analyzed in real time through the emotion fluctuation index and emotion recognition algorithm, so that the lighting effects can be adjusted according to the changes in the audience's emotions, so that the stage lighting can be consistent with the audience's emotional state; at the same time, the brightness control speed and color control speed are used to achieve a smooth transition of lighting parameters, avoiding the discomfort caused to the audience by sudden lighting changes and maintaining the continuity of the stage atmosphere.
[0070] T4: After each control is completed, the intelligent control module 112 sends a re-analysis instruction to the body sensing interaction module 112 to perform emotion fluctuation analysis until no control instruction is generated (i.e., repeating steps T2-T3);
[0071] By continuously sending re-analysis instructions to the somatosensory interaction module to analyze emotional fluctuations, the intelligent control module can continuously make adjustments based on the emotional feedback of the audience and actors during the performance, realizing real-time interactivity in stage lighting control.
[0072] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the stage lighting control method based on somatosensory interaction.
[0073] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0074] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0075] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A stage lighting control method based on somatosensory interaction, characterized in that: The following steps are involved: T1: Real-time acquisition of stage video and audience video through communication connection with various sensors on the stage; in addition, background music is pre-stored; T2: Perform rhythm analysis based on background music to obtain the background emotion value corresponding to each music clip; at the same time, analyze the movements of the actors and the audience based on the dance video and the audience video to obtain the performance emotion value and the audience emotion value corresponding to each acquisition moment; Comprehensively analyze the background emotion value, performance emotion value and audience emotion value to obtain the emotion evaluation index at each acquisition moment, and analyze the emotion fluctuation based on it to generate control instructions, and send the control instructions to T3; The specific process of emotional fluctuation analysis based on the emotional evaluation index is as follows: With time as the horizontal axis and the emotion evaluation index as the vertical axis, a curve relationship diagram of the emotion evaluation index changing with time is plotted; a tangent line is drawn at the evaluation point, and the tangent expression is obtained by data fitting. The evaluation derivative corresponding to each evaluation point is obtained by derivative calculation and recorded as Wj, where j = 1, 2, 3...J, J is a positive integer, J represents the total number of acquisition moments, and j represents the sequence number of any acquisition moment; the evaluation derivatives greater than zero are summed up to obtain the upper emotional volatility and recorded as A1, and the evaluation derivatives less than zero are summed up and the absolute value is taken to calculate the lower emotional volatility and recorded as A2; using the set formula The emotional fluctuation index Af is calculated, where f1 and f2 are respectively the set proportional coefficients; when the emotional fluctuation index is greater than the set fluctuation threshold, a control instruction is generated; T3: Control the lighting according to the received control command. The specific control steps are as follows: Step 1: Analyze the control instructions to obtain the emotional fluctuation index; use the emotion recognition algorithm to perform emotion recognition on the video and background music to obtain the emotion type corresponding to the current moment and the emotion intensity corresponding to the emotion type; Step 2: Compare the emotion type with all set emotion types to match the corresponding light brightness range as the target light brightness range; Multiply the emotion intensity by the set brightness conversion coefficient to get the target light brightness; Step 3: Compare the emotion type with all set emotion types to match the corresponding light color set; Set a color value corresponding to each color in the light color set, multiply the emotion intensity by the set color conversion coefficient to obtain the target color value, and compare the target color value with the color value corresponding to each color in the light color set to match the target color; thereby, the control parameters of this light control can be obtained, wherein the control parameters include the target light brightness, the target color, and the color value corresponding to the target color; Step 4: Obtain the control parameters corresponding to the most recent control instruction at the current moment, and perform formulaic calculation and analysis on the control parameters of the previous light control and the mood fluctuation index to obtain the brightness control speed and the color control speed. Based on the brightness control speed and the color control speed, the control parameters corresponding to the previous control instruction are transferred to the control parameters of this light control. T4: After each control is completed, steps T2-T3 are repeated until no control instructions are generated.
2. The stage lighting control method based on somatosensory interaction according to claim 1 is characterized in that: The specific process of rhythm analysis based on background music is as follows: 201: Segment the background music into a number of music clips, and use a neural network model to identify each music clip to obtain the number of beats of each music clip and the intensity of each beat; 202: Arrange the beats in order according to their order in the music clip, calculate the interval between two adjacent beats, and compare and analyze them to obtain the background emotion value; similarly, compare and analyze the beat intensity to obtain the intensity value; 203: Perform weighted calculation and analysis on the rhythm value and the intensity value to obtain the background emotion value of each music clip.
3. The stage lighting control method based on somatosensory interaction according to claim 2 is characterized in that: The specific analysis process for generating background emotion value and intensity value is as follows: Compare and analyze the interval duration with the set duration interval to divide the interval duration into fast rhythm, medium rhythm and slow rhythm; count the cumulative times of fast rhythm, medium rhythm and slow rhythm respectively, and sum up the interval durations corresponding to the fast rhythm, medium rhythm and slow rhythm to obtain the total value of fast rhythm, the total value of medium rhythm and the total value of slow rhythm; perform formulaic calculation and analysis on the cumulative times of fast rhythm, the cumulative times of medium rhythm, the cumulative times of slow rhythm, the total value of fast rhythm, the total value of medium rhythm and the total value of slow rhythm to obtain the rhythm value; Similarly, the beat intensity is compared and analyzed with the set beat interval to divide the beat intensity into first-level intensity beats, second-level intensity beats and third-level intensity beats, and the cumulative number of first-level intensity beats, second-level intensity beats and third-level intensity beats are counted respectively, and the beat intensities corresponding to the first-level intensity beats, second-level intensity beats and third-level intensity beats are summed up to obtain the first-level intensity value, the second-level intensity value and the third-level intensity value. The cumulative number of first-level intensity beats, the cumulative number of second-level intensity beats, the cumulative number of third-level intensity beats, the first-level intensity value, the second-level intensity value and the third-level intensity value are formulated and analyzed to obtain the intensity value.
4. The stage lighting control method based on somatosensory interaction according to claim 1 is characterized in that: The specific process of motion analysis of dance videos is as follows: A neural network model is used to identify dance movements in stage videos, and computer vision technology and deep learning models are used to extract skeleton information from each dance movement, where the skeleton information includes movement speed, step frequency, and step strength. The skeleton information is then calculated and analyzed to obtain the performance emotion value of each dance movement. In this way, the dance movements corresponding to each acquisition moment and the performance emotion values corresponding to the dance movements can be identified.
5. The stage lighting control method based on somatosensory interaction according to claim 1 is characterized in that: The specific process of action analysis based on audience video is as follows: A neural network model and computer vision technology are used to identify the viewing actions of audience videos. A number of preset actions are set, so that the actions of each audience member in the auditorium at each acquisition moment can be identified and compared with the preset actions. When the audience action is consistent with the preset action, the audience action is recorded as a valid action, and the maintenance time of the valid action is obtained; otherwise, it is recorded as an invalid action; each valid action is compared with all preset actions to match the corresponding expression base value, and a weighted calculation and analysis is performed on it and the corresponding maintenance time to obtain the expression value of each viewing action; any acquisition moment is selected, the number of audience members in the auditorium at the acquisition moment is counted, and the number of valid actions corresponding to each audience member in the auditorium at the acquisition moment is counted, thereby obtaining the expression value corresponding to each valid action; each expression value is compared and analyzed with the set value range to divide the expression value into high expression, medium expression and low expression; The number of highly expressed, moderately expressed and lowly expressed genes was counted respectively, and the corresponding expression values of highly expressed, moderately expressed and lowly expressed genes were summed up to obtain the total value of highly expressed, the total value of moderately expressed and the total value of lowly expressed genes; The number of audience members, the number of effective actions, the number of high expressions, the number of medium expressions, the number of low expressions, the total value of high expressions, the total value of medium expressions and the total value of low expressions are formulated and analyzed to obtain the audience emotion value; thereby, the audience emotion value corresponding to each collection moment can be obtained.
6. Stage lighting control equipment based on somatosensory interaction, characterized in that: Applied to the stage lighting control method based on somatosensory interaction as claimed in any one of claims 1 to 5, the device comprises: The server acquires the stage video and the audience video in real time by communicating with the sensors on the stage; in addition, the server stores background music in advance; The somatosensory interaction module performs rhythm analysis based on the background music to obtain the background emotion value corresponding to each music clip; at the same time, the movements of the actors and the audience are analyzed based on the dance video and the audience video to obtain the performance emotion value and the audience emotion value corresponding to each acquisition moment; a comprehensive analysis is performed based on the background emotion value, the performance emotion value and the audience emotion value to obtain the emotion evaluation index at each acquisition moment, and the emotion fluctuation analysis is performed based on this to generate control instructions, and the control instructions are sent to the intelligent control module; The intelligent control module controls the lighting according to the control instructions received, specifically: Step 1: Analyze the control instructions to obtain the emotional fluctuation index; use the emotion recognition algorithm to perform emotion recognition on the video and background music to obtain the emotion type corresponding to the current moment and the emotion intensity corresponding to the emotion type; Step 2: Compare the emotion type with all the set emotion types to match the corresponding light brightness range as the target light brightness range; multiply the emotion intensity by the set brightness conversion coefficient to obtain the target light brightness; Step 3: Compare the emotion type with all the set emotion types to match the corresponding light color set; set a color value for each color in the light color set, multiply the emotion intensity by the set color conversion coefficient to obtain the target color value, and compare the target color value with the color values corresponding to each color in the light color set to match the target color; thereby, the control parameters of this light control can be obtained, wherein the control parameters include the target light brightness, the target color, and the color value corresponding to the target color; Step 4: Obtain the control parameters corresponding to the most recent control instruction at the current moment, and perform formulaic calculation and analysis on the control parameters of the previous light control and the mood fluctuation index to obtain the brightness control speed and the color control speed. Based on the brightness control speed and the color control speed, the control parameters corresponding to the previous control instruction are transferred to the control parameters of this light control. Step 5: After each control is completed, the intelligent control module sends a re-analysis instruction to the body sensing interaction module for emotion fluctuation analysis until no control instruction is generated.
7. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the stage lighting control method based on somatosensory interaction as described in any one of claims 1 to 5 is implemented.
