A method, device and medium for realizing intelligent stage performance

Abstract

The present application relates to the field of stage lighting, and specifically discloses a method, device and medium for realizing intelligent stage performance, which comprises the following steps: establishing a pose prediction model for predicting the pose data of performers after a first time length by using a target detection algorithm, a target tracking algorithm, a human pose estimation algorithm and a time series prediction model; establishing and training a lighting effect model for calculating stage lamp control data from the pose data of performers by using a machine learning algorithm and a lighting parameter formula; and calculating and obtaining stage lamp prediction control data according to the real-time pose data of performers obtained by a sensor node, so as to control the stage lamp. The present application realizes intelligent control of the stage lamp, realizes real-time monitoring and tracking of the performers, and ensures that the light and shadow can accurately follow the motion changes of the performers by predicting the trajectories of the performers in advance.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of stage lighting, in particular to a method, device and medium for realizing intelligent stage performance. BACKGROUND

[0002] With the rapid development of modern stage equipment and the rise of artificial intelligence technology, the development of intelligent stage performance technology is provided. Intelligent stage performance is an innovative solution that combines intelligent follow-spot tracking methods with stage performance. By applying intelligent algorithms and sensing technology to the stage lighting system, more accurate light tracking of performers is achieved, thereby improving performance effects and audience visual experience, and avoiding the disconnection between preset light control and performance caused by improvisation or mistakes of performers.

[0003] However, existing technical solutions or public data related to intelligent stage performance technology, such as a Chinese invention patent (application number 202410185349.0, a method for intelligent control of stage lighting), although it also provides technical inspiration for realizing performer action prediction and automatic light effect tracking, it identifies and predicts the highlight area on the stage through intelligent algorithms, and predicts the trajectory of the performer, thereby realizing intelligent control of the light. However, the disclosed scheme has various problems that are difficult to solve, such as single means of obtaining pose information, large model volume, multiple variables involved, wide range covered, model predicting at least four degrees of freedom of the highlight area, low correlation between degrees of freedom, resulting in poor learning efficiency, requiring large sample size, resulting in uneven prediction effect, long calculation period, and difficulty in training.

[0004] In addition, similar schemes are prone to failure when stage effects are mixed with a large amount of smoke, flames, and stroboscopic light, as they cannot accurately obtain stage images. Most existing similar schemes are still in the conceptual design stage and do not have practical implementation feasibility and conditions. SUMMARY

[0005] To overcome the above problems, the present application provides a method, device and medium for realizing intelligent stage performance.

[0006] The present application provides a method for realizing intelligent stage performance, comprising:

[0007] establishing a pose prediction model for predicting the pose data of the performer after a first time length, and obtaining a plurality of continuous performer historical pose data in advance as a training set and a validation set to complete the training of the pose prediction model; wherein the pose prediction model is provided with a target detection algorithm, a target tracking algorithm, a human pose estimation algorithm and a time series prediction model, and the first time length is a preset time length;

[0008] A light effect model for calculating stage light control data from performer pose data is established using a machine learning algorithm and a lighting parameter formula;

[0009] A plurality of sensor nodes are arranged around the stage, and the position information and action data of the performer are continuously acquired using the plurality of sensor nodes; wherein the sensor nodes include one or any combination of a camera, a depth sensor, a motion capture sensor, and an infrared sensor;

[0010] The position information and action data are preprocessed according to the installation positions of the plurality of sensor nodes to obtain real-time pose data of the performer;

[0011] The real-time pose data is imported into a pose prediction model for calculation to obtain predicted pose data, and the predicted pose data is imported into a light effect model to obtain stage light prediction control data;

[0012] The stage light prediction control data is used as stage light control data after a first time length in the future to control the stage light;

[0013] The lighting parameter formula is specifically:

[0014]

[0015] Wherein, P represents the lighting parameter, A represents the influence coefficient of the action type, ω A represents the action weight, L represents the influence coefficient of the performer's position, ω L represents the position weight, B represents the background influence coefficient, ω B represents the background weight, S represents the influence coefficient of the performance style, and ω S represents the style weight.

[0016] Preferably, the data preprocessing includes filtering of the position information, and denoising, edge detection, and color space conversion of the action data.

[0017] Preferably, the target detection algorithm, target tracking algorithm, human pose estimation algorithm, and time series prediction model are specifically:

[0018] The target detection algorithm is a YOLO model, which is used to identify the human target of the performer on the stage, and the output is a target frame;

[0019] The target tracking algorithm is a MOSSE model, which is used to track the trajectory data of the target frame;

[0020] The human pose estimation algorithm is an OpenPose model, which is used for identifying the skeletal key point coordinates of a performer from a target frame and outputting human pose data;

[0021] The human pose data and the trajectory data are taken as the pose data of the performer;

[0022] The time sequence prediction model is an LSTM model, which is used for predicting the pose data after a first time length in the future of the performer according to the continuous pose data of the performer.

[0023] Preferably, the method further comprises the following steps:

[0024] A plurality of preset strategies are set, and the preset strategies are used to limit the stage lamp control data of the specific performer pose data before model training.

[0025] Preferably, the method further comprises the following steps:

[0026] All performer pose data and stage lamp control data in the current performance are recorded, and the output is an instruction time axis;

[0027] The stage lamp control data corresponding to the plurality of performer pose data in the time axis is modified, the output is an artificial control instruction, and the execution priority of the artificial control instruction is higher than that of the stage lamp prediction control data.

[0028] Preferably, the method further comprises the following steps:

[0029] The identity information of the performer is identified through a face recognition algorithm;

[0030] Different pose prediction models and light effect models are trained and executed for different performers.

[0031] The application also provides a device for realizing intelligent stage performance, comprising a pose prediction module, a light effect module, an acquisition module, a pose calculation module, a light effect calculation module and a light control module.

[0032] The pose prediction module is used to establish a pose prediction model for predicting the pose data of the performer after a first time length, and to acquire a plurality of continuous performer historical pose data in advance as a training set and a verification set, so as to complete the training of the pose prediction model; wherein the pose prediction model is provided with a target detection algorithm, a target tracking algorithm, a human pose estimation algorithm and a time sequence prediction model, and the first time length is a preset time length.

[0033] The light effect is used to establish a light effect model for calculating stage lamp control data by performer pose data by using a machine learning algorithm and a lighting parameter formula; and a plurality of stage lamp control data and corresponding performer pose data are obtained as a training set and a verification set to complete training of the light effect model.

[0034] The lighting parameter formula is specifically as follows:

[0035]

[0036] Wherein, P represents the lighting parameter, A represents the influence coefficient of the action type, ω A represents the action weight, L represents the influence coefficient of the performer position, ω L represents the position weight, B represents the background influence coefficient, ω B represents the background weight, S represents the influence coefficient of the performance style, and ω S represents the style weight.

[0037] The acquisition module is used to set a plurality of sensor nodes around the stage, and continuously acquire the position information and action data of the performer by using the plurality of sensor nodes; wherein the sensor nodes include one or any combination of a camera, a depth sensor, a motion capture sensor and an infrared sensor.

[0038] The pose calculation module is used to perform data preprocessing on the position information and action data according to the installation positions of the plurality of sensor nodes to obtain the real-time pose data of the performer.

[0039] The light effect calculation module is used to import the real-time pose data into the pose prediction model for calculation to obtain predicted pose data, and import the predicted pose data into the light effect model to obtain stage lamp predicted control data.

[0040] The light control module is used to control the stage lamp by taking the stage lamp predicted control data as the stage lamp control data after a future first time length.

[0041] The present application provides a computer readable storage medium, the computer readable storage medium includes stored computer program, wherein, when the computer program runs, control the computer readable storage medium is in the device for executing the above-mentioned method for realizing intelligent stage performance.

[0042] The present application has the following advantages:

[0043] (1) The pose prediction model is combined by a target detection algorithm, a target tracking algorithm, a human pose estimation algorithm and a time series prediction model, realizes real-time monitoring and tracking of the performer, ensures that the light and shadow can accurately follow the action changes of the performer by predicting the trajectory of the performer in advance, and can be widely applied to various performance occasions such as stage plays, concerts and dance performances, and brings new possibilities and innovations to stage art.

[0044] (2) The intelligent control of the stage lamp is realized by combining the machine learning algorithm and the lighting parameter formula, the labor cost is saved, the work efficiency of the lighting designer is improved, the participation and immersion of the audience are enhanced, and the overall viewing experience is improved. BRIEF DESCRIPTION OF DRAWINGS

[0045] The application will be further described below with reference to the accompanying drawings of the specification.

[0046] Figure 1 The method flowchart of the embodiment one of the application. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0048] Referring to Figure 1 As the embodiment one of the application, a method for realizing intelligent stage performance is disclosed, which comprises:

[0049] S1, a pose prediction model for predicting the pose data of a performer after a first time length is established, and a plurality of continuous historical pose data of the performer in advance is obtained as a training set and a verification set, so as to complete the training of the pose prediction model; wherein the pose prediction model is provided with a target detection algorithm, a target tracking algorithm, a human pose estimation algorithm and a time series prediction model, and the first time length is a preset time length;

[0050] S2, the lighting effect is obtained by using a machine learning algorithm and a lighting parameter formula, and a plurality of stage lamp control data in advance is obtained as a training set and a verification set, so as to establish and train a lighting effect model for calculating the stage lamp control data from the pose data of the performer; wherein the stage lamp control data comprises the direction, rotation speed, intensity, color and light effect of each stage lamp;

[0051] S3, acquiring data, using a plurality of sensor nodes around the stage, continuously acquiring position information and action data of the performer; wherein the sensor nodes include one or any combination of a camera, a depth sensor, a motion capture sensor and an infrared sensor;

[0052] S4, pose calculation, according to the installation positions of the plurality of sensor nodes, data preprocessing is performed on the position information and the action data to obtain real-time pose data of the performer;

[0053] S5, light effect calculation, importing the real-time pose data into a pose prediction model for calculation to obtain predicted pose data, and importing the predicted pose data into a light effect model to obtain stage light prediction control data;

[0054] S6, light control, taking the stage light prediction control data as the stage light control data after a future first time length to control the stage light.

[0055] The data preprocessing includes filtering of the position information, and denoising, edge detection and color space conversion of the action data.

[0056] The embodiment also discloses a device for realizing intelligent stage performance, comprising a pose prediction module, a light effect module, an acquisition module, a pose calculation module, a light effect calculation module and a light control module.

[0057] The pose prediction module is used to establish a pose prediction model for predicting the performer's pose data after a first time length by using a target detection algorithm, a target tracking algorithm, a human pose estimation algorithm and a time series prediction model, and to acquire a plurality of continuous performer historical pose data in advance as a training set and a verification set to complete training of the pose prediction model; wherein the first time length is a preset time length.

[0058] The light effect is used to establish and train a light effect model for calculating stage light control data from performer pose data by using a machine learning algorithm and a lighting parameter formula, and to acquire a plurality of stage light control data in advance and corresponding performer pose data as a training set and a verification set;

[0059] The acquisition module is used to continuously acquire position information and action data of the performer by using a plurality of sensor nodes around the stage; wherein the sensor nodes include one or any combination of a camera, a depth sensor, a motion capture sensor and an infrared sensor.

[0060] The pose calculation module is used to perform data preprocessing on the position information and the action data according to the installation positions of the plurality of sensor nodes to obtain real-time pose data of the performer.

[0061] The light effect calculation module is configured to import the real-time pose data into a pose prediction model for calculation, obtain predicted pose data, and import the predicted pose data into a light effect model to obtain stage light prediction control data.

[0062] The light control module is configured to use the stage light prediction control data as stage light control data after a first time length in the future to control the stage light.

[0063] The embodiment realizes real-time monitoring and tracking of the performer through the target detection algorithm, the target tracking algorithm, the human pose estimation algorithm, and the time series prediction model, ensures that the light and shadow can accurately follow the changes in the performer's actions by predicting the performer's trajectory in advance, and can be widely applied to various performance occasions such as stage plays, concerts, and dance performances, bringing new possibilities and innovations to stage art. Through the machine learning algorithm and the lighting parameter formula, the intelligent control of the stage light is realized, the labor cost is saved, the work efficiency of the lighting engineer is improved, the audience's participation and immersion are enhanced, and the overall viewing experience is improved.

[0064] The following is Embodiment Two of the present application, which is a more specific application example of Embodiment One. Through the camera and the infrared sensor, the stage picture and the infrared image of the performer are collected. The software architecture used is: the algorithm logic is built using the Python programming language; image processing is performed using the OpenCV library; and the machine learning model is built using the TensorFlow, PyTorch, and other frameworks.

[0065] In step S1 of the embodiment, the pose prediction model is provided with a target detection algorithm, a target tracking algorithm, a human pose estimation algorithm, and a time series prediction model, specifically:

[0066] The target detection algorithm (such as YOLO, FasterR-CNN, etc.) is used to identify the human target of the performer from the stage, and the output is a target frame;

[0067] The target tracking algorithm (such as MOSSE, KCF, etc.) is used to track the trajectory data of the target frame;

[0068] The human pose estimation algorithm (such as OpenPose, AlphaPose, etc.) is used to identify the skeletal key point coordinates of the performer from the target frame, and the output is human pose data;

[0069] The human pose data and the trajectory data are used as the pose data of the performer;

[0070] According to the continuous pose data of the performer, the time series prediction model (such as LSTM, GRU, etc.) is used to predict the pose data of the target after a first time length in the future.

[0071] The motion capture sensors in this embodiment can be WIFI routers that transmit data to each other, and use WiFi-based human motion recognition technology through a dual-stream convolution-enhanced Transformer to identify the movements of actors on the stage. Different movements of the human body will produce "subtle" but "unique" signal changes in the channel state information (CSI) between the signal transmitter (TX) and the receiver (RX); a dual-stream convolution-enhanced Transformer (THAT) model is used to distinguish the original CSI input into a channel stream and a timing stream, and the channel stream and time stream are processed separately by two parallel neural modules to extract signal change features, thereby performing table verification of position and motion information. This motion capture sensor relies on occlusion during radio transmission to determine the actor's movements and positions, and is not affected by visual occlusions and light changes caused by smoke, lights, etc., so it can adapt to more complex special effects stages.

[0072] The target tracking algorithm uses the MOSSE algorithm, which was the first to apply correlation filtering to the field of target tracking. While maintaining good tracking results, its speed far exceeds other tracking algorithms of the same period, and can reach 669FPS at the fastest.

[0073] Assuming that this embodiment specifically adopts a human posture estimation model based on a convolutional neural network (CNN) and combines it with a long short-term memory network (LSTM) for motion prediction, the process is summarized as follows:

[0074] 1. Human pose estimation:

[0075] 1.1. Use the OpenPose model to identify the key point coordinates of the performer;

[0076] 1.2. Input video frames to the CNN model and output key point coordinates;

[0077] 2. Action Prediction

[0078] 2.1. Input the continuous sequence of key point coordinates into the LSTM model;

[0079] 2.2. The LSTM model outputs the position and movement trends of the performer in the next few seconds;

[0080] 3. Follow spot control:

[0081] 3.1. Calculate the moving angle of the follow spot based on the predicted position information;

[0082] 3.2. Send instructions to the follow spot controller to adjust the direction and height of the follow spot;

[0083] 4. Light and shadow effect adjustment:

[0084] 4.1, Calculate the required light effect parameters according to the action type and stage background;

[0085] 4.2, Change the color and intensity of the follow light through API call or pre-defined program.

[0086] With the iteration and innovation of the algorithm, the algorithm in this scheme can be replaced by an updated algorithm of the same kind to further improve the accuracy of prediction and operation efficiency. Those skilled in the art can understand that all algorithm alternatives based on the same principle belong to the protection scope of the present application.

[0087] The lighting parameter formula of the embodiment is specifically:

[0088]

[0089] Wherein, P represents the lighting parameter (including intensity, angle, color, etc.), A represents the influence coefficient of action type, ω A represents the action weight, L represents the influence coefficient of performer position, ω L represents the position weight, B represents the background influence coefficient, ω B represents the background weight, S represents the influence coefficient of performance style, ω S represents the style weight.

[0090] The above formula is a simple linear model, which is used to simplify the model learning and training process of various parameters and light effects in the stage. In actual application, more complex functions and parameters can be added to enhance the calculation accuracy of lighting parameters.

[0091] The specific implementation goal of the stage light control data of the embodiment is follow light control and light effect adjustment:

[0092] 1. Follow light control: use the predicted position information to adjust the direction and height of the follow light in real time, ensure that the light always focuses on the performer; adjust the speed of the follow light according to the moving speed of the performer, keep stable tracking;

[0093] 2. Light effect adjustment: combine the results of action prediction and lighting demand evaluation, automatically adjust the lighting angle, intensity and color; through the preset lighting mode or real-time calculation of lighting parameters, create unique light and shadow effect for performance.

[0094] The method of intelligent stage performance of the embodiment can further include the following steps:

[0095] S7, set a plurality of preset strategies, the preset strategies are used to limit the model to specific performer pose data stage light control data before model training.

[0096] This step is used to limit the model's light effect changes in specific scenarios, improving the accuracy of the model's output results. It allows the lighting designer to preset some scene corresponding light effects, thereby improving the accuracy of stage light control data prediction.

[0097] The method of the smart stage performance of the embodiment can further include the following steps:

[0098] S8, record all performer pose data and stage light control data in the current performance, and output as instruction timeline;

[0099] S9, modify the stage light control data corresponding to the pose data of a number of performers in the timeline, output as artificial control instructions, and make the execution priority of the artificial control instructions higher than that of the stage light prediction control data.

[0100] It is convenient for the lighting designer to observe the automatic control effect of the light in the subsequent performance process after the rehearsal and the preview, to insert artificial control instructions or modify algorithm parameters in some time periods, and to manually optimize the stage effect.

[0101] The method of the smart stage performance of the embodiment includes the following steps:

[0102] S10, identify the identity information of the performer through a face recognition algorithm;

[0103] S11, train and execute different pose prediction models and light effect models for different performers.

[0104] In order to arrange different light effects for different actors.

[0105] In other embodiments, a closed-loop control system can also be established through a feedback mechanism to adjust the algorithm parameters according to the actual stage lighting effect observed by the lighting designer or the evaluation model. By continuously optimizing the algorithm performance, the tracking accuracy and stability of the overall model are improved.

[0106] The following is Embodiment Three of the present application, which is a more specific application of Embodiment One. The data set, i.e. the training set and the validation set, of this embodiment is constructed in the following manner:

[0107] Data source: collect a large number of performer action videos, which should cover various dance styles, action types and performance situations to ensure the generalization ability of the model.

[0108] Data annotation: use human pose estimation algorithm (such as OpenPose) to process video frames and extract performer key point coordinates. These coordinates should include timestamps to maintain the continuity of actions.

[0109] Sequence data creation: The sequence of consecutive keypoint coordinates is used as input to the model, and the sequence length needs to be determined based on the complexity of the action and the range of prediction, for example, using the keypoint coordinates of the previous 5 seconds to predict the action of the next 1 second.

[0110] Data division: The dataset is divided into training, validation, and test sets, usually in the ratio of 70%, 15%, and 15%. The training set is used to train the model, the validation set is used to adjust the model parameters, and the test set is used to evaluate the final performance of the model.

[0111] The return function and standard of the model training in this embodiment are as follows:

[0112] Loss function: The Mean Squared Error (MSE) is usually used as the loss function for training the LSTM model, which measures the difference between the predicted keypoint coordinates and the actual coordinates. For each time step, the loss function can be represented as:

[0113]

[0114] where yi is the actual keypoint coordinate, y^i is the predicted keypoint coordinate by the model, and N is the number of keypoints.

[0115] Regularization term: To avoid overfitting, a regularization term such as L1 or L2 regularization can be added to the loss function to penalize large weight values.

[0116] Optimizer selection: Gradient descent optimizers such as Adam or RMSprop are used to minimize the loss function. These optimizers can automatically adjust the learning rate to speed up the convergence.

[0117] The basic unit of the LSTM model contains a memory cell and three gates (input gate, forget gate, output gate). Each gate is a linear layer plus an activation function (such as sigmoid or tanh) to control the flow of information. The mathematical formula of the LSTM cell is as follows:

[0118] Forget gate:

[0119]

[0120] where Wf is the weight matrix, bf is the bias vector, ht−1 is the hidden state of the previous time step, xt is the input of the current time step, and σ is the sigmoid activation function.

[0121] Input gate:

[0122]

[0123] Candidate memory cell state:

[0124]

[0125] Memory cell state:

[0126]

[0127] where denotes element-wise multiplication.

[0128] Output gate:

[0129]

[0130] Hidden state:

[0131]

[0132] The model training process is as follows:

[0133] Initialization: randomly initialize the weights and biases of the model.

[0134] Forward propagation: input the sequence data into the model, and calculate the output through the above formula.

[0135] Backpropagation: calculate the gradient of the loss function and update the weights.

[0136] Iterative training: repeat the forward propagation and backpropagation processes until the model converges or reaches the predetermined number of training rounds.

[0137] Through the above steps, the LSTM model learns the dynamic pattern of the performer's actions and predicts the position and action trend in the next few seconds.

[0138] The application also discloses a terminal device comprising a processor and a storage device, wherein the storage device is used to store one or more programs; when the one or more programs are executed by the processor, the processor realizes the method for realizing intelligent stage performance. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The processor is the control center of the test device, and connects each part of the test device through various interfaces and lines.

[0139] The storage device can be used to store computer programs and / or modules, and the processor realizes various functions of the terminal device by running or executing the computer programs and / or modules stored in the storage device, and calling data stored in the storage device. The storage device can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function, etc.; and the data storage area can store data created according to the use of the terminal device, etc. In addition, the storage device can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0140] If the modules / units for integrating the equipment for realizing the intelligent stage performance are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in at least one computer readable storage medium. When the processor executes the computer program, the steps of the above-mentioned various method embodiments can be realized. The computer program includes computer program code, which can be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer readable medium can include any entity or device capable of carrying computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0141] It should be noted that the above-described embodiments of the device and the apparatus are only illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme.

Claims

1. A method for realizing intelligent stage performance, characterized in that: include: Establishing a posture prediction model for predicting the performer's posture data after a first time length, and obtaining a plurality of previous continuous historical posture data of the performer as a training set and a validation set to complete the training of the posture prediction model; wherein the posture prediction model is provided with a target detection algorithm, a target tracking algorithm, a human posture estimation algorithm and a time series prediction model, and the first time length is a preset time length; Using machine learning algorithms and lighting parameter formulas, a lighting effects model is established to calculate stage lighting control data from performer posture data. Several sets of stage lighting control data and their corresponding performer posture data are obtained as training and validation sets to complete the training of the lighting effects model. The illumination parameter formula is specifically: Among them, P represents the lighting parameter, A represents the influence coefficient of the action type, ω A represents the action weight, L represents the influence coefficient of the performer's position, ω L represents the position weight, B represents the background influence coefficient, ω B represents the background weight, S represents the influence coefficient of performance style, ω S represents style weight; A plurality of sensor nodes are arranged around the stage, and the position information and motion data of the performers are continuously acquired by using the plurality of sensor nodes; wherein the sensor nodes include one or any combination of a camera, a depth sensor, a motion capture sensor, and an infrared sensor; According to the relative positions of several sensor nodes, the position information and motion data are preprocessed to obtain the performer's real-time posture data; Importing the real-time pose data into the pose prediction model to obtain predicted pose data, and importing the predicted pose data into the lighting effect model to obtain stage light prediction control data; The stage light prediction control data is used as the stage light control data after the first time length in the future to control the stage light.

2. A method for realizing intelligent stage performance according to claim 1, characterized in that: The data preprocessing includes filtering of position information, and denoising, edge detection and color space conversion of motion data.

3. A method for realizing intelligent stage performance according to claim 1, characterized in that: The target detection algorithm, target tracking algorithm, human posture estimation algorithm and time series prediction model are specifically: The target detection algorithm is the YOLO algorithm, which is used to identify the human targets of performers on the stage and output them as target boxes; The target tracking algorithm is a MOSSE algorithm, which is used to track the trajectory data of the target frame; The human pose estimation algorithm is the OpenPose algorithm, which is used to identify the coordinates of the performer's skeleton key points from the target frame and output them as human pose data; The human body posture data and trajectory data are used as the performer's posture data; The time series prediction model is an LSTM algorithm, which is used to predict the posture data of the target after a first time length in the future based on the performer's continuous posture data.

4. A method for realizing intelligent stage performance according to claim 1, characterized in that: The following steps are also included: Several preset strategies are set, and the preset strategies are used to enable the model to limit the stage light control data of specific performer posture data before model training.

5. The method for realizing intelligent stage performance according to claim 1, characterized in that: The following steps are also included: Record all the performer posture data and stage lighting control data in the current performance, and output them as a command timeline; Modify the stage light control data corresponding to several performer posture data in the timeline, output them as manual control instructions, and make the execution priority of the manual control instructions higher than the stage light prediction control data.

6. A method for realizing intelligent stage performance according to claim 1, characterized in that: The following steps are also included: Identify the performer’s identity information through facial recognition algorithms; Different pose prediction models and lighting effect models are trained and executed for different performers.

7. A device for realizing intelligent stage performance, characterized in that: include: Posture prediction module, lighting effect module, acquisition module, posture calculation module, lighting effect calculation module and lighting control module; The posture prediction module is used to establish a posture prediction model for predicting the performer's posture data after a first time length, and obtain a plurality of previous continuous performer's historical posture data as a training set and a validation set to complete the training of the posture prediction model; wherein the posture prediction model is provided with a target detection algorithm, a target tracking algorithm, a human posture estimation algorithm and a time series prediction model, and the first time length is a preset time length; The lighting effect is used to establish a lighting effect model for calculating stage light control data based on performer posture data using a machine learning algorithm and a lighting parameter formula; a plurality of stage light control data and their corresponding performer posture data are obtained as training sets and validation sets to complete the training of the lighting effect model; The illumination parameter formula is specifically: Among them, P represents the lighting parameter, A represents the influence coefficient of the action type, ω A represents the action weight, L represents the influence coefficient of the performer's position, ω L represents the position weight, B represents the background influence coefficient, ω B represents the background weight, S represents the influence coefficient of performance style, ω S represents style weight; The acquisition module is used to set up a plurality of sensor nodes around the stage, and continuously acquire the position information and motion data of the performers using the plurality of sensor nodes; wherein the sensor nodes include one of a camera, a depth sensor, a motion capture sensor and an infrared sensor, or any combination thereof; The posture calculation module is used to perform data preprocessing on the position information and motion data according to the installation positions of the plurality of sensor nodes to obtain the performer's real-time posture data; The lighting effect calculation module is used to import the real-time posture data into the posture prediction model for calculation to obtain predicted posture data, and import the predicted posture data into the lighting effect model to obtain stage light prediction control data; The lighting control module is used to control the stage lights by using the stage light prediction control data as the stage light control data after a first time length in the future.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method for realizing intelligent stage performance according to any one of claims 1 to 6.

Images