Method for changing at least one lighting state by voice command, computer system configured to execute the method, and computer program product comprising software code sections

Abstract

The invention relates to a method for changing at least one lighting state by means of a voice command (2), comprising the steps of receiving and inputting the voice command (2) into a language model (3), interpreting and decomposing the voice command (2) into at least one subtask (4), obtaining at least one lighting feature (6), at least one lighting object (7), and at least one setting option (8) of the lighting feature (6), determining at least one lighting object (7) and at least one lighting feature (6) of the determined lighting object (7) by means of inference with a trained machine learning model (5) with input of the lighting feature (6), the setting option (8), and the subtask (4), and creating a sequence plan (10) by means of inference with the trained machine learning model (5) with input of the determined lighting object (7).of the specific lighting feature (6) and / or the setting option (8) and of changing the lighting state according to the sequence plan (10). Furthermore, the invention relates to a computer system (20) configured to execute the method, and to a computer program product comprising software code sections designed such that the method can be executed by at least one processor.

Description

[0001] The invention relates to a method for changing at least one lighting state by voice command, a computer system configured to execute the method, and a computer program product comprising software code sections by means of which the method for changing at least one lighting state by voice command is executed.

[0002] It is a known practice to enhance performances such as concerts or plays, as well as purely auditory performances like music in nightclubs, with light shows and / or to increase the visibility of the presentation. With advancing lighting technology, operators are offered an ever-increasing number of options. Therefore, it is now necessary for lighting systems to be automatically controlled by complex control technology with regard to both the emitted light and its movement. Various control protocols for controlling individual lighting elements have become established, requiring programming with sometimes differing commands. The light shows are generally partially or completely pre-programmed and played back during the performance, with limited adjustments possible.

[0003] The widespread use of the aforementioned light show programming means that, due to its complexity, it can only be carried out by trained personnel. These personnel must, on the one hand, be proficient in the relevant programming language, which is sometimes developed specifically by the control technology manufacturer and therefore not standardized across the industry, and on the other hand, have a thorough understanding of the layout and features of the existing lighting equipment as well as the venue itself in order to create an appealing and coordinated light show.

[0004] Few assistance systems are known in the prior art that make suggestions to the user based on user preferences, a light show library, or previous light shows. However, US patent 11,687,760 B2 discloses a method that, using a large number of previously created light shows, suggests at least one further command to the user after initial commands have been entered, which, statistically speaking, frequently follows the commands entered so far.

[0005] Nevertheless, creating, modifying, and adapting light shows remains a complex process that can only be carried out after adequate training. The application is likely to become even more complex due to advancing technological possibilities.

[0006] There is therefore a great need for a method that enables the simple and intuitive changing of the lighting state of lighting objects for a light show, and which is usable for all conceivable applications. Consequently, a further focus is on ensuring that the method is compatible with all devices known in the prior art and can be implemented cost-effectively. In this way, the creation and / or adaptation of light shows can be largely supported and / or automated. The invention therefore aims to provide a method, a computer system, and a computer program product to overcome the aforementioned difficulties.

[0007] This problem is solved in a surprisingly simple but effective way by a method according to the teaching of independent claim 1, a computer system according to the teaching of main claim 14 and a computer program product according to the teaching of main claim 15.

[0008] According to the invention, a method for changing at least one lighting state by means of a voice command is proposed, wherein the method comprises the following steps: a. in the first step receiving a voice command and entering a voice command into a language model; b. in the second step, an interpretation and breakdown of the speech command into at least one subtask; c. in the third step, obtaining at least one lighting feature of at least one lighting object and at least one setting option of the at least one lighting state and determining at least one lighting object and at least one lighting feature of the at least one specific lighting object by means of inference with the trained machine learning model by inputting the at least one lighting feature, the at least one setting option and the at least one subtask; d. in the fourth step, a process plan is created by means of inference with the trained machine learning model, taking into account at least one specific lighting object, at least one specific lighting feature and / or at least one setting option; and e. in the fifth step a change of at least one lighting state according to the schedule.

[0009] The invention is based on the fundamental idea that the ever-advancing development of language models means they can be used to interpret a user's command written in natural language and ultimately translate it into a change to one or more lighting objects for a light show and / or to create a light show. For this to happen, it is necessary to first break down the voice command into subtasks and then, for execution of these subtasks by a specially trained machine learning model, translate them into corresponding control codes and / or other possible control methods and / or visualizations for the user.

[0010] In the first step (a), a voice command is received and entered into a language model. This enables the execution of the subsequent step (b). Receipt preferably occurs via a suitable interface through real-time input from a user applying the method. Suitable interfaces are described elsewhere. Preferably, the language model used is specifically trained for the application of the method according to the invention. Even more preferably, the language model is a large language model (LLM), in particular a generative pre-trained transformer (GPT).

[0011] In the next, here second, step b, the voice command is interpreted by the language model and broken down into at least one subtask. In other words, the language model decomposes the voice command into individual aspects that must be processed to achieve the desired change. Preferably, the voice command is broken down into at least 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, or 100 subtasks. The subtasks may in particular involve the modification of various lighting characteristics of the lighting state of a lighting object, individual lighting characteristics of the lighting state of different lighting objects, various lighting characteristics of different lighting objects and / or other aspects outside the lighting object, in particular aspects that directly or indirectly affect the at least one lighting object.At least one subtask is preferentially converted into a predefined syntax. This is easily achievable due to the limited number of modification options and leads to better and / or more precise results.

[0012] In the next, here third, step c, at least one lighting feature of at least one lighting object and at least one setting option of the at least one lighting object are obtained. The at least one lighting feature, the at least one setting option, and the at least one subtask are then input into a trained machine learning model. This trained machine learning model uses inference to determine at least one lighting object and at least one lighting feature of the previously determined lighting object. Through this determination, the at least one lighting object becomes at least one specific lighting object, and the at least one specific lighting feature becomes at least one specific lighting feature.To bring about the desired change via voice command, it is usually necessary and / or desired to modify individual lighting characteristics and / or lighting objects. Obtaining the lighting characteristic can relate to the type of lighting characteristic, its value, and / or its value range. In other words, obtaining the lighting characteristic could involve acquiring knowledge about the presence of a specific lighting characteristic in at least one lighting object, the specific setting of the lighting characteristic, and / or the technically possible settings.It is preferred that the technically possible settings are taken into account in such a way that the determination process prevents the identification of at least one lighting object and / or at least one lighting feature that cannot implement the desired change due to its setting options. The identification of the at least one lighting feature and / or at least one setting option is preferably achieved by retrieving it from a database and / or stored data, by querying the settings of the at least one lighting feature, by evaluating previous control commands, and / or by querying at least one state sensor. This identification is made possible by training the machine learning model using training datasets prior to executing the procedure.The training preferably involves supervised learning, unsupervised learning, or reinforcement learning. This determination specifically involves selecting and / or setting an assumed value for these lighting characteristics and / or lighting objects. Preferably at least 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1000, 1500, 2000, 2500 3,000, 3,500, 4,000, 4,500, 5,000, 5,500, 6,000, 6,500, 7,000, 7,500, 8,000, 8,500, 9,000, 9,500, 10,000, 15,000, 20,000, 25,000, 30,000, 35,000, 40,000, 45,000, 50,000, 55,000, 60,000, 65,000, 70,000, 75,000, 80,000, 85,000, 90,000, 95,000 100,000, 150,000, 200,000, 250,000, 300,000, 350,000, 400,000, 450,000, 500,000, 550,000, 600,000, 650,000, 700,000, 750,000, 800,000, 850,000, 900,000, 950,000, 1,000,000, 1,500,000, 2,000,000, 2,500,000, 3,000.000, 3,500,000, 4,000,000, 4,500,000, 5,000,000, 5,500,000, 6,000,000, 6,500,000, 7,000,000, 7,500,000, 8,000,000, 8,500,000, 9,000,000, 9,500,000, 10,000,000, 15,000,000, 20,000,000, 25,000,000, 30,000,000, 35,000,000, 40,000,000, 45,000,000 50,000,000, 55,000,000, 60,000,000, 65,000,000, 70,000,000, 75,000,000, 80,000,000, 85,000,000, 90,000,000, 95,000,000, 100,000,000, 150,000,000, 200,000,000, 250,000,000, 300,000,000, 350,000,000, 400,000,000, 450,000,000, 500,000,000 550,000,000, 600,000,000, 650,000,000, 700,000, 750,000,000, 800,000, 850,000,000, 900,000, 950,000,000 or 1,000,000,000 different or identical lighting characteristics are achieved. Even more preferably, the lighting state of at least one lighting object is achieved.Further preferred is at least one lighting feature of at least 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1000, 1500, 2000, 2,500, 3,000, 3,500, 4,000, 4,500, 5,000, 5,500, 6,000, 6,500, 7,000, 7,500, 8,000, 8,500, 9,000, 9,500, 10,000, 15,000, 20,000, 25,000, 30,000, 35,000, 40,000, 45,000, 50,000, 55,000, 60,000, 65,000, 70,000, 75,000, 80,000, 85,000, 90,000 95,000, 100,000, 150,000, 200,000, 250,000, 300,000, 350,000, 400,000, 450,000, 500,000, 550,000, 600,000, 650,000, 700,000, 750,000, 800,000, 850,000, 900,000, 950,000, 1,000,000, 1,500,000, 2,000,000, 2,500,000, 3,000,000, 3,500,000 4,000,000, 4,500,000, 5,000,000, 5,500,000, 6,000,000, 6,500,000, 7,000,000, 7,500,000, 8,000,000, 8,500,000, 9,000,000, 9,500,000, 10,000,000, 15,000,000, 20,000,000, 25,000,000, 30,000,000, 35,000,000, 40,000.000, 45,000,000, 50,000,000, 55,000,000, 60,000,000, 65,000,000, 70,000,000, 75,000,000, 80,000,000, 85,000,000, 90,000,000, 95,000,000, 100,000,000, 150,000,000, 200,000,000, 250,000,000, 300,000,000, 350,000,000, 400,000,000, 450,000,000 500,000,000, 550,000,000, 600,000,000, 650,000,000, 700,000, 750,000,000, 800,000,000, 850,000,000, 900,000, 950,000,000, or 1,000,000,000 different or identical lighting objects are obtained, which can have the same and / or different lighting characteristics. The lighting state of all lighting objects is the preferred method. This allows for particularly precise and comprehensive adjustments; however, the amount of data obtained is very large, depending on the setup. Preferably at least 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1,000, 1.500, 2.000, 2.500, 3.000, 3.500, 4.000, 4.500, 5.000, 5.500, 6.000, 6.500, 7.000, 7.500, 8.000, 8.500, 9.000, 9.500, 10.000, 15.000, 20.000, 25.000, 30.000, 35.000, 40.000, 45.000, 50.000, 55.000, 60.000, 65.000, 70.000, 75.000, 80.000, 85.000, 90.000, 95.000, 100.000, 150.000, 200.000, 250.000, 300.000, 350.000, 400.000, 450.000, 500.000, 550.000, 600.000, 650.000, 700.000, 750.000, 800.000, 850.000, 900.000, 950.000, 1.000.000, 1.500.000, 2.000.000, 2.500.000, 3.000.000, 3.500.000, 4.000.000, 4.500.000, 5.000.000, 5.500.000, 6.000.000, 6.500.000, 7.000.000, 7.500.000, 8.000.000, 8.500.000, 9.000.000, 9.500.000, 10.000.000, 15.000.000, 20.000.000, 25.000.000, 30.000.000, 35.000.000, 40.000.000, 45.000.000, 50.000.000, 55.000.000, 60.000.000, 65.000.000, 70.000.000, 75.000.000, 80.000.000, 85.000.000, 90.000.000, 95.000.000, 100.000.000, 150.000.000, 200.000.000, 250.000.000, 300.000.000, 350.000.000, 400.000.000, 450.000.000, 500.000.000, 550.000.000, 600.000.000, 650.000.000, 700.000.000, 750,000,000, 800,000,000, 850,000,000, 900,000,000, 950,000,000 or 1,000,000,000 setting options have been achieved. A minimum of one setting option of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1000, 1500, 2000 is preferred. 2,500, 3,000, 3,500, 4,000, 4,500, 5,000, 5,500, 6,000, 6,500, 7,000, 7,500, 8,000, 8,500, 9,000, 9,500, 10,000, 15,000, 20,000, 25,000, 30,000, 35,000, 40,000, 45,000, 50,000, 55,000, 60,000, 65,000, 70,000, 75,000, 80,000, 85,000, 90,000 95,000, 100,000, 150,000, 200,000, 250,000, 300,000, 350,000, 400,000, 450,000, 500,000, 550,000, 600,000, 650,000, 700,000, 750,000, 800,000, 850,000, 900,000, 950,000, 1,000,000, 1,500,000, 2,000,000, 2,500,000, 3,000,000, 3,500,000 4,000,000, 4,500,000, 5,000,000, 5,500,000, 6,000,000, 6,500,000, 7,000,000, 7,500.000, 8,000,000, 8,500,000, 9,000,000, 9,500,000, 10,000,000, 15,000,000, 20,000,000, 25,000,000, 30,000,000, 35,000,000, 40,000,000, 45,000,000, 50,000,000, 55,000,000, 60,000,000, 65,000,000, 70,000,000, 75,000,000, 80,000,000, 85,000,000 90,000,000, 95,000,000, 100,000,000, 150,000,000, 200,000,000, 250,000,000, 300,000,000, 350,000,000, 400,000,000, 450,000,000, 500,000,000, 550,000,000, 600,000,000, 650,000,000, 700,000,000, 750,000,000, 800,000,000, 850,000,000, 900,000,000 950,000,000 or 1,000,000,000 different or identical lighting objects are obtained, which may have the same and / or different setting options. Ideally, all setting options for all lighting objects are obtained. This allows for particularly precise and comprehensive adjustments; however, the amount of data obtained is very large, depending on the setup.When determining the number of lighting objects and / or lighting features, a number that is less than the total number of available lighting objects is preferably determined. Further preferred are at least 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1,000, 1,500, 2,000, 2,500. 3,000, 3,500, 4,000, 4,500, 5,000, 5,500, 6,000, 6,500, 7,000, 7,500, 8,000, 8,500, 9,000, 9,500, 10,000, 15,000, 20,000, 25,000, 30,000, 35,000, 40,000, 45,000, 50,000, 55,000, 60,000, 65,000, 70,000, 75,000, 80,000, 85,000, 90,000, 95,000 100,000, 150,000, 200,000, 250,000, 300,000, 350,000, 400,000, 450,000, 500,000, 550,000, 600,000, 650,000, 700,000, 750,000, 800,000, 850,000, 900,000, 950,000, 1,000,000, 1,500,000, 2,000,000, 2,500,000, 3,000,000, 3,500,000, 4,000,000 4,500,000, 5,000,000, 5,500,000, 6,000.000, 6,500,000, 7,000,000, 7,500,000, 8,000,000, 8,500,000, 9,000,000, 9,500,000, 10,000,000, 15,000,000, 20,000,000, 25,000,000, 30,000,000, 35,000,000, 40,000,000, 45,000,000, 50,000,000, 55,000,000, 60,000,000, 65,000,000, 70,000,000 75,000,000, 80,000,000, 85,000,000, 90,000,000, 95,000,000, 100,000,000, 150,000,000, 200,000,000, 250,000,000, 300,000,000, 350,000,000, 400,000,000, 450,000,000, 500,000,000, 550,000,000, 600,000,000, 650,000,000, 700,000,000, 750,000,000 800,000,000, 850,000,000, 900,000,000, 950,000,000 or 1,000,000,000 lighting objects and / or lighting features are defined. Even more preferably, at least one lighting feature and / or at least one setting option of the at least one specific lighting object is defined, which is to undergo a change or which is to bring about the change.The most preferred quantities are at least 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700, 750, 800, 850, 900, 950, 1000, 1500, 2000. 2,500, 3,000, 3,500, 4,000, 4,500, 5,000, 5,500, 6,000, 6,500, 7,000, 7,500, 8,000, 8,500, 9,000, 9,500 or 10,000 setting options of at least one specific lighting object are determined.

[0013] In the next, here fourth, step d, a process plan is generated by inferring the input of at least one specific lighting object, at least one specific lighting feature, and / or at least one setting option into the trained machine learning model. Generating the process plan by inference is possible after training the machine learning model with training datasets prior to executing the procedure. Preferably, this training involves supervised learning, unsupervised learning, or reinforcement learning. The process plan executes all subtasks according to the user's specifications.Preferably, the trained machine learning model is updated during and / or after the execution of the method according to the invention by having the user evaluate the generated process plan or by evaluating intermediate information provided to the user during the execution of the method, and by feeding this evaluation as a new training data set to the machine learning model. This enables continuous improvement and individualization of the trained machine learning model by the user. Preferably, at least one setting option for adjusting the at least one specific lighting object and / or the at least one specific lighting feature is defined in step c. and / or in step d.

[0014] The trained machine learning model is preferably divided into two parts, with the first part being used in step c and the second part in step d. This allows for easy retrieval of intermediate results, in particular the result of determining the at least one lighting object and / or the at least one lighting feature, thus facilitating error correction through adjustment. Furthermore, it is conceivable that there are two independent trained machine learning models, with the trained machine learning model in step c being a first trained machine learning model and the trained machine learning model in step d being a second trained machine learning model. Even more preferably, the trained machine learning model is a single unit.This allows step c. and step d. to be executed partially or completely in parallel and / or to be repeated at least once. This leads to better results with faster execution time. The trained machine learning model is particularly preferred as an artificial neural network, especially a recurrent neural network (RNN), a feedforward neural network (FNN), a convolutional neural network (CNN), a transformer, a flow-based generative model, an evolving neural network, an encoder-decoder model, a variational autoencoder, an autoregressive model (AR-MA model), a restricted Boltzmann machine (RBM), and / or a diffusion model.a hidden Markov model (HMM) and / or a support vector machine (SVM). Furthermore, it is conceivable to use the methods of genetic programming, boosting, decision tree machine learning, kernel density estimation (KDE), expert systems (ES), a (naive) Bayesian classifier, gradient boosting, linear discriminant analysis, nearest neighbor classification, cluster analysis methods, in particular single-linkage, complete-linkage, Ward's method, K-means algorithm, fuzzy C-means algorithm, expectation maximization algorithm (EM algorithm), DBSAN (density-based spatial clustering of applications with noise), the STING algorithm (statistical information grid-based clustering algorithm), and / or the CLI-QUE algorithm (clustering). Inquest algorithm),and / or an anomaly detection method, in particular the Local Outlier Factor (LOF), the Isolation Forest, and / or the Autoencoder, and / or Principal Component Analysis (PCA). Furthermore, reinforcement learning methods can be used, such as associative reinforcement learning, deep reinforcement learning, adversarial deep reinforcement learning, fuzzy reinforcement learning, and / or safe reinforcement learning. In particular, it is conceivable that data clustering methods are also used. Suitable measures for creating, using, and / or training are known to those skilled in the art. It is also conceivable that the training data is stored in a database.wherein the database is continuously expanded with new training data during operation or use of the inventive method. Furthermore, it is conceivable that the trained machine learning model is the language model from step b. Other machine learning-capable models and methods for their creation, use, and / or training are known to those skilled in the art.

[0015] Furthermore, it is conceivable that after the execution of steps a, b, c, and / or d, an agent system checks and validates the result, namely the at least one subtask, the determination of the at least one lighting object and / or the at least one lighting feature of the at least one specific lighting object, and / or the process flow. It is conceivable to repeat steps a, b, c, and / or d, partially or completely, with modified input, should the check fail. In a preferred extension of the agent system, several agents with dedicated roles are used, particularly for checking the results, analyzing error messages, generally monitoring and distributing the subtasks, and / or executing the commands themselves.

[0016] Preferably, the workflow comprises at least one simulation dataset, at least one model, in particular a virtual 3D model, at least one cue, in particular a series of cues, at least one preset, at least one sequence, at least one stack, at least one video file, in particular a video recording and / or simulation, and / or at least one data package, in particular comprising recipes, MAtricks, phasers, timecodes, macros, Lua plugins, filters, selections, effects, bitmaps, and / or generators. The workflow preferably comprises a DMX data package or an ArtNet data package. These are the most widely used protocols for controlling lighting technology. The data package itself preferably comprises at least one preset, at least one cue, at least one sequence, and / or at least one stack.The simulation dataset preferably represents a simulation of a stage with lighting objects, particularly lighting devices, arranged on it. More preferably, the model also depicts a stage with lighting objects, particularly lighting devices. More preferably, the video file, in particular the video recording and / or the simulation, shows the sequence of a light show.

[0017] Subsequently, in the next, here fifth, step e, at least one lighting state is changed according to the sequence. It is also conceivable that the sequence is saved in a series of sequences, particularly at the end of the sequence, for the creation and storage of a light show. Furthermore, it is conceivable that the sequence replaces another sequence within the sequence. This modifies an existing light show according to the user's specifications. Preferably, the change according to the sequence includes a movement, in particular a movement speed, and / or a change of state, such as switching on and off, changes in the intensity, focus, shape, and / or color of at least one lighting object, in particular of the at least one specific lighting object.Even more preferably, the process flow includes a data package that can be transferred directly to a lighting control system, in particular a lighting control system, via an output interface.

[0018] The process can be carried out using a real-world lighting object, particularly a lighting fixture on a performance area, especially a stage, or virtually in a simulation. It can also be carried out using a virtual lighting object, particularly parameters in a control code. Furthermore, the process can be partially or fully automated. In lighting show technology, it is common practice to first create complex light shows in a simulation program and then apply them to the real-world stage technology represented by the simulation. The sequence can also take into account possible limitations imposed by the lighting object or at the user's request. In particular, it is conceivable that the sequence includes switching off the light source of the lighting object before it moves into its final position.When the final position is reached, the light source of the lighting object is switched on again, and the lighting characteristics specified in the modification plan are set.

[0019] It is clear to a person skilled in the art that the procedure described above comprises at least the five steps mentioned, which can be repeated individually or collectively any number of times. It is also conceivable that the procedure includes further steps, as described elsewhere, and / or that the described steps can be broken down into sub-steps, sub-steps, or sub-tasks.

[0020] The inventive method implements the user's desired change expressed in a voice command in a simple yet effective manner, without requiring the user to employ a complicated programming language. In particular, the user can use everyday language and / or colloquial language. This enables the simple and rapid creation of light shows without prior training.

[0021] The term "language model" refers to a mathematical model that models the sequence of elements in a sequence of natural language and is adapted to capture and isolate at least one subtask in the natural language.

[0022] The term "subtask" refers to a user-expressed request to change the lighting state of at least one lighting feature of the lighting state, a user-assigned selection of at least one lighting object or at least one group of lighting objects whose lighting state(s) are to be changed, and / or a direct or indirect aspect thereof and / or of an existing light show for at least one lighting object.

[0023] The term "lighting object" refers to an object, with or without a direct physical counterpart, within a lighting control system that has at least one manipulable lighting feature, whereby changing the lighting feature can result in a human-perceivable static or dynamic change in the physical representation of the lighting control system. A lighting object is, in particular, but by no means exclusively, a lighting device, a speedmaster, a cue, a preset, a sequence, a fixture, a temporal sequence, and / or a lighting mood.

[0024] The term "lighting device" refers to a device that can generate light for illumination and / or for effect purposes in a light show and is in particular controllable with a lighting control system.

[0025] The term "cue" refers to a container of data, especially of lighting objects, that corresponds to a single lighting mood or look of the staging, whereby a cue is usually preset and / or presettable.

[0026] The term "preset" refers to a container of data, in particular lighting objects, a limited number of representations of certain lighting features, which are used to create cues.

[0027] The term "sequence" refers to a list and / or a series of cues.

[0028] The term "type of lighting fixture" refers to the type of lighting fixture. In particular, the type of lighting fixture can be a PAR light, a blinder, a floodlight, a lens spotlight, a panning head, a scanner, a show laser, an LED spotlight, a panel light, a horizontal light, and / or a moving light. Those skilled in the art are aware of other lighting fixtures and methods for controlling them.

[0029] The term "lighting feature" refers to a single characteristic of a lighting object, or the value representing that single characteristic and / or the range of settings for that representing value. A lighting feature includes, but is not limited to, the status, color of the light, hue, color temperature, brightness, zoom, focus, iris, shape, orientation, rotation, position, and / or speed.

[0030] The term "illumination state" refers to the set of all lighting characteristics of a lighting object.

[0031] The term "state of the lighting control system" refers to the set of lighting states of all lighting objects in a lighting control system.

[0032] The term "acquisition" refers to the gathering of information and / or knowledge so that it is available to the machine learning model in a way that enables processing.

[0033] The term "machine learning model" refers to a program that is designed to recognize statistical relationships, patterns and / or structures between the information contained in the training data sets without explicit specification in the programming and, based on this, to determine at least one output from information received as input.

[0034] The term "inference" refers to deriving at least one output using the machine learning model created from the training datasets.

[0035] The term "determine" or "determine" refers in particular to the selection of at least one lighting object, whereby at least one lighting characteristic of the at least one specific lighting object is to be changed according to the user's specifications, the selection of at least one lighting characteristic, whereby the at least one lighting characteristic is to be changed according to the user's specifications, and / or the setting of at least one value of at least one specific lighting characteristic that is to correspond to the user's specifications.

[0036] The term "agent" refers to an autonomously acting system that makes decisions based on the input of information.

[0037] Advantageous further developments of the invention, which can be implemented individually or in combination, are presented in the dependent claims.

[0038] It is conceivable that the procedure after step d. and / or after step e. comprises a further step, step f., wherein step f. comprises the following: f. Obtaining and validating at least one modified lighting feature.

[0039] During validation, it is preferably verified, in particular, whether the lighting feature has changed in the manner specified in the flowchart and the subtask. Validation can be used to verify whether the execution of steps b. to e. has led to the desired result. Validation is preferably performed by an agent. More preferably, the agent is a machine learning model trained for validation or a deterministic algorithm designed for validation. For validation, the agent is preferably provided with, in addition to the at least one changed lighting feature, the at least one subtask, namely the at least one lighting feature, the determination of the at least one lighting object and / or the at least one lighting feature, and / or the flowchart.The at least one lighting feature pertains to the setting of the modified lighting feature before the change. Providing more information makes the validation more reliable. Even more preferred is the validation of all lighting features changed in step e. in step f. Most preferred is the acquisition and validation of the lighting system's state. Complete capture allows for the recording and consideration of lighting features that have not been changed but are intended to change following a user voice command. Additionally or alternatively, validation can be performed by querying the user.

[0040] In a training course, it is conceivable that step f. further comprises the following: modifying the subtask, the definition, and / or the process plan, and repeating steps b., c., d., e., and / or f. if the validation yields a negative result. If the modified lighting feature and / or the change to the modified lighting feature does not correspond to the specifications of the subtask and / or the process plan, a corresponding new process plan is created through modification to achieve the desired goal. It is obvious that, depending on where the deviation occurred or where the modification is made, the steps relating to the modification are repeated. If the process plan is modified, it is conceivable not to repeat steps b. and c., or not to repeat them completely. However, it is conceivable to repeat step c.to repeat the process, if necessary, to determine at least one different lighting object and / or at least one different lighting feature. Preferably, the modification includes reverting the at least one modified lighting feature.

[0041] It is also conceivable that the voice command in step a is either oral or written. An oral voice command can be entered via an interface designed as a microphone. A written voice command can be entered, in particular, via an input interface, especially a keyboard. It is conceivable that the voice command comprises both an oral and a written component, whereby the components are combined. Furthermore, it is conceivable that the language model in step a. converts the oral component into a written component, especially text. Finally, it is conceivable that an oral voice command is recorded elsewhere and played back locally.

[0042] The term "oral" refers to spoken language.

[0043] The term "written" refers to written language.

[0044] It is also conceivable that the sequence of events includes an executable control code for a lighting control system, in particular a lighting control console, and that the change in step e. is effected by executing the control code. This allows for a direct and immediate change to the lighting technology at the event venue via voice command using the method according to the invention. The control code preferably comprises at least one control command.

[0045] The term "lighting control system" refers to software, in particular software installed on a lighting control console, with interfaces for controlling at least one real lighting device and interfaces to a human user, which enables the user, insofar as technically possible and supported, to control the shape, focus, color, intensity and / or direction of the emitted light of the lighting device.

[0046] In a further development of the process, it is conceivable that in step d., the creation of the flowchart is carried out by inference with the trained machine learning model, taking the subtask as input. This allows the subtask to be considered again at this stage of the process, leading to better and more precise results. Intermediate validation by an agent or agent system can also be performed to prevent a faulty flowchart. The at least one specific lighting object, the at least one specific lighting feature, and / or the at least one setting option, along with the subtask, continue to be input into the trained machine learning model.

[0047] The term "consider" refers to the provision of the information to be considered in a way that allows the trained machine learning model and / or language model to recognize statistical relationships, patterns and / or structures, particularly in relation to the other inputs and / or information, but does not necessarily have to.

[0048] Furthermore, it is conceivable that in step b. the creation of at least one subtask takes context into account. This allows for the integration of additional information when processing the voice command, enabling the user to use language in a much more natural way. This prevents incorrect execution of the workflow and achieves more precise results. It also makes it possible to extract information from a conversation between at least two users. Preferably, the context is at least one previous voice command entered by the user before the current voice command. Therefore, the user does not have to formulate a complete, self-contained voice command each time. Preferably, at least 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20, and in particular all, previous voice commands are considered.

[0049] Furthermore, it is conceivable that step b. involves creating at least one subtask, step c. involves determining the at least one lighting object and the at least one lighting feature of the at least one specific lighting object by means of inference with the trained machine learning model, and / or step d. involves creating the schedule by means of inference with the trained machine learning model, taking into account at least one further factor. Preferably, the further factor is a piece of music, a location feature, an additional state of at least one additional device, a light show, and / or a sketch. Considering the piece of music allows for a light show that is better tailored to the planned performance, thus simplifying the creation of the light show for the user. A location feature is, in particular, a feature related to the venue, specifically whether it is a hall or an open-air area.Furthermore, a time parameter can be taken into account, allowing conclusions to be drawn about the ambient brightness. The additional equipment can represent common stage technology components, such as a curtain, a screen, and / or an effects unit. It is also conceivable that an agent could be used to create a video sequence for the screen. Inputting an existing light show, particularly as a control code, allows the light show to be adapted to the specific conditions and / or the user's individual preferences. The sketch can include, in particular, a stage design and / or a stage setup.

[0050] Furthermore, it is conceivable that in step c. at least one signal from at least one sensor and / or transmitter is obtained, wherein in step c. the determination of the at least one lighting object and the at least one lighting feature is carried out by inference with the trained machine learning model using the at least one signal as input, and / or in step d. the creation of the sequence of events is carried out by inference with the trained machine learning model using the at least one signal as input. The sensor and / or transmitter allows external data relevant to the light show to be taken into account. In particular, the sensor and / or transmitter can capture information about external data such as ambient brightness, ambient noise, and / or the events on stage. Specifically, the sensor can detect the movements of people performing on stage. A camera capturing the stage is particularly preferred as the sensor.The transmitter is preferably part of a tracker. This allows the stage show to be better coordinated with the events taking place on stage via voice command. In addition to the signal, other factors, mentioned elsewhere, can or must be entered and / or taken into account in steps c and / or d.

[0051] In a further embodiment of the invention, it is conceivable that the voice command comprises a control code in at least one first nomenclature, wherein step b and / or step c includes translating the control code from the at least one first nomenclature into a second nomenclature. Within the scope of the invention, it has been recognized that a common problem in lighting control technology concerns the different nomenclatures used by different manufacturers. Users are often particularly well-trained in at least one specific first nomenclature and can therefore use it especially effectively. If a second nomenclature is used in the setup to which the method is applied, it is advantageous if the language model is trained to enable a translation from the at least one first nomenclature into the second nomenclature.

[0052] Furthermore, it is conceivable that the voice command could be issued in any language or in several languages, with the language model translating them into a unified language. This makes it irrelevant which language the user speaks.

[0053] It is also conceivable that the procedure in step b. includes evaluating the voice command and / or at least one sub-task and prompting the user to enter another voice command if the validation yields a negative result. This allows the voice command to be interpreted at an early stage to determine whether all necessary information for its execution is known and / or unambiguous. In cases of ambiguity and / or missing information, this can easily be added by the user. The initial voice command and the subsequent voice command are particularly advantageously considered. In other words, the initial voice command represents a context described elsewhere as a previous voice command, which is taken into account by the language model. The prompt is preferably provided in writing and / or orally.

[0054] Furthermore, it is conceivable that obtaining at least one setting for at least one lighting state is achieved by accessing a database. This database contains various pieces of information about the at least one lighting object, in particular the at least one lighting device, including the type of lighting device, the available settings, how these settings can be adjusted, the position of the lighting device, any possible grouping with other lighting objects, and / or the possible orientation of the lighting device. In this way, the type of setting and its range can be quickly and easily determined.

[0055] It is assumed that the definitions and / or explanations of the above-mentioned terms apply to all aspects described below in this description, unless otherwise specified. In particular, aspects of a computer system or computer program product disclosed in connection with the method described above are possible embodiments of a computer system or computer program product described below, and aspects disclosed below in connection with a computer system or computer program product that are features of a method are conceivable features of the method described above.

[0056] According to the invention, a computer system is further proposed, wherein the computer system is configured to execute one of the methods described above, and the computer system comprises at least one interface for receiving the speech command to execute step a., a computer-readable storage medium for storing the speech model and the trained machine learning model for executing steps b. to d., and at least one data processing device configured to execute steps b. and d.

[0057] The advantages disclosed in connection with the method can be achieved using this computer system. Preferably, the computer system comprises at least one sensor interface and / or a transmitter interface for receiving a signal emitted by a sensor and / or a transmitter. More preferably, the interface includes a microphone and / or a keyboard. Furthermore, the computer system preferably includes an output interface by means of which a sequence of events and / or a control code can be transmitted to a lighting control system. More preferably, the computer system includes an output interface by means of which speech, in particular written or spoken speech, can be output. Most preferably, the output interface includes a screen and / or a loudspeaker.The computer system can be a physical computer system on site, a computer system connected via a remote communication interface, especially a cloud-based system, and / or a hybrid of these.

[0058] Furthermore, according to the invention, a computer program product comprising software code sections designed such that one of the previously described methods can be executed with at least one processor is proposed.

[0059] Further details, features, and advantages of the invention will become apparent from the following description of the preferred embodiments in conjunction with the dependent claims. The respective features can be implemented individually or in combination with one another. The invention is not limited to the embodiments. The embodiments are shown schematically in the figures. Identical reference numerals in the individual figures denote identical or functionally equivalent elements, or elements that correspond to one another with respect to their function.

[0060] Specifically, we show: Fig. 1: a flowchart of a process according to the invention; and Fig. 2: a computer system according to the invention.

[0061] Fig. Figure 1 shows a flowchart of a method according to the invention. A user 1 speaks a voice command 2, which is transmitted via a suitable interface 21 (see Figure 1). Fig. 2), for example, a microphone, is used. In step a. 101, the voice command 2 is fed to a language model 3. This is followed by the first substep of step b. 102, in which the voice command 3 is broken down by the language model 3 into three subtasks 4, and in a subsequent second substep 103 of step b., the three subtasks 4 are arranged in a sequence. In the subsequent first substep 104 of step c., at least one lighting feature 6 of three lighting objects 7, as well as their setting options 8, are obtained, for example, by reading a database and / or data stored in the lighting control system. The obtained lighting features 6 and the obtained setting options 8, along with the subtasks 4, are entered into the trained machine learning model 5, which evaluates and weights them in the subsequent second substep 105 of step c.This is followed by a third sub-step 106 of step c., in which the trained machine learning model 5 determines, by inference, one of the lighting objects 7 and at least one of the lighting features 6 of the specific lighting object 7 based on the sub-tasks 4, the lighting features 6 and the setting options 8, where this is now the specific lighting object 7 and the specific lighting feature 6.

[0062] In the subsequent first sub-step 107 of step d., the setting options 8 of the specific lighting object 7 are determined by the trained machine learning model 5 by inferring and / or creating control commands 9. Then, in a second sub-step 108 of step d., the determined control commands 9 are arranged in the correct sequence and the commands are implemented in a sequence plan 10, which is designed as control code.

[0063] In the first sub-step 109 of step f., it is then validated whether the sequence 10 is executable on the lighting control system. If not, the (sub-)steps from the first sub-step 107 of step d. are executed again. If the intended result of the control commands 9 has been achieved, the sequence 10 is executed in a step e. 110, and a comparison is made with sub-tasks 4 and / or the voice command 2 issued by user 1 in the second sub-step 111 of step f. User 1 is then asked for feedback. If the result does not correspond to sub-tasks 4, these are modified, and the (sub-)steps from the first sub-step 102 of step c. or from the second sub-step 105 of step c. are executed again, depending on where the discrepancy is located.If the result corresponds to the goal defined in subtasks 4 and / or the voice command 2 uttered by user 1, the procedure is terminated in a final step 112 by saving the flowchart 10.

[0064] Fig. Figure 2 shows a computer system 20 according to the invention. The computer system 20 comprises an interface 21 for receiving the speech command, a computer-readable storage medium 22 comprising the trained machine learning model 5 and the speech model 3, and a data processing device 23. The computer system 20 further comprises an output interface 24 for transmitting a sequence of events 10 and / or a control code to a lighting control panel and an output interface 25 by means of which feedback in speech form can be output to the user.

[0065] The following exemplary embodiments serve only to illustrate the invention. They are not intended to limit the subject matter of the patent claims in any way. Example 1

[0066] The following is a first example of the method according to the invention based on the Fig. 1 explained.

[0067] The voice command 2 uttered by user 1 is "Back of stage in Congo Blue."

[0068] In the first sub-step of step b. 102, a subtask 4 is interpreted from the voice command 2. Part of subtask 4 concerns the determination of the lighting objects 7 designed as lighting fixtures from the "back of the stage" section of voice command 2. The trained machine learning model 5 can, in particular, make a determination in the third sub-step 106 of step c. based on statistical relationships via the group name of lighting objects 7, which are grouped, for example, under the group name "Backtruss," and based on knowledge of the positions of the lighting objects 7 that can illuminate the back of the stage, and / or based on knowledge of the degrees of freedom of the lighting objects 7 that enable the illumination of the back of the stage.

[0069] For these lighting objects 7, suitable lighting characteristics 6 must be determined. This is another part of subtask 4, which is derived from the "Congo-Blue" part of the voice command 2. For this, the technical term Congo-Blue must first be interpreted, which can be done by the language model 3 in the second step b. 102 or by the trained machine learning model 5, particularly in the first substep 104 of step c. This can be done, in particular, by searching a database, such as gel pools, color preset pools, and / or user-created color presets, and / or by combining previously learned knowledge.

[0070] The trained machine learning model 5 creates the sequence plan 10 in the subsequent steps, a second sub-step 105 of step c. to a second sub-step 108 of step d., taking into account the setting options 8 of the lighting characteristics 6, such as in particular the color system used for the lighting objects 7, RGB, HSB or CIE and / or the alignment setting, such as pan / tilt or XYZ coordinates, and determining the sequence plan 10. In doing so, the setting options 8 of brightness, zoom, focus, iris, shape, rotation and / or position should be taken into account in particular.

[0071] To achieve a uniform final result in step e., the trained machine learning model 5 should take the timing of the control commands 9 into account in such a way that the brightness is initially reduced, if necessary, and then increased after the color is set. This is because the lighting objects 7 would have different brightness levels when changing colors, which could lead to an inconsistent image. Furthermore, the brightness should only be increased once the lighting objects 7 have reached the position and / or orientation specified in the sequence 10, so that no unwanted light cones pass over the stage. In other words, the trained machine learning model 5 should automatically pause after the position has been set, based on the positional delta of the necessary movement of the lighting objects 7 and the likely specific duration of this positional delta, before increasing the brightness.In particular, both the at least one lighting feature 6 and all the previous voice commands 2 should be taken into account as context in order to implement the desired intention in a conflict-free and verifiable manner. Example 2

[0072] The voice command 2 uttered by user 1 is "Room in red, please."

[0073] Essentially, the steps are identical to those in the first example. It is particularly important to emphasize that the trained machine learning model 5 and / or the language model 3 must also be able to deduce suitable lighting objects 7, designed as lighting fixtures, from the available data in the lighting control system, without a precise description of what the room is.

[0074] Here, the trained machine learning model 5 should also deduce that, if possible, the light from the lighting objects 7 should be less focused to illuminate such a large area. This is achieved by adjusting the zoom and / or focus, and thus the beam size, which necessitates setting further lighting features 6 while taking into account the required sequence of control commands 9. At this point, partially or completely repeating steps c. and d. is advantageous. Example 3

[0075] The voice command 2 uttered by user 1 is "Make the movements faster".

[0076] Essentially, the steps are identical to those in the first and second examples. However, to achieve the goal, an abstract lighting characteristic 6, such as the speed of moving lighting devices, must be modified to create the sequence 10. The lighting characteristic 6 of speed usually has no direct physical equivalent as a directly controllable lighting characteristic 6 of a real-world lighting device. Therefore, the abstract lighting characteristic 6 of speed must be modified in an existing control code and / or a previously existing sequence 10.

[0077] This can happen in various ways, depending on the lighting control system and its status. In particular, a sequence of actions 10 consisting of control commands 9 can be created, which increases the speed of these motion effects for all lighting scenes with motion effects. Additionally and / or alternatively, it should be recognized that a so-called Speedmaster exists, to which all these movements are assigned. Under these circumstances, the trained machine learning model 5 should interpret and implement the user input in such a way that the speed of this Speedmaster is increased. Example 4

[0078] The voice command 2 uttered by user 1 is "Move the Thunderstorm scene to sunset".

[0079] Essentially, the steps are identical to those in the first, second, and third examples. At this point, it's necessary to deduce what the input "scene thunderstorm and sunrise" means. Based on general knowledge of lighting control and the complete state of the lighting control system, the trained machine learning model 5 should infer that it should move lighting objects 7 of the type "cues," since the term "scene" is often used synonymously with "cue." The trained machine learning model 5 should then infer that "thunderstorm" refers, for example, to cue number five named "lightning and thunder" in the current cue list, and "sunrise" refers to cue number ten in this list, in which most of the lighting elements in the scene transition from yellow to red and then fade out.The machine learning model 5 obtains this knowledge from the state of the lighting control system and uses inference to generate a command that changes the order of the lighting objects 7 cue number five and cue number ten as part of the common lighting object 7 of a sequence that places cue number 5 after cue number ten. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 11,687,760 B2

[0004]

Claims

[1] Method for changing at least one lighting state by voice command (2), comprising the following steps: a. (101) Receiving a voice command (2) and entering the voice command (2) into a language model (3); b. (102) Interpreting and decomposing the speech command (2) into at least one subtask (4) and optionally (103) sorting the at least one subtask (4); c. (104) Obtaining at least one lighting feature (6), at least one lighting object (7), and at least one setting (8) of the at least one lighting feature (6) and (105, 106) Determining at least one lighting object (7) and at least one lighting feature (6) of the at least one specific lighting object (7) by inference with a trained machine learning model (5) using input of the at least one lighting feature (6), the at least one setting (8), and the at least one subtask (4); d. (107, 108) Creating a flow chart (10) by inference with the trained machine learning model (5) by inputting at least one specific lighting object (7), at least one specific lighting feature (6) and / or at least one setting option (8); e. (110) Changing at least one lighting state according to the schedule (10). [2] The method of claim 1, wherein the method comprises step f. (109, 111) after step d. (107, 108) and / or after step e. (110), wherein step f. (109, 111) comprises: f. (109, 111) Obtaining at least one modified lighting feature (6) of at least one modified lighting state and validating at least one modified lighting feature (6). [3] Method according to claim 2, wherein step f. (109, 111) further comprises: modifying the subtask (4), determining and / or the sequence plan (10) and repeating step b. (102, 103), c. (104, 105, 106), d. (107, 108), e. (110), and / or f. (109, 111) if the validation has a negative result. [4] Method according to any of the preceding claims, wherein the voice command (2) in step a. (101) is given orally and / or in writing. [5] Method according to any of the preceding claims, wherein the sequence (10) comprises an executable control code for a light control system and wherein the modification in step e. (110) is carried out by executing the control code. [6] Method according to one of the preceding claims, wherein in step d. (107, 108) the creation of the flow chart (10) is carried out by inference with the trained machine learning model (5) by inputting the at least one subtask (4). [7] Method according to one of the preceding claims, wherein in step b. (102, 103) the creation of the at least one subtask (4) is carried out taking into account context. [8] Method according to one of the preceding claims, wherein in step b. (102, 103) the creation of the at least one subtask (4) is carried out, in step c. (104, 105, 106) the determination of the at least one lighting object (7) and the at least one lighting feature (6) of the at least one specific lighting object (7) is carried out by means of inference with the trained machine learning model (5) and / or in step d. (107, 108) the creation of the sequence plan (10) is carried out by means of inference with the trained machine learning model (5) taking into account at least one further factor. [9] Method according to claim 8, wherein the further factor is a piece of music, a location feature, an additional state of at least one additional device, a light show and / or a sketch. [10] Method according to one of the preceding claims, wherein in step c. (104, 105, 106) at least one signal from at least one sensor and / or one transmitter is obtained and wherein in step c. (104, 105, 106) the at least one illumination object (7) is determined by inference with the trained machine learning model (5) with input of the at least one signal and / or in step d. (107, 108) the sequence plan (10) is created by inference with the trained machine learning model (5) with input of the at least one signal. [11] Method according to any of the preceding claims, wherein the voice command (2) is a control code in at least a first nomenclature and wherein step b. (102, 103) and / or step c. (104, 105, 106) comprises translating the control code from the at least one first nomenclature into a second nomenclature. [12] Method according to one of the preceding claims, wherein the method in step b. (102, 103) comprises evaluating the voice command (2) and / or the at least one subtask (4) and prompting the input of a further voice command (2) if the validation has a negative result. [13] Method according to one of the preceding claims, wherein the at least one setting option (8) of the at least one lighting state is obtained by calling a database. [14] Computer system (20) configured to perform one of the methods according to claims 1 to 13, comprising at least one interface (21) for receiving the speech command (2) for performing step a. (101), a computer-readable storage medium (22) for storing the speech model (3) and the trained machine learning model (5) for performing steps b. to d. (102, 103, 104, 105, 106, 107, 108) and at least one data processing device (23) configured for performing step b. (102, 103) and step d. (107, 108). [15] Computer program product comprising software code sections designed such that a method according to any one of claims 1 to 13 can be executed by at least one processor.

Images