Production control data creation method and production control data creation program

Abstract

The object of the present invention is to provide a method and a program for creating performance control data that accumulate the experience of engineers with lighting and audio equipment skills and create new control data based on that experience. [Solution] A method for creating performance control data that causes a computer to execute the following steps: an updated partial control data acquisition step for acquiring updated partial control data consisting of partial control data updated by a person; an adoption / rejection flag setting step for setting an adoption / rejection flag indicating that the updated partial control data has been adopted and that pre-update partial control data consisting of partial control data before the updated partial control data was updated has not been adopted; and a learning performance control data creation step for creating learning performance control data that includes at least the pre-update partial control data, the updated partial control data, and the adoption / rejection flag.

Description

[Technical Field]

[0001] The present invention relates to a technology for automatically generating, by a computer, production control data for controlling lighting and sound equipment that are linked to a performance and add production to the performance. [Background technology]

[0002] In order to effectively produce performances such as musical instrument playing, singing, musicals, and pantomime at concert halls and on stage, it is important to provide lighting and sound effects that correspond to the content of the performance, and to provide lighting and sound effects that are synchronized with the timing of changes in the content of the performance. To achieve lighting and sound effects, the engineers who create the control data for the lighting and sound equipment understand the concept of the performance, consider the effects of the lighting and sound production based on that concept, create control data to control the lighting and sound based on the results of that consideration, execute the control data in a three-dimensional virtual space, make adjustments based on the execution results, and make final adjustments on-site at the concert venue or stage. However, as these engineers age and young engineers are not being trained, there are concerns about a future shortage of human resources, and technology is being developed to allow computers to create this control data.

[0003] For example, Patent Document 1 discloses a technology in which a library containing lighting data is prepared in advance, the melody of music data is analyzed and divided into phrases, lighting patterns are extracted from the library based on the musical characteristics of each divided phrase, and control data is generated from the lighting patterns. In this way, control data is automatically generated by analyzing music data, reducing the effort required to create the control data.

[0004] Patent Document 2 discloses a technique for adjusting the control data (limited to lighting devices) generated for each phrase so that the control data changes smoothly when there are large changes between phrases. In this way, smooth changes in the control data between phases can add a natural dramatic effect. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent 3743079 [Patent Document 2] Patent Publication No. 2010-192155 Summary of the Invention [Problem to be solved by the invention]

[0006] Patent Documents 1 and 2 disclose technologies for having a computer create control data. However, these technologies do not create new control data based on the accumulated experience of engineers with expertise in lighting and audio equipment.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a performance control data creation method and a performance control data creation program that accumulate the experience of engineers skilled in lighting equipment and audio equipment and create new control data based on that experience. [Means for solving the problem]

[0008] The production control data creation method, which is an invention made to solve the above problems, is as follows: A method for creating performance control data that allows a computer to create performance control data that adds lighting and / or sound effects in conjunction with a performance, comprising: The performance is divisible into a plurality of sub-performances in time and / or space; The performance control data controls performance additional equipment including lighting equipment and / or audio equipment, the performance control data includes partial control data for controlling the performance additional equipment for each of the partial performances, an updated partial control data acquisition step of acquiring updated partial control data consisting of partial control data updated by a person; an adoption / rejection flag setting step for setting an adoption / rejection flag indicating that the updated partial control data has been adopted and indicating that pre-update partial control data consisting of partial control data before the updated partial control data has been updated has not been adopted; A learning performance control data creation step of creating learning performance control data including at least the pre-update partial control data, the post-update partial control data, and the adoption / rejection flag; A model generation step of generating a learning model based on the learning performance control data; a reception step for receiving performance information in order to create new performance control data; a control data creation step of creating new performance control data that is new performance control data based on the performance information received in the reception step and the learning model; The performance here is not limited to actual performances such as playing musical instruments at a concert hall or on stage, singing, musicals, pantomime, etc., but also includes rehearsals before the performance.

[0009] In this way, the production control data creation method is as follows: an adoption flag setting step of setting an adoption flag indicating that the updated partial control data has been adopted and indicating that the pre-update partial control data consisting of the partial control data before the updated partial control data has been updated has not been adopted; A learning performance control data creation step of creating learning performance control data including at least pre-update partial control data, post-update partial control data, and an adoption / rejection flag; The computer executes Engineers with expertise in lighting and audio equipment select and create performance control data, and this experience is created as a learning model, and new performance control data can be created based on this learning model.

[0010] In the production control data creation method according to the present invention, When partial control data is created by a person, it is preferable to have a computer execute a performance control data acquisition step, before the updated partial control data acquisition step, to acquire the performance control data including all of the partial control data at that point in time. In this way, it is possible to identify partial control data that has not been manually updated among all partial control data, and to create a learning model more efficiently.

[0011] The production control data creation method according to the present invention comprises: The performance control data is data for controlling the audio equipment. Before creating the learning performance control data, a feature extraction step of acquiring acoustic information output from an acoustic device and extracting an acoustic feature, which is at least one feature of loudness, pitch, and speed of change of sound; Let the computer run In the step of creating learning performance control data, a computer is caused to create learning performance control data including acoustic features; In the receiving step, the computer is caused to receive a performance including sound output for a sound check; In the control data creation step, it is preferable to cause a computer to input acoustic features extracted from the acoustic information output from the acoustic equipment for the performance accepted in the acceptance step into a learning model to create new performance control data. Here, a sound check refers to testing the acoustics, and the sound output includes sounds with a melody, sounds without a melody such as a single note played on an instrument, and sounds from the same stage or performance as in the actual performance. In this way, learning performance control data including acoustic features extracted from the acoustic information output from the acoustic equipment is created, and in the control data creation step, the acoustic features extracted based on the acoustic information output from the acoustic equipment for the performance accepted in the reception step are input into a learning model to create new performance control data.This allows for accumulated experience in adjusting the control data for the acoustic equipment based on the actual output from the acoustic equipment on site, and new performance control data for the acoustic equipment can be created based on that experience.

[0012] In the production control data creation method according to the present invention, a feature extraction step of acquiring a lighting image output from a lighting performance preparation terminal that prepares performance control data or the lighting device, and extracting a lighting image feature, which is at least one feature of light color, brightness, and movement; on the computer, In the learning performance control data creation step, it is preferable to create learning performance control data that includes the lighting image feature amount. In this way, lighting images output from the lighting performance preparation terminal or lighting equipment are acquired, and lighting image features, which are at least one of the features of light color, brightness, and movement, are extracted to create performance control data for learning.Therefore, the experience of an engineer creating control data taking into account the lighting images output from the lighting performance preparation terminal or lighting equipment is created as a learning model.

[0013] In the production control data creation method according to the present invention, In the control data creation step, it is preferable to input data including lighting image features acquired for the performance accepted in the acceptance step into the learning model to create new performance control data. In this way, new performance control data can be created based on a learning model that reflects the experience of the engineers mentioned above.

[0014] In the production control data creation method according to the present invention, a feature extraction step of acquiring audio information output from the lighting performance preparation terminal or the audio equipment and extracting an audio feature, which is at least one of the feature of the loudness, pitch, and speed of change of the sound; on the computer, It is preferable that the learning performance control data creating step creates learning performance control data including the acoustic feature. In this way, the acoustic information output from the lighting performance preparation terminal or the audio equipment is acquired, and audio features, which are at least one of the features of sound color, brightness, and movement, are extracted to create performance control data for learning.Therefore, the experience of an engineer creating control data taking into account the acoustic information output from the lighting performance preparation terminal or the audio equipment is created as a learning model.

[0015] In the performance control data creation method of the present invention, it is preferable that in the control data creation step, data including acoustic features obtained for a newly input performance is input into the learning model to create new performance control data. In this way, new performance control data can be created based on a learning model that reflects the experience of the engineers mentioned above.

[0016] The production control data creation program, which is an invention made to solve the above problems, A production control data creation program that causes a computer to create production control data that adds lighting and / or sound effects in conjunction with a performance to be performed, The performance is divisible into a plurality of sub-performances in time and / or space; The performance control data controls performance additional equipment including lighting equipment and / or audio equipment, the performance control data includes partial control data for controlling the performance additional equipment for each of the partial performances, a performance control data acquisition step of acquiring the performance control data including all of the partial control data at a time when the partial control data is created by a person; Following the performance control data acquisition step, an updated partial control data acquisition step is performed to acquire updated partial control data consisting of the partial control data updated by the person. an adoption / rejection flag setting step for setting an adoption / rejection flag indicating that the updated partial control data has been adopted and indicating that pre-update partial control data consisting of partial control data before the updated partial control data has been updated has not been adopted; A learning performance control data creation step of creating learning performance control data including at least the pre-update partial control data, the post-update partial control data, and the adoption / rejection flag; A model generation step of generating a learning model based on the learning performance control data; a reception step for receiving performance information in order to create new performance control data; a control data creation step of creating new performance control data that is new performance control data based on the performance information received in the reception step and the learning model; The present invention is characterized in that the computer executes the following.

[0017] In this way, the performance control data creation program an adoption / rejection flag setting step for setting an adoption / rejection flag indicating that the updated partial control data has been adopted and indicating that pre-update partial control data consisting of partial control data before the updated partial control data has been updated has not been adopted; A learning performance control data creation step of creating learning performance control data including at least the pre-update partial control data, the post-update partial control data, and the adoption / rejection flag; The computer executes Engineers with expertise in lighting and audio equipment select and create control data, and this experience is created as a learning model, and new performance control data can be created based on this learning model. [Effects of the Invention]

[0018] As described above, according to the performance control data creation method and performance control data creation program of the present invention, it is possible to accumulate the experience of engineers skilled in lighting equipment and audio equipment, and create new performance control data based on that experience. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a diagram schematically illustrating a configuration of a lighting design support system according to an embodiment of the present invention. [Figure 2] 2 is a functional block diagram of a lighting performance creation device, a lighting operation terminal, and a lighting performance preparation terminal of FIG. 1. FIG. [Figure 3] 3 is a diagram showing the performance control data and updated partial control data stored in the storage unit of FIG. 2. FIG. [Figure 4] FIG. 10 is a diagram showing a schematic diagram of the process of creating learning performance control data. [Figure 5] 3 is a diagram showing learning performance control data stored in the memory unit of FIG. 2. [Figure 6] FIG. 10 is a diagram showing a schematic diagram of the process of creating new performance control data. [Figure 7] FIG. 2 is a diagram illustrating an input to a learning model and an output from the learning model. [Figure 8] FIG. 3 is a diagram schematically illustrating a learning model stored in a storage unit in FIG. 2. [Figure 9] 3 is a diagram showing a correspondence table between features and control methods stored in the storage unit of FIG. 2. FIG. [Figure 10] 2 is a flowchart of the lighting performance creation device of FIG. 1. [Figure 11] 11 is a flowchart of the model generation step of FIG. 10. [Figure 12] 11 is a flowchart of the effect creation step of FIG. 10. [Figure 13] A figure showing performance control data in an audio performance support system according to a modified example of one embodiment of the present invention. [Figure 14] A figure showing learning performance control data in an audio performance support system according to a modified example of one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, one embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the following embodiment, and appropriate modifications can be made without departing from the spirit and scope of the present invention.

[0021] (Lighting production support system 100) 1 shows a lighting performance support system 100 according to one embodiment of the present invention. The lighting performance support system 100 mainly comprises a lighting performance creation device 10, a lighting operation terminal 20 (also called a lighting console), lighting devices 30 (corresponding to performance addition devices), and a lighting performance preparation terminal 40.

[0022] Although not the main components, the lighting performance support system 100 includes an audio operation terminal 20A (also called an audio console) and audio equipment 30A.

[0023] The lighting performance creation device 10 creates performance control data 16A that adds lighting and / or sound effects in conjunction with the performance being performed. The lighting operation terminal 20 operates the lighting devices 30 using the performance control data 16A. The lighting devices 30 are controlled by the performance control data and have lighting effects such as light color, brightness, and movement. The lighting performance preparation terminal 40 prepares performance control data 46A for operating the lighting devices 30.

[0024] A performance is configured so that it can be divided into multiple partial performances in time and / or space. For example, a performance is divided into partial performances such as a time period from 0 to 10 seconds after the start, or a time period from 10 to 20 seconds after the start. Partial performances are assigned consecutive numbers from the start of the performance. A partial performance with a consecutive number one smaller is the preceding partial performance (hereinafter referred to as the preceding partial performance BP), and a partial performance with a consecutive number one larger is the succeeding partial performance (hereinafter referred to as the succeeding partial performance AP).

[0025] The performance control data 16A controls additional performance equipment consisting of lighting equipment 30 and / or sound equipment 30A, and includes partial control data for controlling the additional performance equipment for each partial performance.

[0026] The sound equipment 30A is equipment that adds effects to a performance in conjunction with the performance being performed, and outputs sound at a concert hall or the like. The sound operation terminal 20A is a terminal that operates the sound equipment 30A. The lighting performance support system 100 creates performance control data 16A that controls the lighting equipment 30, taking into consideration the sound output by the sound equipment 30A (details will be described later).

[0027] When creating the production control data for controlling the sound equipment 30A, it is preferable to control it in conjunction with the space (location) in which the performance is performed, and it is preferable that the performance be divided into partial performances for each space (location) in which it is performed. An audio production support system 100A that creates the production control data for controlling the sound equipment 30A will be described later as a modified example of the lighting production support system 100.

[0028] (Configuration of lighting effect creation device 10) 2(a), lighting performance creation device 10 includes an acquisition unit 11, an adoption / rejection flag setting unit 12 (adoption / rejection flag setting step), a learning data creation unit 13, a model generation unit 14, a control data creation unit 15, a storage unit 16, and a communication unit 17. Lighting performance creation device 10 has the function of generating a learning model based on performance control data for learning, and creating new performance control data based on the learning model. The functions of lighting performance creation device 10 will be explained separately for a learning mode (learning model generation) and an inference mode (inference using the learning model).

[0029] (Learning mode) The acquisition unit 11 includes a performance control data acquisition unit 111 (performance control data acquisition step), an updated partial control data acquisition unit 112 (updated partial control data acquisition step), and a feature amount acquisition unit 113. When partial control data is manually created, the performance control data acquisition unit 111 acquires performance control data 16A including all partial control data at that time. The performance control data 16A is data for controlling the lighting devices 30 (see FIG. 3), and will be described in detail later. After the performance control data acquisition unit 111 acquires all the partial control data at that time, the updated partial control data acquisition unit 112 acquires updated partial control data UAC consisting of partial control data updated by a person. The "person" mentioned above is a person who prepares the performance control data using the lighting performance preparation terminal 40 (hereinafter referred to as a preparer) or a person who operates the lighting device 30 using the lighting operation terminal 20 (hereinafter referred to as an operator).

[0030] As described below, the lighting operation terminal 20 is equipped with an imaging unit 24 and a listening unit 25, where the imaging unit 24 captures the lighting output from the lighting device 30, and the listening unit 25 listens to the sound output from the audio device 30A. As described below, the lighting performance preparation terminal 40 is equipped with a photographing unit 44 and a listening unit 45, and the photographing unit 44 photographs the lighting output on the screen of the lighting performance preparation terminal 40, and the listening unit 45 listens to the sound output to the headset or speaker of the lighting performance preparation terminal 40.

[0031] The feature acquisition unit 113 acquires lighting image information captured by the photographing unit 24 of the lighting operation terminal 20 or the photographing unit 44 of the lighting performance preparation terminal 40, and extracts lighting image features, which are features related to the color, brightness, and movement of light. The feature acquisition unit 113 extracts acoustic features, which are features related to the loudness, pitch, and speed of change of sound, from the acoustic information heard by the listening unit 25 of the lighting operation terminal 20 or the listening unit 45 of the lighting performance preparation terminal 40. The lighting image features and sound features are added to the data of the learning performance control data 16B described later (see Figure 4(e)), and are added to the data input into the learning model described later (see Figures 7(a) and (b)).

[0032] The adoption / rejection flag setting unit 12 sets an adoption / rejection flag A based on the performance control data 16A acquired by the performance control data acquisition unit 111 and the updated partial control data UAC acquired by the updated partial control data acquisition unit 112. The adoption / rejection flag setting unit 12 sets an adoption / rejection flag indicating that the updated partial control data UAC has been adopted (for example, 1) and that the pre-update partial control data UBC consisting of the partial control data before the updated partial control data was updated has not been adopted (for example, 0). Furthermore, for the partial control data that has not been updated, the adoption / rejection flag setting unit 12 sets an adoption / rejection flag A to a value (for example, 1) meaning that the partial control data has been adopted. If the updated partial control data acquisition unit 112 does not acquire the updated partial control data UAC for a certain period of time after the performance control data acquisition unit 111 acquires the performance control data 16A, the adoption / rejection flag setting unit 12 sets the adoption / rejection flag A to a value (for example, 1) which means that the partial control data of the performance control data 16A has been adopted by the performance control data acquisition unit 111. The adoption / rejection flag A set by the adoption / rejection flag setting unit 12 is added to the learning performance control data 16B, which will be described later.

[0033] The learning data creation unit 13 creates learning performance control data 16B (see FIG. 4) including the pre-update partial control data UBC, the post-update partial control data UAC, and the adoption / rejection flag A. When one partial performance and another partial performance are adjacent in time or space, the learning data creation unit 13 creates learning performance control data 16B including partial control data of the one partial performance and partial control data of the other partial performance. The adoption / rejection flag setting unit 12 and the learning data creating unit 13 execute the following steps to create learning performance control data 16B (see FIG. 4).

[0034] FIG. 4(a) shows a simplified representation of performance control data 16A, which is input data used by learning data creation unit 13 to create learning performance control data 16B, and updated partial control data UAC.

[0035] FIG. 4(b) shows the learning performance control data (Step 1) created in Step 1. Learning performance control data (Step 1) is information on partial performance (features and control methods) This is data to which the adoption / rejection flag A set by the adoption / rejection flag setting unit 12 has been added. If the partial performances and their control methods are written in parentheses, (intro, colors: deep blue, purple, dark white), (verse, colors: calm blue, cool white), and (chorus, colors: blue, mainly white) were not updated, so the adoption / rejection flag setting unit 12 sets the adoption / rejection flag to 1. Partial control data that has not been updated by humans, such as (intro, colors: deep blue, purple, dark white), (verse, colors: calm blue, cool white), (chorus, colors: blue, white-based), is called unupdated partial control data. The B-melody of the partial performance changes from "Color: bluish white." to "Color: bluish white." → It changes to pure white or slightly yellowish white. " (B melody, color: Bluish white ) is the partial control data UBC before updating, and (B-melody, color: bluish white → pure white or slightly yellowish white) is the partial control data UAC after updating. The adoption / rejection flag setting unit 12 determines whether (B-melody, color: Bluish white . )'s adoption flag A is set to 0, and (B melody, color: changes from bluish white to pure white or slightly yellowish white.)'s adoption flag A is set to 1.

[0036] Figure 4(c) shows the learning production control data (step 2) created in step 2. The learning production control data (step 2) is data in which information on the preceding performance part BP and information on the succeeding performance part AP have been added to the learning production control data (step 1). For example, for the bridge verse, the preceding performance part is (verse verse, colors: calm blue, cool white) and the succeeding performance part is (chorus, colors: blue, primarily white), so the learning production control data (step 2) is data in which information on the respective partial performances (verse verse, colors: calm blue, cool white) and (chorus, colors: blue, primarily white) has been added.

[0037] FIG. 4(d) shows the learning production control data (Step 3) created in Step 3. The learning production control data (Step 3) is data in which program information T, performer information PL, and lighting information L are added to the learning production control data (Step 2). The program information T, performer information PL, and lighting information L are added to create learning production control data 16B so that learning model 16C can learn about various musical pieces. Program information T is information about the program that is the subject of the performance, performer information PL is information indicating how the performer will perform the performance, and lighting information L is information indicating how the performance will be lit.

[0038] Fig. 4(e) shows the learning performance control data (step 4) created in step 4. The learning performance control data (step 4) is data in which lighting image features and sound features are added to the learning performance control data (step 3). The addition of lighting image features and sound features will be explained below. The preparer prepares performance control data 46A on lighting performance preparation terminal 40. Lighting performance preparation terminal 40 simulates performance control data 46A, outputs it to a drawing (not shown), and transmits lighting image information captured by photographing unit 24 to lighting performance creation device 10. When simulating performance control data 46A, lighting performance preparation terminal 40 also plays music, and transmits audio information listened to by listening unit 25 to lighting performance creation device 10. The operator controls the lighting device 30 using the performance control data 26A on the lighting operation terminal 20, and controls the audio device 30A using the control data on the audio operation terminal 20A. The lighting operation terminal 20 transmits lighting image information captured by the imaging unit 24 and audio information heard by the listening unit 25 to the lighting performance creation device 10. The feature acquisition unit 113 extracts lighting image features and extracts sound features. The learning data creation unit 13 adds the lighting image features and / or sound features to the training performance control data. Note that the process of adding the lighting image features and / or sound features to the training performance control data by the learning data creation unit 13 may be performed before the adoption / rejection flag setting unit 12 sets the adoption / rejection flag A.

[0039] 5 shows details of the learning performance control data 16B. The learning performance control data 16B includes program information T, performer information PL, lighting information L, information on the preceding performance portion BP, information on the following performance portion AP, information on the relevant performance portion P, and an acceptance / rejection flag A.

[0040] Model generation unit 14 separates learning performance control data 16B created by learning data creation unit 13 into input data (other than adoption / rejection flag A) and teacher data (adoption / rejection flag A), and causes learning model 16C to learn the data.

[0041] (inference mode) The control data creating unit 15 includes a performance dividing unit 151 and a control method determining unit 152, and creates new performance control data 16E by executing the following steps (see FIG. 6). First, in order to create new performance control data 16E, reception unit 21 of lighting operation terminal 20 or reception unit 41 of lighting performance preparation terminal 40 receives a performance.

[0042] FIG. 6(a) shows the new production control data (Step 1) created in Step 1. The performance dividing unit 151 acquires information (program information T, performer information PL, lighting information E, song) of the received performance (hereinafter referred to as new performance NP). The performance dividing unit 151 divides the song, which is the information of the new performance NP, into partial performances, extracts the characteristics of each partial performance, and creates new production control data 16E (Step 1). In the case of a song, the characteristics of the partial performances include not only information identifying phrases (intro, A melody, B melody, chorus, interlude, C melody, outro), but also the characteristics of the phrases (for example, a calm intro or an A melody that gets exciting in the second half).

[0043] Fig. 6(b) shows the new performance control data (step 2) created in step 2. The performance dividing unit 151 refers to the correspondence table 16D (see Fig. 9) between the characteristics of partial performances and the control methods, initializes the control method for each partial performance, and creates the new performance control data (step 2).

[0044] FIG. 6(c) shows the new performance control data (step 3) created in step 3. The control method determination unit 152 creates input data for the learning model 16C (details will be described later). Next, the control method determination unit 152 determines a control method for the partial control data based on the acceptance degree AD, which indicates the degree to which the performance should be adopted as a performance, and creates new performance control data (step 3). The control method determination unit 152 repeats the following process for each partial performance, from category 1 to the end.

[0045] The creation of input data for the learning model 16C will now be described. The control method determination unit 152 creates input data for the learning model 16C by referring to the new production control data (step 2) (see FIG. 6(b)). The input data for the learning model 16C is composed of program information T, performer information PL, lighting information L, information on the preceding partial performance BP, information on the following partial performance AP, and information on the partial performance P. The information on the partial performance is composed of features and a control method.

[0046] FIG. 7(a) shows, as an example, input data to the learning model 16C for the partial performance (B melody) of Section 3. The program information T, performer information PL, and lighting information L are the same as the information registered in the new production control data 16E. The preceding partial performance BP is information for the partial performance (A melody) of Section 2, and the succeeding partial performance AP is information for the partial performance (chorus) of Section 4. The relevant partial performance P is information for the partial performance (B melody) of Section 3. The control method determination unit 152 determines an appropriate control method from among the "plurality of assumed control methods" for the partial performance P. The "plurality of assumed control methods" are control methods pre-stored in the storage unit 16, and may be control methods for all lighting devices 30, or control methods for each feature of the partial performance (for example, each intro or verse). Both are stored in the storage unit 16. The control method determination unit 152 sequentially selects control methods from the "plurality of assumed control methods" and inputs them into the learning model 16C, and determines the control method based on the acceptance rating AD output from the learning model 16C. The control method determination unit 152 refers to the acceptance degree AD output from the learning model 16C, and prioritizes the control method with a larger value of the acceptance degree AD to determine it as the control method for the relevant partial performance P. In the example shown in Fig. 7(b), the acceptance degree AD of the control method "Color: change from bluish white to pure white or slightly yellowish white" is larger than that of the control method "Color: bluish white", so the control method "Color: change from bluish white to pure white or slightly yellowish white" is prioritized and determined as the control method for the relevant partial performance P.

[0047] The control method determination unit 152 determines the control method for each partial performance in this order, from section 1 to the end. In this way, new production control data 16E can be created that takes into account the know-how of referencing the control method of the preceding partial performance BP. After this series of processes, the control method for each partial performance is determined in reverse order (from the last section to section 1). In this way, new production control data 16E can be created that takes into account the control method for the subsequent partial performance AP. The control method determination unit 152 may repeat a series of processes from section 1 to the end and a series of processes from the last section to section 1 multiple times.

[0048] The acceptance degree AD is a value between 0 and 1. A control method with an acceptance degree AD close to 1 is a control method that has been adopted up to now (a conventional control method), and a control method with an acceptance degree AD close to 0 is a control method that has not been adopted up to now. The control method determination unit 152 may select the control method with the highest value of the degree of adoption AD from among the control methods, or may select the second or third control method. If there is no control method that results in a value of the acceptance / rejection degree AD greater than a predetermined value (for example, 0.5) (for a song for which no previous know-how exists), this fact may be output. For example, a case in which the acceptance / rejection degree AD has a small value would be a song that has never been played before, where the chorus starts immediately after the intro without a verse.

[0049] In the inference mode, as in the learning mode, the lighting performance preparation terminal 40 transmits lighting image information captured by the photographing unit 24 to the lighting performance creation device 10. The lighting performance preparation terminal 40 transmits sound information heard by the listening unit 25 to the lighting performance creation device 10. The lighting operation terminal 20 transmits the lighting image information captured by the photographing unit 24 and the sound information heard by the listening unit 25 to the lighting performance creation device 10. The feature acquisition unit 113 extracts features from the lighting image information and sound information. The control data creation unit 15 inputs data including the lighting image feature and the sound feature acquired by the feature acquisition unit 113 into a learning model (see FIG. 7(a)), and creates new performance control data 16E. The learning mode and the inference mode have been explained above. Finally, the storage unit 16 and the communication unit 17 will be explained.

[0050] The storage unit 16 stores performance control data 16A, learning performance control data 16B, a learning model 16C, a correspondence table 16D between features and control methods, and new performance control data 16E.

[0051] The production control data 16A is made up of program information T, performer information PL, lighting information L, and partial control data C, as shown in FIG. As described above, program information T is information about the program that is the subject of the performance, performer information PL is information indicating how the performers will perform the performance, and lighting information L is information indicating how the performance will be lit. For example, if the program is a ballad (slow and quiet), the lighting information L may be "A: lonely, dark," "B: sad, painful," "C: loneliness, alone," "D: separation, goodbye," or "E: rock bottom, misery." If the program information is a fast-paced (intense, bright), the lighting information L may be "A: fun, happy," "B: uplifting, turns depression into bright," "C: makes you want to dance, makes you forget your sadness," "D: crazy, intense," or "E: forget everything, extraordinary." The partial control data C includes data on the category, time period, features, control method, lighting image features, and sound features for each partial performance. The category is an identifier (serial number) that identifies the partial performance. The time period is the start time and end time of the partial performance. The feature is the feature of the partial performance. The lighting performance creation device 10 may analyze the content of the partial performance and extract its features, or the preparer or operator may extract the features. The control method is a specific method for controlling the lighting equipment 30, such as the color, brightness, and movement of the light. The lighting image feature and the sound feature are, as described above, information extracted by the feature acquisition unit 113 from the lighting image information and sound information.

[0052] As shown in FIG. 5, the learning performance control data 16B includes program information T, performer information PL, lighting information L, information on the preceding performance portion BP, information on the following performance portion AP, information on the performance portion P, and an acceptance / rejection flag A.

[0053] The information of the preceding partial performance BP includes the characteristics of the partial performance, the control method, the lighting image features, and the sound features. The lighting image features and the sound features are as described above. The information of the subsequent partial performance AP and the information of the partial performance P are also the same.

[0054] As mentioned above, the adoption flag A is information indicating whether the partial control data has been adopted or not, and is either 0 (a value meaning that the partial control data has not been adopted) or 1 (a value meaning that the partial control data has been adopted). An example where the adoption flag A is "0" is shown in Fig. 5(1), and an example where the adoption flag A is "1" is shown in Fig. 5(2).

[0055] As shown in Fig. 8(a), learning model 16C receives as input program information T, performer information PL, lighting information L, information on the preceding partial performance BP, information on the following partial performance AP, and information on the relevant partial performance P, and outputs the degree of success AD of the control method for the relevant partial performance. Fig. 8(b) is a diagram schematically illustrating learning model 16C. The left end is input data, and the right end is output data.

[0056] The communication unit 17 receives performance control data 16A or performance control data 46A from the lighting operation terminal 20 or the lighting performance preparation terminal 40. The communication unit 17 receives updated partial control data UAC from the lighting operation terminal 20 or the lighting performance preparation terminal 40. The communication unit 17 receives information about a new performance NP from the lighting operation terminal 20 or the lighting performance preparation terminal 40. The communication unit 17 transmits new performance control data 16E about the new performance NP to the lighting operation terminal 20 or the lighting performance preparation terminal 40.

[0057] (Configuration of lighting operation terminal 20) The lighting operation terminal 20 is a terminal that stores performance control data 26A for controlling the lighting devices 30 and operates the lighting devices 30 using the performance control data 26A. The performance control data 26A stored in the lighting operation terminal 20 (storage unit 26) is substantially the same as the performance control data 16A stored in the lighting performance creation device 10 (storage unit 16). The performance control data 26A does not include lighting image features or sound features.

[0058] As shown in FIG. 2(b), the lighting operation terminal 20 includes a reception unit 21, a control unit 22, an update unit 23, an image capture unit 24, a listening unit 25, a storage unit 26, and a communication unit 27.

[0059] The reception unit 21 receives information about a new performance NP (program information T, performer information PL, lighting information L, and music) via input from a screen (not shown). The program information T, performer information PL, and lighting information L are text information, and the music is sound information (file format: WAVE or MP3, for example). The control unit 22 controls the lighting devices 30 using the performance control data 26A (stored in the storage unit 26). The operator operates the lighting devices 30 using the control data on the lighting operation terminal 20. The operator may operate the lighting devices 30 while moving around the audience seats, or may operate them from a control room (not shown) that overlooks the performance. The operator operates the audio devices 30A using the audio operation terminal 20A.

[0060] The update unit 23 updates the partial control data C of the performance control data 26A from a screen (not shown). For example, the lighting color is updated from "Color: Blue, white tone" to "Color: Bluish white → Changes to pure white or slightly yellowish white." Every time the partial control data C is updated, the update unit 23 transmits the updated partial control data UAC to the lighting performance creation device 10.

[0061] The photographing unit 24 photographs the illumination output from the lighting device 30. The listening unit 25 listens to the sound output from the audio device 30A.

[0062] The storage unit 26 stores the performance control data 26A. The program information T, performer information PL, and lighting information L of the performance control data 26A are assumed to be registered in advance by a preparer or an operator and stored in the storage unit 26.

[0063] The communication unit 27 transmits the performance control data 26A stored in the storage unit 26 to the lighting performance creation device 10, and transmits the updated partial control data UAC updated by the update unit 23 to the lighting performance creation device 10. The communication unit 27 transmits the images captured by the image capturing unit 24 to the lighting performance creation device 10 as lighting image information, and transmits the sounds heard by the listening unit 25 to the lighting performance creation device 10 as sound information. The communication unit 27 transmits the information on the new performance NP received by the reception unit 21 to the lighting performance creation device 10. The communication unit 27 receives new performance control data 16E created by the lighting performance creation device 10 from the lighting performance creation device 10. The communication unit 27 has a function of communicating with the lighting effect preparation terminal 40 .

[0064] (Configuration of lighting effect preparation terminal 40) The lighting performance preparation terminal 40 is a terminal that prepares performance control data 46A for controlling the lighting devices 30, and is usually installed in a location other than the concert venue (for example, an office where a preparer prepares the performance control data 46A). The performance control data 46A stored in the lighting performance preparation terminal 40 is data in the same format as the performance control data 26A stored in the lighting operation terminal 20.

[0065] As shown in FIG. 2( c ), the lighting performance preparation terminal 40 includes a reception unit 41 , a control unit 42 , an update unit 43 , a photographing unit 44 , a listening unit 45 , a storage unit 46 , and a communication unit 47 . The reception unit 41 has the same functions as the reception unit 21.

[0066] The control unit 42 does not actually control the lighting devices 30, but simulates the control of the lighting devices 30. The preparer creates concert venue data (two-dimensional or three-dimensional) and lighting device 30 data (positions of lighting devices and lighting methods) in the memory unit 46 of the lighting performance preparation terminal 40 in advance. The control unit 42 simulates performance control data 46A and outputs it to a screen (not shown). The control unit 42 also plays music data stored in the memory unit 46 and outputs it to a speaker or headset (not shown). The preparer may create data for the audio devices 30A (positions of audio devices and audio methods) in the memory unit 46 in advance, and the control unit 42 may simulate control data for controlling the audio devices 30A. In this way, audio information similar to that of the actual venue, such as a concert venue, can be output to a speaker or headset (not shown).

[0067] The update unit 43 has the same functions as the update unit 23. The photographing unit 44 photographs a lighting image to be output on a screen (not shown) by the control unit 42. The listening unit 45 acquires acoustic information to be output to a speaker or headset (not shown) by the control unit 42. The memory unit 46 has the same equipment as the memory unit 26. The communication unit 47, like the communication unit 27, has a communication function with the lighting performance creation device 10. The communication unit 47 has a communication function with the lighting operation terminal 20.

[0068] (Actions and Effects of the Lighting Production Support System 100) Fig. 10 shows a flowchart of lighting performance creation device 10 of Fig. 1. Lighting performance creation device 10 generates a learning model (model generation step S100) and then performs a performance creation process (performance creation step S200). The former is a learning mode, and the latter is an inference mode.

[0069] (Learning mode) 11 is a detailed flowchart of the model generation step S100. The performance control data 26A is created by an operator and stored in the storage unit 26. When the operator creates the partial control data, the performance control data 26A includes all the partial control data at that time. The operator operates the lighting device 30 using the lighting operation terminal 20, and operates the audio device 30A using the audio operation terminal 20A. The lighting operation terminal 20 controls the lighting devices 30 using the performance control data 26A (control step S101). The lighting operation terminal 20 captures the light output by the lighting devices 30 and creates lighting image information (capturing step S102). The lighting operation terminal 20 listens to the sound output by the audio device 30A and creates audio information (listening step S103). It is preferable that the capturing step S102 and the listening step S103 are performed simultaneously. The lighting operation terminal 20 transmits the performance control data 26A to the lighting performance creation device 10, and also transmits the lighting image information and sound information to the lighting performance creation device 10 (lighting image information and sound information transmission step S104).

[0070] Lighting performance creation device 10 (feature acquisition unit 113) acquires lighting image information and extracts lighting image features, which are features related to the color, brightness, and movement of light, and acquires audio information and extracts audio features, which are features related to the loudness, pitch, and speed of change of sound. Lighting performance creation device 10 stores the received performance control data, together with the lighting image features and the audio features, as performance control data 16A in storage unit 16 (performance control data acquisition step S105). Note that if lighting performance creation device 10 does not receive updated partial control data UAC from lighting operation terminal 20 after a certain period of time has elapsed since receiving the performance control data, lighting performance creation device 10 may create learning performance control data 16B using only the received performance control data.

[0071] The lighting operation terminal 20 operates the lighting devices 30 using the updated partial control data UAC updated by the operator (control data update step S106). The operator operates the audio device 30A using the audio operation terminal 20A. The lighting operation terminal 20 creates lighting image information and audio information (photographing and listening step S107). The lighting operation terminal 20 transmits the updated partial control data UAC, the lighting image information, and the audio information (updated partial control data transmission step S108).

[0072] The lighting performance creation device 10 receives the updated partial control data UAC, lighting image information, and sound information from the lighting operation terminal 20 (updated partial control data acquisition step S109).

[0073] The lighting performance creation device 10 (feature acquisition unit 113) acquires lighting image features and sound features, and the lighting performance creation device 10 (learning data creation unit 13) creates learning performance control data 16B based on the performance control data 16A, updated partial control data UAC, lighting image features, and sound features (learning performance control data creation step S110).The lighting performance creation device 10 (model generation unit 14) generates a learning model 16C based on the learning performance control data 16B (learning model generation step S111).

[0074] The lighting operation terminal 20 returns to S106 (return S112). The lighting operation terminal 20 may return to S101.

[0075] (inference mode) 12 is a detailed flowchart of the effect creation step S200. The inference mode is made up of a first inference mode (S201 to S206) and a second inference mode (S207 to S216).

[0076] (First inference mode) The lighting operation terminal 20 receives information about the new performance NP (program information T, performer information PL, lighting information L, and music) and transmits it to the lighting performance creation device 10 (new performance reception step S201).

[0077] The lighting performance creating device 10 receives the information of the new performance NP (program information T, performer information PL, lighting information L, and music) (new performance receiving step S202). The lighting performance creating device 10 analyzes the received sound data of the song, divides it into partial performances, and extracts features for each partial performance (dividing step S203). The lighting performance creating device 10 determines a control method for each partial performance based on the learning model 16C, and creates new performance control data 16E (control method determining step S204). The lighting performance creation device 10 transmits the new performance control data 16E to the lighting operation terminal 20 (new performance control data transmission step S205).

[0078] The lighting operation terminal 20 receives the new performance control data 16E and stores it in the storage unit 26 as new performance control data 26E (new performance control data receiving step S206).

[0079] (Second inference mode) The lighting operation terminal 20 uses the new performance control data 26E to perform a control step S207, a shooting step S208, a listening step S209, and a lighting image information and sound information transmission step S210. These steps are the same as steps S101, S102, S103, and S104, and therefore will not be described here.

[0080] The lighting performance creation device 10 receives lighting image information and sound information (lighting image information and sound information receiving step S211). The lighting performance creation device 10 (feature amount acquisition unit 113) extracts lighting image feature amounts and sound feature amounts (feature amount extraction step S212).

[0081] The lighting performance creation device 10 (control data creation unit 15) inputs the data including the lighting image feature and the sound feature acquired by the feature acquisition unit 113 into the learning model, determines a control method, and creates new performance control data 16E (control method determination step S213). The lighting performance creation device 10 transmits the new performance control data 16E to the lighting operation terminal 20 (new performance control data transmission step S214).

[0082] The lighting operation terminal 20 receives the new performance control data 16E and stores it in the storage unit (new performance control data receiving step S215). The lighting operation terminal 20 returns to S207 (return S216).

[0083] (Variation) When partial control data is updated, the lighting operation terminal 20 in the lighting performance support system 100 transmits only the updated partial control data UAC to the lighting performance creation device 10, but it may also transmit the updated performance control data 26A instead of the updated partial control data UAC. Lighting operation terminal 20 may transmit performance control data 26A each time partial control data is updated. Updated performance control data 26A may also be transmitted when the operator completes updating performance control data 26A on lighting operation terminal 20 and stores performance control data 26A in storage unit 26.

[0084] In the learning mode, the feature acquisition unit 113 of the lighting performance creation device 10 in the lighting performance support system 100 extracts lighting image features and sound features based on the real lighting and sound output by the lighting equipment 30 and sound equipment 30A, but it is also possible to construct a three-dimensional virtual space such as a concert hall or stage in the lighting performance preparation terminal 40, which outputs virtual lighting and sound, and have the lighting performance creation device 10 extract lighting image features and sound features based on the virtual lighting and sound. In this way, the lighting performance creation device 10 can accumulate experience based on realistic lighting and sound in its learning model without having to go to the concert hall, stage, or other location. In the inference mode, as in the learning mode, the feature acquisition unit 113 of the lighting performance creation device 10 in the lighting performance support system 100 may extract lighting image features and sound features based on the virtual lighting and sound output by the lighting performance preparation terminal 40, and input the lighting image features and sound features into a learning model to create new performance control data. In this way, the lighting performance creation device 10 can create new performance control data using a learning model that has accumulated experience based on lighting and sound that is close to reality. Note that in the inference mode, it is sufficient for the feature acquisition unit 113 to extract lighting image features and sound features; therefore, the preparer does not need to check the virtual lighting and sound output by the lighting performance preparation terminal 40, and the lighting performance preparation terminal 40 does not need to output virtual lighting and sound in real time.

[0085] The performance dividing unit 151 of the lighting performance creation device 10 in the lighting performance support system 100 acquires information about the new performance NP, divides it into partial performances, and extracts the features of each partial performance, but the operator may modify (re-divide or combine) the information about the partial performances divided by the performance dividing unit 151. The operator may also divide the performance into partial performances and extract the features of each partial performance independently of the performance dividing unit 151.

[0086] In the learning mode, the lighting performance creation device 10 in the lighting performance support system 100 may create learning performance control data 16B including the identifier of the lighting technician, and generate a learning model. In this way, in the inference mode, new performance control data can be created by referring only to the experience of a specific technician.

[0087] The lighting performance creation device 10 in the lighting performance support system 100 exchanges information with the lighting operation terminal 20 or the lighting performance preparation terminal 40 via communication (between communication unit 17 and communication unit 27 or communication unit 47), but the lighting operation terminal 20 or the lighting performance preparation terminal 40 may incorporate the functions of the lighting performance creation device 10 (the functions of the acquisition unit 11, the acceptance / rejection flag setting unit 12, the learning data creation unit 13, the model generation unit 14, and the control data creation unit 15).

[0088] Although the lighting operation terminal 20 in the lighting performance support system 100 is equipped with the photographing unit 24 and the listening unit 25, a terminal other than the lighting operation terminal 20 may be equipped with the photographing unit 24 and the listening unit 25. The lighting operation terminal 20 may communicate with the terminal via wired or wireless communication and acquire the lighting image information or audio information acquired by the photographing unit 24 and the listening unit 25. The lighting performance creation device 10 may communicate with the terminal via wired or wireless communication and acquire the lighting image information or audio information acquired by the photographing unit 24 and the listening unit 25. As long as the lighting performance creation device 10 can acquire the lighting image information and audio information, it may acquire them from any terminal.

[0089] Learning model 16C in lighting production support system 100 is trained to take as input program information T, performer information PL, lighting information L, information on the preceding partial performance BP (features, control method), information on the following partial performance AP (features, control method), and information on the relevant partial performance P (features, control method), and to output the degree of adoption AD of the control method for that relevant partial performance, but is not limited to this. For example, learning may be performed by inputting only the following partial performance AP, or only the preceding partial performance BP. If there are no adjacent partial performances (for example, a uniform piece of music without change or a short piece of music), learning may be performed assuming that there are no adjacent partial performances. Also, information on the preceding partial performance BP and the following partial performance AP may not be learned.

[0090] The lighting effect support system 100 has been described as a system for supporting lighting effects. Next, a sound effect support system 100A that supports sound effects will be described. Performances include not only playing instruments, singing, musicals, and pantomime at concert venues and on stage, but also street speeches and karaoke in stores. In audio production support system 100A, the production control data is data for controlling audio equipment (hereinafter referred to as audio production control data). To provide a production effect, audio production control data not only adjusts the volume of devices that output sound, such as speakers, but also adjusts the sensitivity of devices that input voice or musical instrument sounds, such as microphones (hereinafter referred to as input devices). Also, unlike control data for lighting equipment, audio production control data is often not prepared in advance, but is instead adjusted on-site while checking the sound effects output from audio equipment 30A for each location, such as audience seats, and for each input device. In audio production support system 100A, it is important to generate a learning model based on on-site experience. Therefore, in the learning mode, before the learning performance control data creation step S110, the computer is caused to execute a feature extraction step of acquiring the acoustic information output from the audio device 30A and extracting an acoustic feature, which is at least one of the feature quantities of loudness, pitch, and speed of change of the sound. In the learning performance control data creation step S110, the computer is caused to create learning performance control data including the acoustic feature. In the inference mode, a performance including sound output for a sound check is accepted, and in the control data creation step, acoustic features extracted from the acoustic information output from the acoustic device 30A for the performance accepted in the acceptance step are input into a learning model to create new performance control data. Note that a sound check is a test of the acoustics, and the sound output includes sounds with a melody, sounds without a melody such as a single note played on an instrument, and sounds from the same stage or performance as in the actual performance. In this way, experience is accumulated of adjusting the sound performance control data based on the output from the actual sound equipment on-site, and new performance control data for the sound equipment 30A can be created based on this experience.

[0091] In the following example, the performance is a karaoke performance in a bar. In the learning mode, not only the acoustic information output from the audio device 30A but also the sound input to the input device (the voice of the person singing using a microphone) is acquired, and at least one acoustic feature of the loudness, pitch, and speed of change of the sound is extracted and input to the learning model to create new performance control data. In the inference mode, the voice of the person actually singing using a microphone, which was received in the reception step, is acquired, and acoustic features, which are at least one of the features of loudness, pitch, and speed of change of the sound, are extracted and input into a learning model to create new performance control data. In this way, a learning model is generated that takes into account the sound input into input devices such as microphones, and this learning model can be used to create new performance control data (karaoke setting data) that is adjusted to match the voice of a person actually singing using a microphone.

[0092] The following describes partial performances in the audio production support system 100A. In the case of audio equipment, production can also be added in conjunction with the actual performance, and the performance is divided into multiple partial performances that are parts of that performance. Here, a partial performance is a performance that is divided not by time but by space. Specifically, in the case of a performance using multiple instruments, it can be a partial performance for each instrument, or a partial performance for each voice actor (speaker). The performance division unit 151 divides the sound (music, singing, speech, etc.) that is the information of the new performance NP into the above partial performances. The sound staff decides how to control the performance of the part in question, taking into consideration how to control the performance of the left and right and the front and back. The learning model 16C learns to determine a control method for a partial performance based on control methods for spatially adjacent partial performances. FIG. 13 shows the performance control data 16A and the updated partial control data UAC, and FIG. 14 shows the learning performance control data 16B. Similarly to the lighting performance support system 100, when there are no adjacent partial performances (for example, in the case of an instrument on the far left or right, or an instrument in the front or back row), the sound performance support system 100A does not include either the left or right partial performance or either the front or back partial performance in the input, and trains the learning model 16C. [Explanation of symbols]

[0093] Lighting Production Support System 100 Lighting production device 10 Acquisition part 11 Adoption / rejection flag setting unit 12 Learning data creation unit 13 Model generation unit 14 Control data creation unit 15 Performance Division 151 Control method determination unit 152 Storage section 16 Production control data 16A Learning performance control data 16B Learning Model 16C Correspondence Table 16D New performance control data 16E Communications Department 17 Lighting operation terminal 20 Reception Department 21 control unit 22 Update section 23 Filming Division 24 Hearing Section 25 Storage section 26 Production control data 26A lighting equipment 30 Sound Production Support System 100A Acoustic operation terminal 20A Audio equipment 30A Lighting production preparation terminal 40 Reception Department 41 Control unit 42 Update section 43 Filming Division 44 Hearing Section 45 Storage section 46 Acceptance / rejection flag A Admission / rejection AD Partial Performance P Partial control data C Pre-update partial control data UBC Updated partial control data UAC

Claims

1. A method for creating performance control data that allows a computer to create performance control data that adds lighting and / or sound effects in conjunction with a performance, comprising: The performance is divisible into a plurality of partial performances in time and / or space; The performance control data controls performance additional equipment including lighting equipment and / or audio equipment, the performance control data includes partial control data for controlling the performance additional equipment for each of the partial performances, an updated partial control data acquisition step of acquiring updated partial control data consisting of partial control data updated by a person; an adoption / rejection flag setting step of setting an adoption / rejection flag to unupdated partial control data consisting of partial control data that has not been manually updated among the partial control data and the updated partial control data, indicating that these have been adopted, and setting an adoption / rejection flag to pre-updated partial control data consisting of partial control data before the updated partial control data was updated, indicating that this has not been adopted; A learning performance control data creation step for creating learning performance control data including the pre-update partial control data, the post-update partial control data, the non-update partial control data, and the adoption / rejection flag; A learning performance control data creation step for creating performance control data; A model generation step of generating a learning model based on the learning performance control data; a reception step for receiving performance information in order to create new performance control data; a control data creation step of creating new performance control data that is new performance control data based on the performance information received in the reception step and the learning model; A method for creating performance control data that causes a computer to execute the above.

2. When the partial control data is created by a person, a performance control data acquisition step is performed before the updated partial control data acquisition step, in which the performance control data including all of the partial control data at that time is acquired. The method for creating performance control data according to claim 1, which is executed by a computer.

3. A feature extraction step of acquiring a lighting image output from a lighting performance preparation terminal that prepares performance control data or the lighting device, and extracting a lighting image feature that is at least one feature of light color, brightness, and movement; on the computer, In the learning performance control data creation step, learning performance control data including the lighting image feature amount is created. The method for creating performance control data according to claim 1.

4. A method for creating performance control data that causes a computer to create performance control data that adds lighting and / or sound effects in conjunction with a performance, comprising: The performance is divisible into a plurality of partial performances in time and / or space; The performance control data controls performance additional equipment including lighting equipment and / or audio equipment, the performance control data includes partial control data for controlling the performance additional equipment for each of the partial performances, an updated partial control data acquisition step of acquiring updated partial control data consisting of partial control data updated by a person; an adoption / rejection flag setting step for setting an adoption / rejection flag indicating that the updated partial control data has been adopted and indicating that pre-update partial control data consisting of partial control data before the updated partial control data has been updated has not been adopted; A learning performance control data creation step of creating learning performance control data including at least the pre-update partial control data, the post-update partial control data, and the adoption / rejection flag; A model generation step of generating a learning model based on the learning performance control data; a reception step for receiving performance information in order to create new performance control data; a control data creation step of creating new performance control data that is new performance control data based on the performance information received in the reception step and the learning model; When the partial control data is created by a person, a performance control data acquisition step is performed before the updated partial control data acquisition step, in which the performance control data including all of the partial control data at that time is acquired. Let the computer run the performance control data is data for controlling the audio equipment, Before the learning performance control data creation step, a feature extraction step of acquiring acoustic information output from the acoustic device and extracting an acoustic feature, which is at least one feature of loudness, pitch, and rate of change of sound; Let the computer run In the learning performance control data creation step, a computer is caused to create learning performance control data including the acoustic feature; In the receiving step, the computer is caused to receive a performance including sound output for a sound check; In the control data creation step, the computer is caused to input into the learning model acoustic features extracted based on the acoustic information output from the acoustic equipment for the performance accepted in the acceptance step, thereby creating new performance control data. How to create performance control data.

5. A method for creating performance control data that causes a computer to create performance control data that adds lighting and / or sound effects in conjunction with a performance, comprising: The performance is divisible into a plurality of partial performances in time and / or space; The performance control data controls performance additional equipment including lighting equipment and / or audio equipment, the performance control data includes partial control data for controlling the performance additional equipment for each of the partial performances, an updated partial control data acquisition step of acquiring updated partial control data consisting of partial control data updated by a person; an adoption / rejection flag setting step for setting an adoption / rejection flag indicating that the updated partial control data has been adopted and indicating that pre-update partial control data consisting of partial control data before the updated partial control data has been updated has not been adopted; A learning performance control data creation step of creating learning performance control data including at least the pre-update partial control data, the post-update partial control data, and the adoption / rejection flag; A model generation step of generating a learning model based on the learning performance control data; a reception step for receiving performance information in order to create new performance control data; a control data creation step of creating new performance control data that is new performance control data based on the performance information received in the reception step and the learning model; a feature extraction step of acquiring a lighting image output from a lighting performance preparation terminal that prepares performance control data or the lighting device, and extracting a lighting image feature, which is at least one feature of light color, brightness, and movement; on the computer, In the learning performance control data creation step, learning performance control data including the lighting image feature amount is created, A performance control data creation method in which, in the control data creation step, data including lighting image features obtained for the performance accepted in the acceptance step is input into the learning model to create new performance control data.

6. A method for creating performance control data that causes a computer to create performance control data that adds lighting and / or sound effects in conjunction with a performance, comprising: The performance is divisible into a plurality of partial performances in time and / or space; The performance control data controls performance additional equipment including lighting equipment and / or audio equipment, the performance control data includes partial control data for controlling the performance additional equipment for each of the partial performances, an updated partial control data acquisition step of acquiring updated partial control data consisting of partial control data updated by a person; an adoption / rejection flag setting step for setting an adoption / rejection flag indicating that the updated partial control data has been adopted and indicating that pre-update partial control data consisting of partial control data before the updated partial control data has been updated has not been adopted; A learning performance control data creation step of creating learning performance control data including at least the pre-update partial control data, the post-update partial control data, and the adoption / rejection flag; A model generation step of generating a learning model based on the learning performance control data; a reception step for receiving performance information in order to create new performance control data; a control data creation step of creating new performance control data that is new performance control data based on the performance information received in the reception step and the learning model; a feature extraction step of acquiring a lighting image output from a lighting performance preparation terminal that prepares performance control data or the lighting device, and extracting a lighting image feature, which is at least one feature of light color, brightness, and movement; on the computer, In the learning performance control data creation step, learning performance control data including the lighting image feature amount is created, a feature extraction step of acquiring audio information output from the lighting performance preparation terminal or the audio device, and extracting an audio feature, which is at least one of the feature of the loudness, pitch, and speed of change of the sound, by a computer; The learning performance control data creation step creates learning performance control data including the acoustic feature. How to create performance control data.

7. A method for creating performance control data as described in claim 6, wherein in the control data creation step, data including acoustic features acquired about the performance accepted in the acceptance step is input into the learning model to create new performance control data.

8. A production control data creation program that causes a computer to create production control data that adds lighting and / or sound effects in conjunction with a performance to be performed, The performance is divisible into a plurality of partial performances in time and / or space; The performance control data controls performance additional equipment including lighting equipment and / or audio equipment, the performance control data includes partial control data for controlling the performance additional equipment for each of the partial performances, an updated partial control data acquisition step of acquiring updated partial control data consisting of partial control data updated by a person; an adoption / rejection flag setting step of setting an adoption / rejection flag to unupdated partial control data consisting of partial control data that has not been manually updated among the partial control data and the updated partial control data, indicating that these have been adopted, and setting an adoption / rejection flag to pre-updated partial control data consisting of partial control data before the updated partial control data was updated, indicating that this has not been adopted; A learning performance control data creation step for creating learning performance control data including the pre-update partial control data, the post-update partial control data, the non-update partial control data, and the adoption / rejection flag; A model generation step of generating a learning model based on the learning performance control data; a reception step for receiving performance information in order to create new performance control data; a control data creation step of creating new performance control data that is new performance control data based on the performance information received in the reception step and the learning model; A performance control data creation program that causes a computer to execute the above.

Images