Intelligent light control system for virtual studio

Abstract

The invention provides a virtual studio intelligent light control system, which belongs to the field of intelligent light control systems and comprises an image acquisition module, an image analysis module, a human body positioning analysis module, an intelligent light control system and a user interaction interface. Through the system, light regulation and control, light brightness and color temperature of a studio scene can be automatically adjusted, and a natural and situational light effect is achieved. The system is simple and convenient to operate, can improve the production efficiency and quality of studio, has good expandability and compatibility, can adapt to studio requirements of different scales and types, and has a wide market application scene. According to the system, manual intervention is not needed, the needed studio light effect is determined through image collection, video, photosensitive recognition and image content analysis, manual contextual model selection and the like are not needed, and real intelligence is achieved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent lighting control, and particularly to a virtual studio intelligent lighting control system and method thereof. Background Art

[0002] With the rapid development of digital media technology, virtual studio technology has become an important part of fields such as live broadcast and stage performance. As a key link in this technical field, the development background of the virtual studio intelligent lighting control system can be traced back to the limitations of traditional studio lighting control technology and the continuous upgrading of modern technology requirements. The traditional studio lighting system relies on manual operation, which is not only inefficient but also difficult to achieve rapid switching and precise control of complex scenes. Operators need to manually adjust the position, brightness, and color temperature of each lighting device according to the director's instructions, which is time-consuming and error-prone. With the progress of computer technology, graphics processing capabilities, and sensor technology, virtual studio technology has emerged. It brings an immersive visual experience to the audience through the real-time synthesis of a computer-generated virtual environment and real people or objects.

[0003] The intelligent lighting control system is an important part of virtual studio technology. It uses advanced control algorithms and hardware devices to achieve automated and intelligent management of lighting. Through seamless docking with the scene, the intelligent lighting system can automatically adjust the color temperature, brightness, and direction of the lighting according to the changes in the virtual scene, thus achieving a perfect fusion of reality and virtuality. In recent years, with the continuous maturity of artificial intelligence technology, the intelligent lighting control system has begun to integrate machine learning and deep learning algorithms, further improving the naturalness and adaptability of lighting effects. The system can learn the lighting setting habits of studio operators, automatically optimize lighting scenes, and even make autonomous decisions in specific situations to adapt to the changing requirements of programs and scenes. In summary, the development background of the virtual studio intelligent lighting control system is multi-faceted. It not only stems from the limitations of traditional technologies but is also the product of the progress of modern technologies such as digital media technology, artificial intelligence, and the Internet of Things. With the continuous integration and innovation of these technologies, the future virtual studio intelligent lighting control system will be more intelligent, efficient, and flexible, bringing revolutionary changes to live broadcast and stage performance. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a virtual studio intelligent lighting control system and method to solve at least one technical problem raised in the background art.

[0005] A virtual studio intelligent lighting control system includes an image acquisition module, an image analysis module, a human body positioning and analysis module, an intelligent lighting control module, and a user interface; the image acquisition module is used to capture high-definition videos and images in the studio environment in real time; The image analysis module is used to analyze high-definition videos and images, extract the action information and environmental information of the studio personnel, accurately identify the actions and environment of the studio personnel in the videos and images, and output the action and environmental information as the analysis result to the human body positioning analysis module; The human body positioning analysis module parses and judges the action and environmental information output by the image analysis module, identifies the outline of the human body and the specific position information where the human body is located, and this unit outputs the parsed position signal in a stable manner; The intelligent lighting control module dynamically adjusts the lighting effect parameters of the lighting control module according to the action signals, environmental signals and position signals of the image analysis module and the human body positioning analysis module; The user interaction interface adopts a touch screen operation panel, and the operator can set and adjust the lighting effect parameters.

[0006] The image acquisition module is a high-resolution camera, including one or a combination of a wide-angle camera and a patrol camera.

[0007] The high-resolution camera is an infrared / near-infrared camera.

[0008] The image analysis module includes a human body classifier, and the human body classifier is used to identify the human body in the image; The method for the image analysis module to output the analysis result is as follows: S01: Train the human body classifier with a large amount of human and non-human image data to improve the human body image recognition ability; the human body classifier is optimized by a deep learning algorithm and can accurately identify various objects in the image, especially human body features; S02 Input the high-definition video and image into the trained human body classifier; S03 Use the human body classifier to perform feature extraction and classification processing on each sub-window provided by the image acquisition module, including outputting and marking the window label with a human body as the window with human body features, and outputting and marking the window label without a human body as the window without human body features.

[0009] The human body positioning analysis module has the function of background modeling for images, and the specific modeling method is as follows: S01 Collect a series of consecutive frame images and perform background modeling to determine the static elements, basic outlines and background ranges in the scene; S02 On the basis of completing the background modeling, perform precise calibration, label annotation and classification processing on the main areas of the background to more carefully identify and distinguish different scenes.

[0010] The method for the human body positioning analysis module to identify the outline of the human body and the position information where the human body is located is as follows: S01 receives the window marked with human features transmitted by the image analysis module, subtracts the window marked with human features from the background model to obtain a foreground image; the foreground image contains all the changing areas in the current frame image, and the areas correspond to moving objects and humans; S02 further processes the foreground image, excludes non-human areas, and only retains the human areas in the foreground image; S03 compares the background modeling, analyzes the relative position of the human area in the background modeling, and determines the specific position where the human is located.

[0011] The human body positioning and analysis module has a photosensitive acquisition function and outputs environmental information through the human body positioning and analysis module. The specific photosensitive acquisition method is as follows: S01 Photosensitive elements are installed in the studio environment to capture the light in the studio environment in real time; S02 converts the light signal captured by the photosensitive element into an electrical signal; S03 The electrical signal outputs specific environmental information through the human body positioning and analysis module.

[0012] The human body positioning and analysis module also includes a human body behavior analysis unit. The human body behavior analysis unit is used to capture human body behavior characteristics. The method for capturing human body behavior characteristics is as follows: S01 Selects the image containing the marked human area from the image analysis module; S02 The human body behavior analysis unit is equipped with a human body behavior preset table for distinguishing human body behaviors and associating actions with human body behaviors; S03 Extracts the features of the calibrated image and matches them with the actions in the human body behavior preset table to capture human body behavior characteristics.

[0013] In the intelligent lighting control system, a comparison table of human body behavior characteristics and scene lights, a comparison table of environment and light brightness and color, and a comparison table of human body location and light brightness and color are pre-built; the comparison table of human body behavior characteristics and scene lights is used to adaptively adjust the lights according to the human body behavior characteristic signal: when the human body behavior characteristic signal is input, the lights are automatically adjusted according to the comparison table of human body behavior characteristics and scene lights; The comparison table of environment and light brightness and color is used to adaptively adjust the lights according to the environmental information signal; when the corresponding environment and time signals are input, the lights are automatically adjusted according to the comparison table of environment and light brightness and color; The table of the position of the human body, lighting brightness, and color is used to adaptively adjust the lighting according to the specific position signal of the human body. After inputting the specific position signal of the human body, the lighting is automatically adjusted according to the table of the position of the human body, lighting brightness, and color. The automatic adjustment of the lighting refers to the control table in the table of human behavior characteristics and scene lighting, the table of environment and lighting brightness and color, and the table of the position of the human body, lighting brightness, and color, which can adjust the brightness or position of the scene lights, atmosphere lights, and follow lights.

[0014] Beneficial effects: The virtual studio intelligent lighting control system provided by the present invention integrates infrared / near-infrared imaging technology to ensure clear images can still be obtained in low-light or lightless environments. To ensure that the monitoring system can effectively identify individuals at any position, it is necessary to ensure that the shooting range of the camera can cover the entire monitoring area. Strategies to achieve this goal include using wide-angle cameras, rotatable cameras, and multi-camera joint systems. In a multi-camera system, the application of image stitching technology is crucial. It ensures seamless stitching of images and avoids the generation of monitoring blind spots through methods such as wavelet domain processing and matrix switching. In low-light or lightless environments, the application of infrared / near-infrared cameras becomes the key. This technology can capture clear black-and-white images without visible light through the built-in infrared lighting system, greatly enhancing the adaptability and reliability of the monitoring system. Brief Description of the Drawings

[0015] Figure 1 It is the structural block diagram of the intelligent lighting control system in the embodiment of the present invention; Figure 2 It is the method flowchart of the intelligent lighting control system in the embodiment of the present invention; Figure 3 It is the flowchart of selecting corresponding lamps according to the human body position in the embodiment of the present invention; Figure 4 It is the flowchart of determining the brightness of the lamps in the embodiment of the present invention; Figure 5 It is the flowchart of adjusting the brightness of the lamps in the embodiment of the present invention. Detailed Embodiments

[0016] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0017] Please refer to Figures 1-5As shown, a virtual studio intelligent lighting control system of the present invention includes an image acquisition module, an image analysis module, a human body positioning analysis module, an intelligent lighting control module, and a user interaction interface; Among them, the image acquisition module is used to capture high-definition video and images in the studio environment in real time; The image analysis module analyzes the real-time video and image information captured by the aforementioned modules, extracts the position, action information and environmental information of the performers, accurately identifies the key factors such as the position and action of the performers in the real-time image and the environment at that time, and outputs signals; Among them, the human body positioning analysis module is highly intelligent and accurate. It analyzes and judges the real-time video and image data captured by the aforementioned units, identifies the outline and movement of the human body and the specific location information. The unit outputs the position signal obtained by the analysis in a stable manner; Among them, the intelligent lighting control module dynamically adjusts the lighting effect parameters of the lighting control module according to the output signals of the image analysis module and the human body positioning analysis module; Among them, the user interaction interface uses a touch screen operation panel, which allows users to set and adjust the lighting effect parameters intuitively and conveniently. Users can easily select and modify various lighting parameters by simply touching the screen, which greatly improves the convenience of operation. In addition, the interface also provides a preview function, so that users can preview the changes in lighting effects before actually adjusting the parameters. Through this preview function, users can intuitively see the adjustment results of the lighting effects, so as to more accurately control the lighting atmosphere and create an ideal lighting environment. Whether adjusting brightness, color temperature or color, users can use the preview function to ensure that the final effect meets their needs, so as to achieve the best lighting effect.

[0018] The image acquisition module is equipped with high-resolution cameras that support a variety of video formats and frame rate options. They can be designed with wide-angle lenses to capture a wider field of view, or have a patrol function that can switch between multiple preset positions to achieve all-round monitoring. In addition, these cameras can also be used in conjunction with multiple devices to achieve more complex and comprehensive image acquisition needs. These high-resolution cameras are equipped with infrared or near-infrared imaging technology, which can provide clear images in low-light or no-light conditions, thereby meeting the needs of various specific application scenarios.

[0019] See also Figure 1As shown, a virtual studio intelligent lighting control system in an embodiment of the present invention includes an image acquisition module for acquiring images, an image analysis module for analyzing images and comparing human behaviors, a human position analysis module with functions of human position detection and photosensitive acquisition, an intelligent lighting control module for forming lighting control of the above signals, and a user interface interactively connected to the intelligent lighting control module. The image information acquired by the image acquisition unit is transmitted to the image analysis unit, and the image analysis unit analyzes the image and outputs the analysis result to the intelligent lighting control module.

[0020] The aforementioned image acquisition module has high-resolution characteristics and supports multiple video formats and frame rates. Its design includes a wide-angle camera, a rotatable camera, and a multi-camera combined system, aiming to achieve full coverage of the monitoring area. In addition, this module is particularly integrated with infrared / near-infrared imaging technology to ensure clear images can still be obtained in low-light or lightless environments. To ensure that the monitoring system can effectively identify individuals at any position, it is necessary to ensure that the shooting range of the camera can cover the entire monitoring area. The strategies to achieve this goal include using a wide-angle camera, a rotatable camera, and a multi-camera combined system. In a multi-camera system, the application of image stitching technology is crucial. It ensures seamless stitching of images through methods such as wavelet domain processing and matrix switching, avoiding the generation of monitoring blind spots. In low-light or lightless environments, the application of infrared / near-infrared cameras becomes the key. This technology can capture clear black-and-white images under no visible light conditions through an internal infrared lighting system, greatly enhancing the adaptability and reliability of the monitoring system.

[0021] Please refer to Figure 2 As shown, it shows a virtual studio intelligent lighting control system, and the control method includes the following steps: The image acquisition module performs video or image acquisition on the images in the studio area and transmits the acquired video or image information to the image analysis unit.

[0022] The image analysis module has a human body detection function, which can accurately detect human bodies in each frame of a video or image sequence, and provide key data support for the intelligent system by identifying human body contours and movements. At the same time, the module has a built-in photosensitive monitoring mechanism that can monitor the light intensity in the scene in real time and compare and analyze it with the preset environmental parameters, time thresholds, and brightness standards. When the ambient lighting conditions do not meet the preset standards, the system will suspend operation until the conditions are met. Once the environmental conditions meet the preset standards, the system will activate the human body positioning analysis module to identify the human body contour and movements and determine the specific position of the human body in the scene. Through the combination of human body detection and tracking technology, the system can determine whether there are human bodies in the image, as well as the number and specific location of human bodies. This information is then transmitted to the intelligent lighting control module. After receiving the behavior information, the module will adjust the brightness and color temperature of light sources such as atmosphere lights, ambient lights, and chase lights based on the pre-built-in human behavior characteristics and scene lighting comparison table, the environment and light brightness and color comparison table, and the human body location and light brightness and color comparison table. In addition, the intelligent lighting control module can also make fine lighting adjustments based on human activity patterns.

[0023] In the above steps, when all lighting conditions are met, the ambient light will be activated, and the system will immediately start the pre-recorded atmosphere light or follow-up light according to the pre-set program and the information in the environment, light and brightness comparison table to create a suitable atmosphere for the studio scene. At the same time, the system will use the signal of the photosensitive sensor to further adjust the lighting brightness of the lighting system to ensure that the light is both soft and bright to meet the needs of different scenes. In the above steps, once the sensor detects the presence of people in the studio environment, the system will start a series of analysis programs. These programs include not only the precise analysis of the position of the human body in the image, but also the in-depth interpretation of the behavior of the personnel. Through these analyses, the system can identify the specific actions and positions of the personnel, so as to further adjust the lighting system. If the system detects that a staff member is standing at a certain position in the scene, it will automatically adjust the follow-up light to ensure that the light is focused on the person. Through these advanced functions, the lighting system can not only provide basic lighting needs, but also become an important tool for creating atmosphere and enhancing performance effects.

[0024] See also Figure 3 As shown, the method for confirming the specific position of the human body comprises the following steps: By adopting the method of background modeling, we first need to collect several frames of images in the studio environment for background modeling. Background modeling is a commonly used technique for extracting the static background from a series of images, so that foreground objects can be effectively identified and separated in subsequent image processing. Specifically, we can analyze these collected image frames, extract the static elements from them, and construct a stable background model. This model can be used in real-time video processing to help us distinguish between the background and foreground objects.

[0025] However, over time, the studio environment may change, such as the adjustment of lighting, the movement of props, or the replacement of the background scenery. These changes may cause the original background model to be inaccurate, thus affecting the subsequent image processing effect. Therefore, to ensure the accuracy and effectiveness of background modeling, we need to regularly monitor the environmental changes. If an environmental update is detected, background modeling needs to be performed again. The process of re-modeling is similar to the initial modeling. We need to collect several frames of the latest images again, and then extract a new background model through the same algorithm. In this way, we can ensure that the background model always matches the current studio environment, thereby improving the accuracy and reliability of image processing.

[0026] After background modeling, the main areas within the studio area can be calibrated, and the main lights used by the human body in different areas can be determined according to the lighting comparison table based on the human body's location.

[0027] The human body positioning analysis module obtains the current frame image, subtracts the current frame image from the background to obtain the foreground, and then excludes the non-human areas in the foreground. After that, the human body area in the foreground is obtained. The human body area in the foreground is compared with the background to determine the area location where the human body is located and determine the corresponding lights.

[0028] After the human body positioning analysis module analyzes the specific position of the human body, it outputs the analysis result to the intelligent lighting control module, and the intelligent lighting control module selects the corresponding lighting mode.

[0029] Please refer to Figure 4 as shown, for the method of determining human behavior and making corresponding lighting adjustments, which includes the following steps: The human body positioning analysis module receives the human body area image information analyzed by the human body detection module.

[0030] Since different behaviors have different characteristics, a human behavior classifier can be trained for human behavior classification. The specific approach is to select multiple human body images, calibrate their behaviors, extract features from the calibrated images, and for the input image, use the human behavior classifier in it to determine the human behavior according to the classification result.

[0031] In the intelligent lighting control system, a comparison table of human behaviors and lighting modes is entered, and the intelligent lighting control system will adjust the corresponding light source according to the determined human behavior.

[0032] Please refer to Figure 5 as shown, to confirm the method for adjusting the lighting brightness in the studio scene, which includes the following steps: In the flowchart, Ic is the current brightness detected by the photosensitive detector, and It is the comparison table of the desired brightness at different time periods. If the following formula is satisfied, it means that the current brightness is very close to the desired brightness and the brightness adjustment is completed; otherwise, the brightness adjustment is not completed yet. Among them, Eth is a preset threshold, which is not fixed but will be adjusted accordingly according to different seasons and months. Specifically, the setting of this threshold will be comprehensively considered in combination with the actual effect and actual situation on site. In summer and winter, due to the large changes in temperature and humidity, the value of Eth will vary according to the changes in temperature and humidity; in spring and autumn, due to the relatively stable climate, the value of Eth will be relatively fixed. In short, the setting of Eth is a dynamic adjustment process aimed at ensuring the best operation effect of the system.

[0033] Formula 1: |Ic - It| < Eth If the above formula is not satisfied and Ic < It, it means that the detected brightness is less than the desired brightness, and the brightness needs to be increased and then compared again; if Ic > It, it means that the detected brightness is greater than the desired brightness, and the brightness needs to be decreased and then compared again. Repeat the above loop until Formula 1 is satisfied, and then the brightness adjustment will stop.

[0034] The change of the lamp brightness belongs to the existing technology, and the lamp brightness can be changed by means of voltage-dividing resistors, shunt, etc., which will not be elaborated here. In the adjustment of brightness, in order to improve the adjustment speed and prevent the eyes from feeling uncomfortable due to too rapid brightness changes, a hierarchical adjustment method can be adopted.

[0035] The above description has fully disclosed the specific implementation manners of the present invention. It should be noted that any changes made by those skilled in the art to the specific implementation manners of the present invention do not depart from the scope of the claims of the present invention. Correspondingly, the scope of the claims of the present invention is not limited only to the foregoing specific implementation manners.

Claims

1. A virtual studio intelligent lighting control system, characterized in that: It includes an image acquisition module, an image analysis module, a human body positioning analysis module, an intelligent lighting control module, and a user interaction interface; the image acquisition module is used to capture high-definition videos and images in the studio environment in real time; The image analysis module is used to analyze the high-definition videos and images, extract the action information and environmental information of the studio personnel, accurately identify the actions and environment of the studio personnel in the videos and images, and output the action and environmental information as the analysis result to the human body positioning analysis module; The human body positioning analysis module parses and judges the action and environmental information output by the image analysis module, identifies the contour and position information of the human body, and outputs the parsed position signal in a stable manner; The intelligent lighting control module dynamically adjusts the lighting effect parameters of the lighting control module according to the action, environmental information, and position information of the image analysis module and the human body positioning analysis module; The user interaction interface adopts a touch screen operation panel, and the operator can set and adjust the lighting effect parameters.

2. The virtual studio intelligent lighting control system according to claim 1, characterized in that: The image acquisition module is a high-resolution camera, including one or a combination of a wide-angle camera and a patrol camera.

3. The intelligent lighting control system for virtual studio according to claim 2, characterized in that: The high-resolution camera is an infrared / near-infrared camera.

4. A virtual studio intelligent lighting control system according to claim 1, characterized in that: The image analysis module includes a human body classifier, and the human body classifier is used to identify the human body in the image; the detection result output by the image analysis module includes an image marked with the human body area or an unmarked image; The method for the image analysis module to output the analysis result is as follows: S01: Train the human body classifier with a large amount of human and non-human image data to improve the human body image recognition ability; the human body classifier is optimized by a deep learning algorithm and can accurately identify various objects in the image, especially human body features; S02 Input the high-definition videos and images into the trained human body classifier; S03 Use the human body classifier to perform feature extraction and classification processing on each sub-window provided by the image acquisition module, including outputting and marking the window label with a human body as a window with human body features, and outputting and marking the window label without a human body as a window without human body features.

5. The intelligent lighting control system for virtual studio as claimed in claim 1, wherein: The human body positioning analysis module is also used to perform background modeling on the image, and the background modeling method is as follows: S01 Collect a series of consecutive frame images and perform background modeling to determine the static elements, basic contours, and background range in the scene; S02 On the basis of completing the background modeling, perform precise calibration, label annotation, and classification processing on the main area of the background to more carefully identify and distinguish different scenes.

6. The virtual studio intelligent lighting control system according to claim 1, characterized in that: The method for the human body positioning analysis module to identify the contour and position information of the human body is as follows: S01 Receive the window marked with human body features transmitted by the image analysis module, subtract the window marked with human body features from the background model to obtain a foreground image; the foreground image contains all the changing areas in the current frame image, and the areas correspond to moving objects and human bodies; S02 Further process the foreground image, exclude the non-human areas, and only retain the human body area in the foreground image; S03 compares the background modeling, analyzes the relative position of the human body area in the background modeling, and determines the specific position where the human body is located.

7. The intelligent lighting control system for virtual studio according to claim 1, characterized in that: The human body positioning analysis module has a photosensitive acquisition function and outputs environmental information through the human body positioning analysis module. The specific photosensitive acquisition method is as follows: S01 A photosensitive element is installed in the studio environment, which can capture the light in the studio environment in real time; S02 The light signal captured by the photosensitive element is converted into an electrical signal; S03 The electrical signal outputs specific environmental information through the human body positioning analysis module.

8. The intelligent lighting control system for virtual studio as claimed in claim 6, wherein: The human body positioning analysis module further includes a human body behavior analysis unit, and the human body behavior analysis unit is used to capture human body behavior characteristics. The method for capturing human body behavior characteristics is as follows: S01 Select an image containing the marked human body area from the image analysis module; S02 The human body behavior analysis unit is internally provided with a human body behavior preset table for distinguishing human body behaviors and associating actions with human body behaviors; S03 Extract features from the calibrated image and match them with the actions in the human body behavior preset table to capture human body behavior characteristics.

9. The intelligent lighting control system for virtual studio according to claim 1, wherein: In the intelligent lighting control system, a comparison table of human body behavior characteristics and scene lights, a comparison table of environment and light brightness and color, and a comparison table of the position of the human body and light brightness and color are pre-built; The comparison table of human body behavior characteristics and scene lights is used to adaptively adjust the lights according to the human body behavior characteristic signal: When the human body behavior characteristic signal is input, the lights are automatically adjusted according to the comparison table of human body behavior characteristics and scene lights; The comparison table of environment and light brightness and color is used to adaptively adjust the lights according to the environmental information signal: When the corresponding environment and time signals are input, the lights are automatically adjusted according to the comparison table of environment and light brightness and color; The comparison table of the position of the human body and light brightness and color is used to adaptively adjust the lights according to the specific position signal of the human body. When the specific position signal of the human body is input, the lights are automatically adjusted according to the comparison table of the position of the human body and light brightness and color; The automatic adjustment of the lights means that the comparison tables in the comparison table of human body behavior characteristics and scene lights, the comparison table of environment and light brightness and color, and the comparison table of the position of the human body and light brightness and color have the control adjustment of the brightness or position of the scene lights, atmosphere lights, and follow spotlights.

Images