Immersive light control method based on audio and video and computer readable storage medium

Abstract

The invention discloses an immersive light control method based on audio and video and a computer readable storage medium, and the method comprises the steps: light control equipment obtains audio and video features in real time from audio and video equipment connected with the light control equipment, and the audio and video features are obtained by the audio and video equipment through the real-time analysis of audio and video data; the light control equipment determines lamp control parameters according to the audio and video features, and controls light equipment connected with the light control equipment according to the lamp control parameters; wherein the audio and video features comprise picture color, contrast and drumbeat intensity; the lamp control parameters comprise at least one of the following parameters: light color, light brightness, lamp displacement parameters, motor rotating speed and stroboscopic switch parameters. Linkage change of audio and video and light can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of lighting control, and in particular to an audio-video based immersive lighting control method and a computer-readable storage medium. Background Art

[0002] Digital lighting scene effects have been widely used in digital entertainment venues, such as light and shadow restaurants, KTV stages, banquet halls and other public places. The lighting engineer pre-arranges the lighting according to the on-site environment and atmosphere and finally imports it into the lighting control equipment to form the corresponding lighting scene. This method of importing the programmed lighting scene into the lighting control equipment presents a relatively simple lighting scene effect and cannot interact with other related equipment, such as the video content and audio played, and can only rely on the dimming engineer to adjust the lighting atmosphere on site. However, the cost of the dimming engineer's permanent stay is high, which will result in high labor costs. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an immersive lighting control method based on audio and video and a computer-readable storage medium, which can realize the linkage change of audio and video and lighting.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: an immersive lighting control method based on audio and video, comprising: The lighting control device obtains audio and video features in real time from the audio and video device connected thereto, wherein the audio and video features are obtained by the audio and video device through real-time analysis of audio and video data; The lighting control device determines the lighting control parameters according to the audio and video characteristics, and controls the lighting device connected thereto according to the lighting control parameters; Among them, the audio and video characteristics include picture color, contrast and drum density; the lamp control parameters include at least one of light color, light brightness, lamp displacement parameters, motor speed and strobe switch parameters.

[0005] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0006] The beneficial effects of the present invention are as follows: by analyzing the audio and video data, visual and auditory features are extracted; by associating the audio and video features with the lamp control parameters, the lighting equipment is controlled according to the audio and video features; on the basis of the original lighting scene effect, the lighting effect can be adjusted as the audio and video features change, thereby achieving an immersive lighting effect experience, and the lighting effect can be automatically adjusted without human intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1It is a flow chart of an immersive lighting control method based on audio and video of the present invention; Figure 2 This is a flow chart of a method according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0008] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0009] Please refer to Figure 1 , an immersive lighting control method based on audio and video, comprising: The lighting control device obtains audio and video features in real time from the audio and video device connected thereto, wherein the audio and video features are obtained by the audio and video device through real-time analysis of audio and video data; The lighting control device determines the lighting control parameters according to the audio and video characteristics, and controls the lighting device connected thereto according to the lighting control parameters; Among them, the audio and video characteristics include picture color, contrast and drum density; the lamp control parameters include at least one of light color, light brightness, lamp displacement parameters, motor speed and strobe switch parameters.

[0010] From the above description, it can be seen that the beneficial effects of the present invention are: it can realize the linkage changes of audio, video and lighting, thereby realizing an immersive lighting effect experience, and the lighting effect can be automatically adjusted without human intervention.

[0011] Furthermore, the audio and video device analyzes the audio and video data in real time to obtain audio and video features, including: The audio and video device obtains a video frame in the audio and video data in real time as the current video frame, and obtains an audio frame in the audio and video data in real time as the current audio frame according to a preset audio frame length; Determine the screen color corresponding to the current video frame according to the color that accounts for the largest proportion in the current video frame; Determine the contrast corresponding to the current video frame; Drum beat recognition is performed on the current audio frame, and the density of drum beats corresponding to the current audio frame is determined according to the number of recognized drum beats.

[0012] From the above description, it can be seen that by extracting the video screen color, contrast and audio drum beats as audio and video features, it is convenient to control the lighting equipment according to the audio and video features later.

[0013] Furthermore, after determining the picture color corresponding to the current video frame according to the color with the largest proportion in the current video frame, the method further includes: If the picture color is a dark color, the picture color corresponding to the current video frame is determined according to the color that accounts for the largest proportion and is a bright color in the current video frame.

[0014] From the above description, it can be seen that the color of the subsequent control light is made bright, so that the lighting effect is brighter and the user experience is improved.

[0015] Furthermore, the lighting control device determines the lighting control parameters according to the audio and video characteristics, including: The lighting control device determines the lighting color according to the color of the picture; Determine the light brightness based on the contrast; According to the density of drum beats, determine the displacement parameters of the lamp and the motor speed; Determine the strobe switch parameters according to the changes in picture color and the density of drum beats.

[0016] From the above description, it can be seen that the lighting control parameters are determined according to the audio and video characteristics, thereby realizing the linkage between audio and video and lighting.

[0017] Further, determining the light brightness according to the contrast includes: If the contrast is greater than or equal to a preset contrast threshold, the light brightness is set to a first brightness, where the first brightness is greater than or equal to a preset brightness threshold; If the contrast is less than a preset contrast threshold, the light brightness is set to a second brightness, which is less than a preset brightness threshold.

[0018] From the above description, it can be seen that if the contrast of the video screen is high, the light brightness will be set to high brightness, and if the contrast of the video screen is low, the light brightness will be set to low brightness.

[0019] Furthermore, determining the lamp displacement parameter and the motor speed according to the drum beat density includes: If the drum beat density is less than or equal to a preset first quantity threshold, the displacement step value is increased according to a preset first change value; If the drumbeat density is greater than a preset first quantity threshold and less than or equal to a preset second quantity threshold, the displacement step value is increased according to a preset second change value, and the motor speed is increased according to a preset first multiple; If the density of drum beats is greater than a preset second quantity threshold, the displacement step value is increased according to a preset third change value, and the motor speed is increased according to a preset second multiple.

[0020] From the above description, we can know that according to the density of drum beats, the displacement step value and the motor speed (motor steering speed) are adjusted, that is, the motion parameters of the lamp are adjusted so that the light movement speed changes with the audio rhythm, thereby achieving a sound-light linkage effect.

[0021] Furthermore, the step of determining the strobe switch parameters according to the change of the picture color and the density of the drum beats includes: If the change value of the picture color within the preset time length exceeds the preset change threshold and the number of changes exceeds the preset number threshold, and the drum beat density is greater than the preset third number threshold, the strobe switch parameter is set to open.

[0022] From the above description, it can be seen that if the video screen color changes greatly and the audio drum beats are dense, the strobe function is turned on.

[0023] Furthermore, before the lighting control device acquires the audio and video features from the audio and video device connected thereto in real time, the lighting control device further comprises: The lighting control device obtains the channel information of each lighting device connected thereto, and manages the channels of each lighting device in groups, wherein the channels of the lighting devices include color control channels, displacement control channels, brightness control channels, strobe control channels and motor control channels.

[0024] From the above description, it can be seen that by classifying and grouping the digital lighting channel functions, the destination channel of the target lamp can be quickly located, and the interactive control is more efficient.

[0025] Furthermore, the grouping management of the channels of each lighting device includes: According to the function of the channel, the channels of each lighting device are managed in groups to uniformly control the channels with the same function of the lighting equipment; Or, the channels of each lighting device can be grouped and managed according to the lighting device to which the channel belongs, so as to uniformly control all channels of the same lighting device.

[0026] From the above description, it can be seen that flexible control of the lamp channels is achieved.

[0027] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0028] Embodiment 1 Please refer to Figure 2 , Embodiment 1 of the present invention is: an immersive lighting control method based on audio and video, which can be applied to digital entertainment venues. The method is based on a lighting control architecture, including audio and video devices, lighting control devices and digital lighting devices connected in sequence, the number of digital lighting devices can be multiple, and multiple digital lighting devices are respectively connected to the lighting control device. Among them, the audio and video device can be a set-top box, or a mobile terminal such as a mobile phone, and the digital lighting device can be a DMX512 lamp. The lighting control device and the digital lighting device can be connected through a serial port, a network, a USB, etc., and the DMX512 protocol or the ART-NET protocol can be used.

[0029] like Figure 2 As shown, the following steps are included: S1: The lighting control device obtains the channel information of each lighting device connected thereto, and manages the channels of each lighting device in groups.

[0030] Specifically, the lighting control device identifies the channel definition and model of the digital lighting device. In this embodiment, the channel of the lighting device includes at least one of a color control channel, a displacement control channel, a brightness control channel, a strobe control channel and a motor control channel. Each channel has a corresponding function. The color control channel is used to control the RGB color of the light, the displacement control channel is used to control the movement of the lamp on the X-axis and Y-axis, the brightness control channel is used to control the brightness of the light, the strobe control channel is used to control the turning on and off of the strobe, and the motor control channel is used to control the motor speed.

[0031] In this embodiment, the channels of each lighting device can be grouped and managed according to the function of the channel, so that the same function channels of different lighting devices can be uniformly controlled. In other embodiments, the channels of each lighting device can also be grouped and managed according to the lighting device to which the channel belongs, so that all channels of the same lighting device can be uniformly controlled.

[0032] S2: The audio and video device analyzes the audio and video data in real time to obtain audio and video features, and sends them to the lighting control device.

[0033] Specifically, the audio and video device obtains a video frame in the audio and video data in real time as the current video frame, and obtains an audio frame in the audio and video data in real time as the current audio frame according to a preset audio frame length. The video frame includes a complete video picture. In this embodiment, the preset audio frame length is 1s.

[0034] Then, the picture color and contrast degree corresponding to the current video frame are obtained. Specifically, the color with the largest proportion in the current video frame is used as the picture color corresponding to the current video frame. Furthermore, if the color with the largest proportion in the current video frame belongs to a dark color (or a cold tone), the color with the largest proportion in the current video frame and belongs to a bright color (or a warm tone) is used as the picture color corresponding to the current video frame. Among them, bright colors include red, orange, yellow, purple, etc., and dark colors include black, gray, etc.

[0035] At the same time, the drum beat density corresponding to the current audio frame is obtained. Specifically, the drum beats of the current video frame are recognized, and the number of recognized drum beats is counted as the drum beat density.

[0036] Finally, the picture color and contrast corresponding to the current video frame and the drum density corresponding to the current audio frame are sent to the lighting control device.

[0037] If the audio and video device is a set-top box, you can directly obtain audio and video data by installing an APP to extract audio and video features, and convert them into an agreed protocol (custom protocol) and send them to the lighting control device. If the audio and video device is a device such as a TV with an HDMI (High Definition Multimedia Interface) interface, you need to use a dedicated audio and video feature extraction device, which decodes the HDMI signal to obtain audio and video features and forwards them to the lighting control device in real time according to the agreed protocol.

[0038] S3: The lighting control device determines lighting control parameters according to the audio and video characteristics.

[0039] Specifically, the light color is determined according to the picture color.

[0040] Determine the light brightness based on the contrast. For example, if the contrast is greater than or equal to the preset contrast threshold (such as 80, the contrast threshold range is 0-100), the light brightness is set to the first brightness, which is greater than or equal to the preset brightness threshold (such as 204, the brightness value range is 0-255); if the contrast is less than the preset contrast threshold (such as 1, the contrast threshold range is 0-100), the light brightness is set to the second brightness, which is less than the preset brightness threshold (such as 0, the brightness value range is 0-255). In other words, if the contrast of the video screen is high, the light brightness will be set to high brightness, and if the contrast of the video screen is low, the light brightness will be set to low brightness (a brightness of 0 is equivalent to off).

[0041] According to the density of drum beats, the displacement parameters of the lighting fixture and the motor speed are determined. For example, when the drum beats are not dense (such as only 1 or 0 drum beats are recognized in an audio frame), the displacement step value of the XY axis of the lighting device is increased by 1 based on the original value, and the motor speed remains unchanged; when the density of drum beats is small (such as more than 1 but no more than 3 drum beats are recognized in an audio frame), the displacement step value is increased by 3 based on the original value, and the motor speed becomes 3 times the original value; when the density of drum beats is large (such as more than 3 drum beats are recognized in an audio frame), the displacement step value is increased by 5 based on the original value, and the motor speed becomes 5 times the original value.

[0042] Determine the strobe switch parameters based on the color change of the screen and the density of drum beats. For example, if the color change value of the screen within a preset duration (such as 1 second) exceeds the preset change threshold (such as 200, RGB ranges from 0 to 255) and the number of changes exceeds the preset number threshold (such as 3 times), and the density of drum beats is large (such as more than 3 drum beats are recognized in the audio frame collected in 1 second), then set the strobe switch parameters to on.

[0043] S4: The lighting control device controls the lighting devices connected thereto according to the lighting control parameters.

[0044] Specifically, before obtaining the audio and video features, the lighting control device will output and control the digital lighting device in real time according to the original preset lighting effects. After obtaining the audio and video features and determining the lighting control parameters, the corresponding channel data value is modified by calling the control interface of the corresponding channel according to the lighting control parameters, so that the lighting effects corresponding to the audio and video features are superimposed on the preset lighting effects in real time, so as to achieve real-time linkage between the external audio and video equipment and the lighting control device, and realize immersive sound and light control effects.

[0045] This embodiment utilizes the characteristics and driving methods of digital lighting equipment to abstract a channel driving control method. At the same time, visual and auditory features are obtained by extracting and analyzing audio and video equipment in the environment. The channel driving control method is associated with audio and video features to achieve the original lighting scene effect. As the audio and video features change, the corresponding associated lighting channels are controlled, and the audio and video and lighting are linked to each other, thereby achieving a better immersive lighting effect experience.

[0046] Embodiment 2 This embodiment is a computer-readable storage medium corresponding to the above embodiment, on which a computer program is stored. When the program is executed by the processor, each step of an immersive lighting control method based on audio and video in the above embodiment is implemented, and the same technical effect can be achieved, which will not be repeated here.

[0047] In summary, the present invention provides an immersive lighting control method based on audio and video and a computer-readable storage medium, which parses audio and video data to extract visual and auditory features, associates audio and video features with lighting control parameters, and controls lighting equipment according to the audio and video features. On the basis of the original lighting scene effect, the lighting effect can be adjusted as the audio and video features change, thereby achieving an immersive lighting effect experience, and the lighting effect can be automatically adjusted without human intervention.

[0048] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An immersive lighting control method based on audio and video, characterized in that: include: The lighting control device obtains audio and video features in real time from the audio and video device connected thereto, wherein the audio and video features are obtained by the audio and video device through real-time analysis of audio and video data; The lighting control device determines the lighting control parameters according to the audio and video characteristics, and controls the lighting device connected thereto according to the lighting control parameters; Among them, the audio and video characteristics include picture color, contrast and drum density; the lamp control parameters include at least one of light color, light brightness, lamp displacement parameters, motor speed and strobe switch parameters.

2. The audio and video based immersive lighting control method according to claim 1, characterized in that: The audio and video equipment analyzes the audio and video data in real time to obtain audio and video features, including: The audio and video device obtains a video frame in the audio and video data in real time as the current video frame, and obtains an audio frame in the audio and video data in real time as the current audio frame according to a preset audio frame length; Determine the screen color corresponding to the current video frame according to the color that accounts for the largest proportion in the current video frame; Determine the contrast corresponding to the current video frame; Drum beat recognition is performed on the current audio frame, and the density of drum beats corresponding to the current audio frame is determined according to the number of recognized drum beats.

3. The audio and video based immersive lighting control method according to claim 2, characterized in that: After determining the picture color corresponding to the current video frame according to the color with the largest proportion in the current video frame, the method further includes: If the picture color is a dark color, the picture color corresponding to the current video frame is determined according to the color that accounts for the largest proportion and is a bright color in the current video frame.

4. The audio and video based immersive lighting control method according to claim 1, characterized in that: The lighting control device determines the lighting control parameters according to the audio and video features, including: The lighting control device determines the lighting color according to the color of the picture; Determine the light brightness based on the contrast; According to the density of drum beats, determine the displacement parameters of the lamp and the motor speed; Determine the strobe switch parameters according to the changes in picture color and the density of drum beats.

5. The audio and video based immersive lighting control method according to claim 4, characterized in that: Determining the light brightness according to the contrast includes: If the contrast is greater than or equal to a preset contrast threshold, the light brightness is set to a first brightness, where the first brightness is greater than or equal to a preset brightness threshold; If the contrast is less than a preset contrast threshold, the light brightness is set to a second brightness, which is less than a preset brightness threshold.

6. The audio and video based immersive lighting control method according to claim 4, characterized in that: The step of determining the displacement parameter of the lamp and the motor speed according to the drum beat density includes: If the drum beat density is less than or equal to a preset first quantity threshold, the displacement step value is increased according to a preset first change value; If the drumbeat density is greater than a preset first quantity threshold and less than or equal to a preset second quantity threshold, the displacement step value is increased according to a preset second change value, and the motor speed is increased according to a preset first multiple; If the density of drum beats is greater than a preset second quantity threshold, the displacement step value is increased according to a preset third change value, and the motor speed is increased according to a preset second multiple.

7. The audio and video based immersive lighting control method according to claim 4, characterized in that: Determining the strobe switch parameters according to the color change of the picture and the density of the drum beats includes: If the change value of the picture color within the preset time length exceeds the preset change threshold and the number of changes exceeds the preset number threshold, and the drum beat density is greater than the preset third number threshold, the strobe switch parameter is set to open.

8. The audio and video based immersive lighting control method according to claim 1, characterized in that: Before the lighting control device acquires the audio and video features from the audio and video device connected thereto in real time, the lighting control device further comprises: The lighting control device obtains the channel information of each lighting device connected thereto, and manages the channels of each lighting device in groups, wherein the channels of the lighting devices include color control channels, displacement control channels, brightness control channels, strobe control channels and motor control channels.

9. The audio and video based immersive lighting control method according to claim 8, characterized in that: The grouping management of the channels of each lighting device includes: According to the function of the channel, the channels of each lighting device are managed in groups to uniformly control the channels with the same function of the lighting equipment; Or, the channels of each lighting device can be grouped and managed according to the lighting device to which the channel belongs, so as to uniformly control all channels of the same lighting device.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images