Automatic lamplight orientation control system and method based on neural network

Abstract

The invention discloses an automatic lamplight directional control system and method based on a neural network. The system comprises a lamplight control module, a neural network module, a target position recognition module and a feedback module. Wherein the neural network module receives target position data (such as image coordinates and environmental parameters) through the input layer, and after feature extraction of the hidden layer (DNN / CNN), the output layer generates a light rotation angle instruction to control directional illumination of a lamp; the target position identification module captures and enhances a target image in real time through an image acquisition and processing unit; the feedback module detects the actual irradiation position error to optimize the neural network prediction precision. The method comprises the steps of obtaining target data in real time and preprocessing, predicting a rotation angle through a neural network, and dynamically adjusting a control strategy in combination with feedback information. High-precision automatic light orientation is realized by using deep learning, the method is suitable for a dynamic environment, and the response speed and the positioning accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent lighting, and particularly relates to an automatic light directional control system and method based on a neural network. BACKGROUND

[0002] The multi-modal light control system is an advanced technical system applied in the field of intelligent lighting, which combines artificial intelligence, sensor technology, data analysis and light control technology to realize intelligent and personalized light control and adjustment. Through the application of the multi-modal light control system in the field of intelligent lighting, users can enjoy intelligent and personalized lighting experience, improve the quality of life and save energy. The system has wide application potential in the fields of smart home, commercial lighting, entertainment places, urban lighting and artistic devices.

[0003] The traditional light control system lacks industry data-driven optimization and lacks analysis data on a large amount of historical data and performance indicators, which cannot be continuously optimized. The system is not automated enough, has low work efficiency, requires high professional degree of operators, and the convenience of complex operation limits the application scenarios and on-site light effects of the light control system. At the same time, the scalability and compatibility are limited, and the industry products lack product upgrades suitable for AI technology.

[0004] In terms of manpower, the attention of light control personnel cannot be maintained efficiently for a long time, and long-time work causes efficiency to decline. The light control personnel are limited by their professional and experience, and the efficiency of operating control equipment is uneven. In the face of emergencies, the optimal method cannot be guaranteed. At the same time, the design and control personnel cannot monitor the overall stage. SUMMARY

[0005] The purpose of the present application is to provide an AI-driven multi-modal light control system, which aims to solve the technical problems existing in the prior art identified in the background.

[0006] The present application is implemented as follows: the AI-driven multi-modal light control system comprises:

[0007] A light control module for selecting a specified position from a monitoring picture and moving a light spot to the specified position in the monitoring picture;

[0008] A light automatic focusing module for automatically focusing the light spot on the specified position after the light spot is moved to the specified position;

[0009] A light spot cutting module for setting a target shape in the monitoring picture, enlarging the size of the light spot until the light spot covers the target shape, and cutting the light spot until the light spot is cut into the target shape;

[0010] A light effect module for automatically adjusting the color and color temperature of the light fixture.

[0011] As a further scheme of the present application, the working process of the light control module is:

[0012] S1, randomly generate a horizontal and vertical rotation (pan, tilt) command, and adjust the horizontal rotation angle pan and the vertical rotation angle tilt of the lamp according to the command;

[0013] S2, acquire a monitoring picture, identify a light spot and calculate the light spot center position coordinates (x, y), if the light spot cannot be identified, re-perform the step S1;

[0014] S3, take the position coordinates (x, y) in the monitoring picture as model input, and take the light command (pan, tilt) value as model output, model, obtain an N-layer neural network model including an input layer, N hidden layers and an output layer;

[0015] S4, acquire the optimal model through the prediction result of the neural network model and deploy, point to the position in the video monitoring picture which needs to be irradiated by the light, and acquire the position information of the pointed position;

[0016] S5, input the position information into the neural network model, calculate the prediction output (pan, tilt) command value through the forward propagation of the neural network, generate the light control signal through the command value, and irradiate the light to the specified position.

[0017] As a further scheme of the present application, the moving light spot to the specified position in the monitoring picture specifically includes:

[0018] S1, acquire the selected pixel position coordinates (Xe, Ye) in the monitoring picture;

[0019] S2, acquire the light spot position coordinates through the monitoring picture;

[0020] S3, acquire the light moving step,

[0021] Let the light spot coordinates be (X1, Y1), the system issues the instruction pan+1 to increase the horizontal rotation of the lamp by one degree, and repeats S2 to acquire the adjusted light spot coordinates (X2, Y2);

[0022] The moving vector of the light spot in the screen is (Xp, Yp) corresponding to each increase of one degree of pan;

[0023] Wherein Xp=X2-X1, Yp=Y2-Y1;

[0024] Let the light spot coordinates be (X1ˋ, Y1ˋ), the system issues the instruction tilt+1 to increase the vertical rotation of the lamp by one degree, and repeats S2 to acquire the adjusted light spot coordinates (X2ˋ, Y2ˋ);

[0025] The moving vector of the light spot in the screen is (Xt, Yt) for each degree of increase in tilt;

[0026] Wherein Xt=X2'-X1', Yt=Y2'-Y1';

[0027] S4, calculate the values P and T of pan and tilt respectively, and the solving formula is:

[0028] Xe=P*Xp+T*Xt

[0029] Ye=P*Yp+T*Yt

[0030] According to the values of P and T, issue a command to adjust the horizontal and vertical rotation angles of the lamp, and move the light spot to the specified position;

[0031] S5, repeat S2 to obtain the light spot coordinates (X1, Y1), if |Xe-X1| and |Ye-Y1| are within the threshold, the light spot has moved to the target position, otherwise repeat S3.

[0032] As a further scheme of the application, the obtaining of the light spot position coordinates comprises:

[0033] By flashing the light, comparing the gray scale picture when the light is off and on, the pixel block position with a brightness difference exceeding the threshold is the position covered by the light spot, the center of the shape is calculated to obtain the light spot position coordinates.

[0034] As a further scheme of the application, the obtaining of the light spot position coordinates comprises:

[0035] Input the macro command to reset all blades, so that the light spot maintains a circular shape.

[0036] Obtain the monitoring picture, and apply Hough circle transformation to identify the circular light spot in the picture.

[0037] As a further scheme of the application, the working process of the light automatic focusing module is:

[0038] S1, obtain the distance from the light to the light receiving surface and the parameters set by the user, including: zoom, iris, and gobo1, gobo2, and gobo3;

[0039] S2, establish an Actor model, integrate all the above data as the input of the Actor model, and calculate the focus value;

[0040] S3, generate a focus adjustment command according to the focus value, and adjust the lamp to achieve the focal length specified by the calculation module to complete the focusing operation.

[0041] As a further aspect of the present application, the enlarged spot size is adjusted until the spot covers the target shape, specifically comprising:

[0042] S1, draw the target shape in the monitoring picture;

[0043] S2, output macro command to adjust zoom and iris to minimize the spot;

[0044] S3, obtain the polygon vertex position coordinates of the target shape, and obtain the average gray value of the pixels in the area around each vertex;

[0045] S4, output macro command to adjust the zoom value and iris value to gradually enlarge the spot;

[0046] S5, determine whether the gray value increment of the area around each vertex exceeds the threshold value, if it exceeds the threshold value, the target polygon is covered by the spot;

[0047] If it does not exceed the threshold value, repeat S3 and S4 until the gray value increment exceeds the threshold value or the spot cannot be enlarged any more;

[0048] If the gray value does not change, it indicates that the minimum spot has initially covered the target shape or that the maximum spot cannot cover the target;

[0049] If it indicates that the minimum spot has initially covered the target shape, repeat S1;

[0050] If it indicates that the maximum spot cannot cover the target, abandon the spot cutting operation.

[0051] As a further aspect of the present application, the spot is cut until the spot is cut into the target shape, specifically comprising:

[0052] S1, output macro command to set the spot as a rectangle, and identify the horizontal rotation angle of the spot;

[0053] S2, use edge detection and contour finding algorithm to identify the rectangular contour coordinates of the spot, and use the rotation puzzle algorithm to obtain the horizontal rotation angle of the rectangular shape of the spot;

[0054] S3, use the rotation puzzle algorithm to obtain the minimum circumscribed rectangle of the target shape and its horizontal rotation angle;

[0055] S4, calculate the rotation angle difference between the rectangular spot and the minimum circumscribed rectangle of the target shape, and control the rotation angle of the blade disc to match the rotation angle of the spot with the target shape;

[0056] S5, perform the spot cutting operation, and determine whether the spot fits the target shape by detecting the brightness change of the point pixels.

[0057] As a further scheme of the present application, the working process of the light effect module is:

[0058] Capture the color of the current light in the picture captured by the camera, and convert the color RGB to the color space CMY

[0059] Calculate the error value for each color channel respectively:

[0060] e(t) = target channel value - actual channel value

[0061] The target channel value and the actual channel value are obtained by color decomposition;

[0062] Calculate the parameters of each color channel using the PID control formula:

[0063] u(t) = K p ·e(t) + K i ·∫e(t)dt + K d ·dtde(t)

[0064] Where: K p , K i , K d are the PID parameters of each channel, u(t) is the adjusted output value, used to adjust the light color, e(t) is the calculated error;

[0065] According to the output u(t) of the PID controller, issue a light control command to adjust the parameter value of each color channel and modify the light color output.

[0066] The beneficial effects of the present application are:

[0067] This scheme proposes a method of combining manual operation and reinforcement learning, which can automatically complete shape cutting according to the instructions of the lighting director, thereby saving manpower, and the model performance is not affected by the use environment. Even if the camera is blocked, the light spot shape and color of the lamp change, the overall light changes on site, etc. do not affect the model operation, at the same time the hardware is simple and easy to deploy and maintain, the cost is low. BRIEF DESCRIPTION OF DRAWINGS

[0068] Figure 1 The structural block diagram of the AI-driven multi-modal light control system provided by the embodiment of the present application is shown in the figure;

[0069] Figure 2 The structural schematic diagram provided by the embodiment of the present application is shown in the figure;

[0070] Figure 3 The neural network model example diagram is shown in the figure;

[0071] Figure 4 The structural block diagram of the transfer learning scheme is shown in the figure;

[0072] Figure 5 Structure block diagram of the light automatic focusing module provided in the embodiment of the present application;

[0073] Figure 6 Relationship diagram of Zoom and Focus with Gobo focusing;

[0074] Figure 7 Relationship diagram of Zoom and Focus without Gobo focusing. DETAILED DESCRIPTION

[0075] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0076] It can be understood that the terms "first", "second", and the like used in the present application can be used herein to describe various elements, but unless specifically stated, these elements are not limited by these terms. These terms are only used to distinguish the first element from another element. For example, without departing from the scope of the present application, the first xx script can be referred to as the second xx script, and similarly, the second xx script can be referred to as the first xx script.

[0077] As shown in Figure 1 and Figure 2 , the AI-driven multi-modal light control system, the system comprises:

[0078] A light control module for selecting a specified position from a monitoring picture and moving a light spot to the specified position in the monitoring picture;

[0079] A light automatic focusing module for automatically focusing the light spot on the clarity after the light spot is moved to the specified position;

[0080] A light spot cutting module for setting a target shape in the monitoring picture and expanding the size of the light spot until the light spot covers the target shape, and cutting the light spot until the light spot is cut into the target shape;

[0081] A light effect module for automatically adjusting the color and color temperature of the lamp.

[0082] As shown in Figure 3 , in the embodiment of the present application, the working process of the light control module is:

[0083] S1, randomly generate horizontal and vertical rotation (pan, tilt) commands, and adjust the horizontal rotation angle pan and the vertical rotation angle tilt of the lamp according to the commands;

[0084] S2, acquire a monitoring picture, identify a light spot and calculate a light spot center position coordinate (x, y), if the light spot cannot be identified, re-perform S1 step;

[0085] S3, take the position coordinate (x, y) in the monitoring picture as a model input, take the light command (pan, tilt) value as a model output, perform modeling, and obtain an N-layer neural network model including an input layer, N hidden layers and an output layer;

[0086] S4, acquire an optimal model through a prediction result of the neural network model and deploy, point to a position in the video monitoring picture which needs to be irradiated by the light, and acquire the pointed position information;

[0087] S5, input the position information into the neural network model, calculate a prediction output (pan, tilt) command value through forward propagation of the neural network, generate a light control signal through the command value, and irradiate the light to the specified position.

[0088] In the process of constructing the neural network model, the model also needs to be trained,

[0089] 1. Hyperparameter selection:

[0090] Hidden layer neuron number: Determine the number of hidden layers and neurons in each layer through cross-validation method. Start with a small number to avoid overfitting, and gradually increase the number of neurons to improve the learning ability of the model until the performance on the validation set no longer improves.

[0091] Learning rate: Select adaptive learning rate adjustment strategies such as learning rate decay or Adam optimizer to quickly converge in the early stage of training and stabilize learning in the later stage of training.

[0092] Batch size: Select an appropriate batch size according to available computing resources. Smaller batches can provide more frequent model updates, while larger batches may benefit from memory optimization and accelerated computation.

[0093] Activation function: Compare the effects of ReLU, Sigmoid and Tanh functions at different levels to select the activation function that produces the best training results.

[0094] Number of iterations (Epochs): Determine the number of iterations through early stopping method. If the performance of the validation set does not improve significantly in consecutive epochs, stop training to prevent overfitting.

[0095] Regularization: To prevent overfitting, apply L2 regularization to the weights, and select an appropriate regularization coefficient to balance model complexity and generalization ability.

[0096] Weight Initialization: He initialization or Xavier initialization is used to optimize the weights, providing a good starting point for the training process, helping the gradient descent algorithm work more effectively.

[0097] 2. Training Process:

[0098] After determining the hyperparameters, the neural network begins training. Training uses forward propagation to calculate the predicted output, then updates the weights through backpropagation. In each iteration, the loss function (such as mean squared error) is calculated and optimized through the gradient descent algorithm. During the training process, model performance will be monitored and optimized by adjusting hyperparameters such as learning rate.

[0099] 3. Model Evaluation and Saving:

[0100] Evaluate the model using the test dataset to determine the model's generalization ability. For the best-performing model configuration, save its parameters for use in actual fixture position adjustment.

[0101] Further, the following methods can be used to identify the light spot position in the picture.

[0102] 1. Shape Recognition Method

[0103] A. Output blade signal, return the light blade to the initial position, ensure the light spot output ideal circular state.

[0104] B. Picture preprocessing: Obtain the camera monitoring picture. Perform grayscale processing to reduce computational complexity. Apply filtering algorithm to remove picture noise. Perform binaryzation processing to highlight shape outline.

[0105] C. Edge detection: Find the shape boundary in the image through edge detection algorithm (such as canny edge detector).

[0106] D. Circle-like detection: Identify the circle-like light spot in the picture through Hough transform algorithm and calculate the center coordinates.

[0107] 2. Flash method to identify the center point of the light spot in the monitoring picture

[0108] A. Output light brightness signal (fixture@100%). The light control unit adjusts the light to the brightest.

[0109] Obtain the monitoring picture and save it as image 1.

[0110] B. Output light brightness signal (fixture@0%). The light control unit adjusts the light to the darkest. Obtain the monitoring picture and save it as image 2.

[0111] C. Perform grayscale conversion, noise reduction, and other preprocessing on images 1, 2.

[0112] D. Compare the processed images 1, 2, and set a suitable threshold to convert the difference image into a binary image, where the light spot region is white and the rest is black.

[0113] E. Find the connected white regions in the binary image and calculate the geometric center of the white region. Use the average of pixel coordinates to determine the light spot center position.

[0114] 3. Color contrast method

[0115] Light color setting and image capture

[0116] A. Output light color adjustment commands. The light control unit sets the light to red, green, and blue in sequence. For each color illumination state, capture the monitoring screen and save it as image 1, image 2, and image 3, respectively.

[0117] B. Color space conversion and image preprocessing

[0118] C. Convert the captured RGB images to HSV or LAB color space, which helps more effectively perform color segmentation and analysis.

[0119] D. Apply histogram equalization to each color space converted image to improve image contrast and make color differences more clearly identifiable.

[0120] Difference analysis

[0121] E. Use image processing techniques (such as difference images and color histogram comparison) to analyze color changes between different images. Extract significant changes in the light spot region, which shows the greatest difference under different color illumination.

[0122] Binaryzation and morphological processing

[0123] F. Automatically calculate the optimal binaryzation threshold based on the results of difference analysis (e.g., using the Otsu method). Convert the difference image to a binary image, where the light spot region is white and the rest is black.

[0124] G. Perform morphological operations such as opening or closing on the binary image to remove small noise points and smooth the light spot boundary, further refining the light spot shape.

[0125] Light spot center positioning

[0126] H. Find the connected white regions in the binary image. Apply geometric analysis to each white region to calculate its geometric center, and use the average of pixel coordinates or least squares fitting of a circle to determine the precise center position of the light spot.

[0127] As Figure 4 shown, in actual use scenarios, such as changes in camera position, the accuracy of the pre-trained model will be affected.

[0128] Model transfer learning method

[0129] When the spatial positions of the camera, working surface, and light fixture change, the (pan, tilt) and (x, y) data statistical characteristics will change, resulting in a decrease in model prediction accuracy.

[0130] This solution uses a transfer learning method to solve this problem. The specific description is as follows:

[0131] Fine-tuning method:

[0132] Selection layer: determines which layers need to be frozen and which layers need to be updated.

[0133] Training parameters: use a new dataset to train the unfrozen layers, adjust the learning rate and batch size to optimize the training process.

[0134] Adapter model method:

[0135] Adapter model g(x', y'), which can map new input (x', y') back to the input space (x, y) of the original model.

[0136] Model construction: define the structure of the Adapter model g(x', y'), determine the number and type of neural network layers required.

[0137] Train the Adapter model: train the Adapter model using only new data while keeping the parameters of the original model f(x, y) fixed.

[0138] Integration: connect the output layer of the Adapter model with the input layer of the original model to ensure that the overall model can seamlessly process changed input data.

[0139] Model performance is not affected by the use environment. After the model is trained, the prediction of pan and tilt values does not depend on any external environment. Even if the camera is blocked, the shape and color of the light spot change, the overall light changes, etc., it does not affect the model operation.

[0140] This solution only needs to add a camera, which is simple in hardware, easy to deploy and maintain, and low in cost.

[0141] Compared with alternative solutions, this method can directly illuminate the target area with the light fixture without the need for multiple calculations of (pan, tilt) values for step-by-step adjustment.

[0142] Alternative

[0143] I. Vector calculation based method

[0144] Step 1: Obtain the pixel position coordinate (Xe, Ye) selected by the operator in the monitoring screen.

[0145] Step 2: Obtain the light spot position coordinate through the light spot recognition scheme in the foregoing scheme.

[0146] Step 3: Calculate the light moving step length according to the following method.

[0147] Record the light spot coordinate as (X1, Y1), the system issues an instruction (pan+1) to increase the horizontal rotation of the lamp by one degree, and repeats step 2 to obtain the adjusted light spot coordinate (X2, Y2). The moving vector of the light spot in the screen corresponding to each increase of one degree of pan is (Xp, Yp). Wherein Xp=X2-X1, Yp=Y2-Y1. The calculation amount needs to be increased to obtain the accuracy.

[0148] Similarly, the moving vector corresponding to the vertical rotation (tilt+1) of the lamp is (Xt, Yt)

[0149] Calculate the values P and T that need to be moved for pan and tilt. Xe=P*Xp+T*Xt, Ye=P*Yp+T*Yt, and solving P and T.

[0150] Issue the pan and tilt command, and the control unit adjusts the horizontal and vertical rotation angles of the lamp to move the illumination position of the lamp.

[0151] Step 5: The system repeats step 2 to obtain the light spot coordinate (X1, Y1). If |Xe-X1| and |Ye-Y1| are within the threshold, the light spot has moved to the target position. Otherwise, repeat step 3.

[0152] II. Vector calculation method integrated with PID algorithm

[0153] Step 1: Target position acquisition

[0154] Obtain the pixel position coordinate (Xe, Ye) in the monitoring screen. These coordinates represent the target position to which the light spot needs to be moved.

[0155] Step 2: Light spot position recognition

[0156] Obtain the position coordinate (X1, Y1) of the current light spot through the light spot recognition scheme. This step can adopt the light spot recognition method described in the foregoing scheme.

[0157] Step 3: PID controller integration and error calculation

[0158] Proportional (P): Calculate the current error ex = Xe - X1 and ey = Ye - Y1, and multiply by the proportional coefficient Kp.

[0159] Integral (I): Accumulate the history error, update the integral term Ix and Iy, and multiply by the integral coefficient Ki.

[0160] Derivative (D): Calculate the error rate of change, i.e. the difference from the error of the last period, and then multiply by the derivative coefficient Kd.

[0161] Add the proportional, integral, and derivative outputs to get the final control signal

[0162] ux = Px + Ix + Dx

[0163] uy = Py + Iy + Dy

[0164] Step four: execute the control instruction

[0165] The control unit issues the corresponding pan + ux and tilt + uy commands to adjust the horizontal and vertical rotation angles of the luminaire according to the output of the PID controller.

[0166] The luminaire adjusts its position according to the control signal to move the light spot to the target position.

[0167] Step five: position confirmation and feedback adjustment

[0168] The system repeats step two to obtain the adjusted light spot coordinates (X1, Y1).

[0169] Check the error between the light spot position and the target position. If |Xe - X1| and |Ye - Y1| are within the preset threshold range, it is confirmed that the light spot has been positioned correctly.

[0170] If the error exceeds the threshold, the system recalculates the control signal according to the current error and returns to step four for further adjustment until the error meets the requirements.

[0171] In an embodiment of the present application, the light spot position coordinates are obtained by:

[0172] By flashing the light, comparing the grayscale pictures when the light is off and on, and the pixel block position with a brightness difference exceeding the threshold is the light spot covered position. Calculate the center of this shape to obtain the light spot position coordinates.

[0173] In an embodiment of the present application, the light spot position coordinates are obtained by:

[0174] Input the macro command to reset all blades, so that the light spot maintains a circular shape.

[0175] Obtain the monitoring picture, and apply Hough circle transformation to identify the circular light spot in the picture.

[0176] As Figures 5 to 7 shown in the embodiment of the application, the working process of the light automatic focusing module is as follows:

[0177] S1, obtain the distance of light to light receiving surface and the parameters set by the user, including zoom, iris, and gobo1, gobo2, and gobo3;

[0178] S2, establish an Actor model, integrate all the above data as the input of the Actor model, and calculate the focus value;

[0179] S3, generate a focus adjustment command according to the focus value, and adjust the lamp to achieve the focal length specified by the calculation module to complete the focusing operation.

[0180] In the focusing process, there are four parts including reinforcement learning environment building, model training, intelligent light control, and feedback cycle.

[0181] Reinforcement learning environment building

[0182] The reinforcement learning environment building includes state parameters, reward calculation method design, and action space setting.

[0183] State parameter

[0184] The state of the system is defined by the following elements: distance, zoom, iris, and the states of the three gobos (gobo1, gobo2, gobo3). Here, distance, zoom, and iris are continuous floating-point values, while the states of the gobos are Boolean values (1 represents that the gobo is on, and 0 represents that the gobo is off).

[0185] Reward calculation method

[0186] The reward is based on the sharpness of the light spot edge. First, we collect monitoring pictures from different states when the focus is adjusted manually by the lighting designer, and use the Sobel operator to calculate the edge sharpness (i.e. the gray gradient) of these pictures. We manually set an acceptable gray gradient range [g_min, g_max] as the sharpness standard. In real-time operation, if the gray gradient produced by the AI adjusted focal length falls within this range, the AI will receive a reward; otherwise, the reward is 0.

[0187] Action space setting

[0188] Action space: The focus value that the AI ​​agent can adjust is defined as a continuous range. For example, if the focus specification is from 10mm to 100mm, then the action space would be [10, 100]. This setting allows the AI ​​to find the optimal focus value within a given range.

[0189] Model training

[0190] This method uses the DDPG algorithm with an Actor-Critic structure.

[0191] 1. Network Architecture

[0192] Actor Network: Consists of multiple fully connected layers. The input is the system state [distance, zoom, iris, gobo1, gobo2, gobo3], and the output is a continuous action value, i.e., the focus parameter. ReLU activation function is used to add non-linearity, and the last layer uses tanh or sigmoid function to ensure the output action is within a reasonable range.

[0193] Critic networks, also composed of multiple fully connected layers, take the state as input and the action values ​​generated by the Actor as input, and output the expected reward value of this state-action pair. Critic networks help evaluate the decisions of Actor networks.

[0194] 2. Training methods

[0195] Critic Network Update

[0196] The goal of a Critic network is to approximate the true Q-function, which gives the expected reward of taking action a given state s. The parameters of the Critic network are updated by minimizing a loss function L, which measures the error between the Critic network's predicted Q-value and the target Q-value. The loss function L has the following form:

[0197] L(θ Q ) = E s,a,r,s′ [(Q(s,a|θ Q )-y) 2 ]

[0198] Where, θ Q These are the parameters of the Critic network, where y is the target Q-value, calculated as follows:

[0199] y=r+γQ′(s′,μ′(s′∣θμ′)∣θQ′);

[0200] s' is the new state after taking action a. γ is the discount factor, used to reduce the impact of future rewards. Q' and μ' are the predictions of the target Critic and Actor networks, respectively. θQ' and θμ' are the parameters of the target Critic and Actor networks, respectively.

[0201] Actor network updates The Actor network is responsible for directly outputting the optimal action a. The parameters θμ of the Actor network are updated by the following gradient ascent rule to maximize the expected return estimated by the Critic network:

[0202]

[0203] ▽ a Q((s,a|θ Q ))| a＝μ(s) is the gradient of the Critic network with respect to action a, evaluating the effect of taking the current Actor network's recommended action in state s.

[0204] is the gradient of the Actor network output with respect to its parameters.

[0205] Intelligent light control

[0206] Ranging module operation

[0207] Function: automatically obtain the distance from the light to the light-receiving surface.

[0208] Data transmission: transmit the measured distance data to the computing module in real time.

[0209] User interaction module settings

[0210] Function: allow users to set other parameters of the light through an interactive interface, including zoom (focal length scaling), iris (aperture size), and the states (on or off) of the three gobos (gobo1, gobo2, gobo3).

[0211] Data transmission: transmit the user-set parameter values to the computing module.

[0212] Computing module processing

[0213] Data integration: after receiving data from the ranging module and the user interaction module, the computing module integrates these data as inputs to the Actor model.

[0214] Model operation: the input data includes [distance, zoom, iris, gobo1, gobo2, gobo3], and the computing module uses the trained Actor model to calculate the corresponding focus value.

[0215] Transmit focus command

[0216] Command sending: The computing module generates a focus adjustment command based on the output of the Actor model and sends this command to the light control module.

[0217] Light control module focuses

[0218] Perform focusing: After receiving the focus command, the light control module adjusts the lamp to achieve the focal length specified by the computing module, completing the focusing operation.

[0219] Feedback loop light focusing

[0220] Auto focus: The computing module automatically calculates the focus value using the trained Actor model and adjusts the lamp through the light control module to achieve preliminary focusing.

[0221] Manual fine-tuning: If the lighting designer finds that the focusing effect is not ideal in actual use, they can manually input a new focus command through the interaction module for fine-tuning. This allows the lighting designer to make fine adjustments based on the actual situation on site to achieve the best focusing effect.

[0222] Data collection and storage

[0223] Data storage module: All lighting setting parameters [distance, zoom, iris, gobo1, gobo2, gobo3] and focus values adjusted by the lighting designer will be recorded and stored in the data storage module. These data are labeled as a labeled dataset for subsequent model training and optimization.

[0224] Data structure: Each piece of data includes complete state parameters and corresponding adjusted focus values, forming a structured dataset.

[0225] Continuous optimization of the model

[0226] Regular fine-tuning: At certain time intervals or when the accumulated data reaches a certain amount, the Actor model is fine-tuned using these labeled datasets.

[0227] Fine-tuning process: Use transfer learning and fine learning rate to adjust model parameters.

[0228] When the distance between the lamp and the illuminated surface is 4.475m, as shown in Figure 6 , the parameters of the fitted polynomial model are: f(x) = ax6 + bx5 + cx4 + dx3 + ex2 + fx + g

[0229] The parameters are as follows:

[0230] a = -5.96 x 10"11a = -5.96 x 10"11

[0231] b = 2.82 x 10"8b = 2.82 x 10"8

[0232] c = -6.80 x 10"6c = -6.80 x 10"6

[0233] d = 9.64 x 10"4d = 9.64 x 10"4

[0234] e = -7.82 x 10"2e = -7.82 x 10"2

[0235] f = 3.07f = 3.07

[0236] g = -11.49g = -11.49

[0237] As shown in the following table, the relationship between Focus and Zoom is found by analyzing the data: Figure 7 Focus = -0.334 x Zoom + 91.63 in the interval of Zoom > 50, and Focus = 73.7 in the interval of 30 to 50.

[0238] In the embodiment of the present application, the enlarged spot size is adjusted until the spot covers the target shape, which specifically includes:

[0239] S1, drawing a target shape in the monitoring screen;

[0240] S2, outputting a macro command to adjust the zoom and iris to the minimum spot size;

[0241] S3, obtaining the polygon vertex position coordinates of the target shape, and obtaining the average gray value of the pixels in the area around each vertex;

[0242] S4, outputting a macro command to adjust the zoom value and iris value to gradually enlarge the spot;

[0243] S5, judging whether the gray value increment of the area around each vertex exceeds the threshold value, if the threshold value is exceeded, the target polygon is covered by the spot;

[0244] If the threshold value is not exceeded, repeat S3 and S4 until the gray value increment exceeds the threshold value or the spot cannot be enlarged any more;

[0245] If the gray value does not change, it indicates that the minimum spot has initially covered the target shape or that the maximum spot cannot cover the target;

[0246] If it indicates that the minimum spot has initially covered the target shape, repeat S1;

[0247] If it indicates that the maximum light spot cannot cover the target, the light spot cutting operation is abandoned.

[0248] In the embodiment of the present application, the cutting of the light spot until the light spot is cut into the target shape specifically comprises:

[0249] S1, output a macro command to set the light spot to a rectangle, and identify the horizontal rotation angle of the light spot;

[0250] S2, identify the rectangular contour coordinates of the light spot using an edge detection and contour finding algorithm, and obtain the horizontal rotation angle of the rectangular shape of the light spot using a rotation jamming algorithm;

[0251] S3, obtain the minimum circumscribed rectangle of the target shape and its horizontal rotation angle using a rotation jamming algorithm;

[0252] S4, calculate the rotation angle difference between the rectangular light spot and the minimum circumscribed rectangle of the target shape, and control the rotation angle of the blade disc to make the rotation angle of the light spot match the target shape;

[0253] S5, perform the light spot cutting operation, and judge whether the light spot fits the target shape by detecting the pixel brightness change of the point.

[0254] The light spot cutting module includes three operation stages:

[0255] The first stage is to move the light spot to the geometric center position of the target shape. The second stage is to expand the light spot to cover the target shape. The third stage is to cut the light spot to the target shape by AI.

[0256] First stage: move the light spot to the geometric center position of the target shape:

[0257] 1. The light control unit adjusts the zoom and iris to set the light spot to the minimum.

[0258] 2. Calculate the geometric center coordinates of the drawn shape.

[0259] 3. Artificially or using the method described in "A Computer Vision Based Intelligent Positioning Method for Light Irradiation Direction and System Thereof", the light is irradiated to the shape center position.

[0260] Further, the geometric center coordinates can be realized by calculating the centroid of the shape:

[0261] 4. Obtain the shape contour coordinates

[0262] 5. Calculate the moments of the contour

[0263] 6. Calculate the centroid: use the formula of the moments to calculate the centroid of the shape. The x and y coordinates of the centroid can be calculated by the moments M: Cx=M10 / M00, Cy=M01 / M00

[0264] Where M10 and M01 are the first order moments, and M00 is the zero order moment, i.e. the area.

[0265] Second stage: enlarge the spot size to cover the target shape:

[0266] 1. First, get the position of each vertex of the target polygon, and calculate the average gray value of the 3x3 pixel area centered on each vertex, define this value as the "initial gray value".

[0267] 2. Send a macro command to the light control unit, instruct it to adjust the zoom and iris parameters of the spot to gradually enlarge the spot.

[0268] 3. After enlarging the spot, re-measure the new gray value of the above-mentioned vertex area, and calculate the difference between the new gray value and the initial gray value.

[0269] If the difference exceeds the preset threshold n, it is considered that the target polygon has been successfully covered by the spot.

[0270] If the difference does not exceed the threshold n, continue to return to step 2 and repeat the adjustment of the spot size.

[0271] 4. When the spot cannot be further enlarged (reaching the maximum zoom and iris values allowed by the device), if the change in gray value still does not exceed the threshold n, it is considered that the spot has covered the target polygon at the initial size. At this time, a macro command will be sent to the light control unit to instruct it to adjust the spot to the initial or minimum setting.

[0272] Third stage: AI cuts the spot to the target shape

[0273] 1. Train the AI operation of 8 blades cutting and overall rotation through reinforcement learning, control [1A, 1B, 2A, 2B, 3A, 3B, 4A, 4B, rotate] command to achieve shape cutting.

[0274] 2. This part further includes four parts of reinforcement learning environment building, model training, intelligent light control and feedback loop, which are realized by the following methods:

[0275] a. Reinforcement learning environment building

[0276] Reinforcement learning environment building includes state parameter, design reward calculation method, action space setting.

[0277] State state parameter

[0278] The state of the system is the target contour pixel coordinates Box [(x, y), (x, y), (x, y), (x, y), (x, y), (x, y)…], and the spot edge coordinates Contour: [(x, y), (x, y), (x, y), (x, y), (x, y), (x, y)…]

[0279] b. Reward reward calculation method

[0280] On each edge of the target shape, two pixel points are set inside and outside as detection points, and the initial brightness values of these detection points are recorded, and a brightness drop threshold x% is set to determine light and dark. Until all the outer detection points of the edge are dark and the inner detection points are bright, it is considered that the shape cutting is completed. When the shape cutting is completed, a large amount of reward is given to the AI agent. In addition to this, the reward is 0.

[0281] c. Action space action space setting

[0282] Action space: eight blade position values that the AI agent can adjust:

[0283] [1A, 1B, 2A, 2B, 3A, 3B, 4A, 4B, rotate], each blade value is defined as a continuous interval. For example, 1A ∈ [0, 100]. Such a setting allows the AI to find the optimal blade position within the given range.

[0284] And specifically to the intelligent light control part, including:

[0285] 1. Calculation processing

[0286] Data integration: after obtaining the data of the camera screen and the user interaction module, integrate these data as the input of the Actor model.

[0287] Model operation: the input data includes the target contour pixel coordinates Box [(x, y), (x, y), (x, y), (x, y), (x, y), (x, y)…], and the spot edge coordinates Contour: [(x, y), (x, y), (x, y), (x, y), (x, y), (x, y)…]

[0288] The calculation module uses the trained Actor model to calculate the corresponding eight blade position values [1A, 1B, 2A, 2B, 3A, 3B, 4A, 4B, rotate].

[0289] 2. Transmission of focus command

[0290] Command sending: the calculation module generates the focus adjustment command according to the output of the Actor model, and sends this command to the light control module.

[0291] Light control module focus

[0292] Perform focus: After the light control module receives the command, adjust the blade position to complete the cutting operation.

[0293] 3. Feedback loop

[0294] 1. Light cutting

[0295] Automatic cutting: The calculation module uses the trained Actor model to automatically calculate the blade and rotation parameters, and adjusts the lamp through the light control module to achieve shape cutting.

[0296] Manual fine-tuning: If the lighting designer finds that the cutting effect is not ideal, they can manually input new cutting commands through the interaction module for fine-tuning. This allows the lighting designer to make fine adjustments based on the actual situation on site to achieve the best focus effect.

[0297] 2. Data collection and storage

[0298] Data storage module: All state parameters and action values adjusted by the lighting designer will be recorded and stored in the data storage module. These data are labeled as a labeled dataset for subsequent model training and optimization.

[0299] Data structure: Each piece of data includes complete state parameters and corresponding adjusted action values, forming a structured dataset.

[0300] 3. Continuous optimization of the model

[0301] Regular fine-tuning: At certain time intervals or when the accumulated data reaches a certain amount, use these labeled datasets to fine-tune the Actor model.

[0302] Fine-tuning process: Use transfer learning and fine learning rate to adjust model parameters.

[0303] For the above scheme, alternative solutions can also be used:

[0304] Premise: The user draws the desired shape, such as a trapezoid, in the monitoring screen through the interaction unit.

[0305] This method includes three operating stages:

[0306] First stage: Place the light spot to the center position of the target shape.

[0307] Second stage: Expand the light spot to cover the target shape.

[0308] Third stage: Move the blade to cut the light spot to the target shape.

[0309] The main difference from the previous solution is in the third stage, only the difference in the third stage is introduced here:

[0310] Third stage: Cutting to target shape

[0311] Step 1: Set the spot shape

[0312] Output command to control unit, instruct to adjust the cutting light, so that it expands to 25% in each of the four main directions, forming an approximate rectangular spot.

[0313] Step 2: Identify the spot shape and its rotation angle

[0314] 1. Get the monitoring picture of the current working area through the camera, and use edge detection and contour finding algorithm to identify the contour coordinates of the rectangular spot.

[0315] 2. Use the rotation puzzle algorithm to calculate the horizontal rotation angle of the spot.

[0316] Step 3: Get the target shape parameters

[0317] Output command to control unit, use rotation puzzle algorithm to identify and get the minimum circumscribed rectangle of the target shape and its horizontal rotation angle.

[0318] Step 4: Match the rotation angle of the spot and the target shape

[0319] Calculate the rotation angle difference (denoted as x) between the spot and the target shape. Then output command to control unit to adjust the blade disc rotation angle x to match the rotation angle of the spot to the target shape.

[0320] Step 5: Perform cutting

[0321] 1. Set two pixel points inside and outside each edge of the target shape as detection points and record the initial brightness values of these detection points, and set a brightness drop threshold x% to determine light and dark.

[0322] 2. According to the real-time collected detection point brightness, automatically adjust the position of the blade:

[0323] If both the inside and outside detection points are bright, the blade moves one step towards the inside.

[0324] If the outside detection point is dark and the inside detection point is bright, the blade stops.

[0325] If the inside detection point is dark and the outside detection point is also dark, the blade retreats a small step towards the outside.

[0326] 3. The control command is issued, and the control unit repeats the above adjustment for the blade operation of each edge until all the outer detection points are dark and the inner detection points are bright, at which time it is considered that the shape cutting is completed.

[0327] 4. If it is detected that the blade has reached the mechanical limit and cannot complete more cutting, the operation will be judged and ended according to the macro command value.

[0328] In the embodiment of the present application, the working process of the light effect module is as follows:

[0329] The color of the current light in the picture captured by the camera is captured, and the color RGB is converted into the color space CMY;

[0330] The error value is calculated for the cyan, magenta, and yellow channels respectively:

[0331] e(t) = target channel value - actual channel value

[0332] The target channel value and the actual channel value are obtained by color decomposition;

[0333] The parameters of each color channel are calculated using the PID control formula:

[0334] u(t) = K p ·e(t) + K i ·∫e(t)dt + K d ·dtde(t)

[0335] Where: K p , K i , K d are the PID parameters of each channel, u(t) is the adjusted output value, used to adjust the light color, e(t) is the calculated error;

[0336] According to the output u(t) of the PID controller, a light control command is issued to adjust the parameter value of each color channel and modify the light color output.

[0337] It should be understood that although each step in the flowchart of each embodiment of the present application is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise stated herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least a part of the steps in each embodiment can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least a part of other steps or sub-steps or stages of other steps.

[0338] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer readable storage medium, and when the program is executed, the processes of the above-mentioned embodiment methods can be included. Any reference to memory, storage, database or other medium used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0339] Any combination of the technical features of the above-mentioned embodiments can be combined. In order to make the description simple, all possible combinations of the technical features in the above-mentioned embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0340] The above-mentioned embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as limiting the scope of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

[0341] The above-mentioned is only the preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A neural network based automatic light orientation control system, characterized in that, The system comprises: a light control module for automatically adjusting the horizontal and vertical rotation angles of the light fixture according to the input signal of the target position, so that the light is irradiated to the specified position, the light control module controls the rotation angle of the light device by receiving the light adjustment instruction generated by the neural network module; a neural network module for predicting the horizontal and vertical rotation angles of the light that need to be adjusted according to the real-time input target position data through the neural network model, and generating the corresponding light control instruction, the neural network module comprises: an input layer for receiving target position data, the data including image data, video stream, and other environmental parameters, the input data including the position coordinates of the target in the image and its dynamic characteristics; at least one hidden layer for extracting and processing features from the input data, the hidden layer uses a deep neural network (DNN) or a convolutional neural network (CNN) architecture to enhance the feature extraction capability of the target position; an output layer for outputting light adjustment instructions, the output layer outputs the corresponding horizontal and vertical rotation angles according to the processing results to instruct the light device to adjust the orientation. a target position recognition module for capturing image information of the target position in real time, the module comprises: an image acquisition unit for acquiring environmental images, including image data of the area where the target position is located; an image processing unit for processing image data, including edge detection, image preprocessing, target recognition and other operations, and inputting the processed data to the neural network module for target position prediction and light directional control by the neural network. a feedback module for receiving feedback signals after light adjustment, calculating the actual irradiation position of the light, and returning the feedback information to the neural network module to optimize the light directional control strategy.

2. The system of claim 1, wherein, The neural network module is trained by a deep learning algorithm, a large amount of historical target position data and light control data are used in the training process, and the neural network is optimized by using the back propagation algorithm, so that the neural network module can quickly and accurately predict the rotation angle required for light directional control according to new target position data, the training process comprises: a data preprocessing module for converting historical data into a format suitable for neural network training, the data including image data of the target position and rotation angle data of the light control; a model training module for training the neural network using historical data to optimize the weights of the neural network and ensure that the neural network can accurately predict the light directional control.

3. The system of claim 1, wherein, The target position recognition module further comprises: a target tracking module for tracking the position of the target in real time, continuously identifying the target in a dynamic environment, and updating the data input to the neural network module according to the movement of the target. An image enhancement module is used to optimize image quality through image enhancement techniques, including adaptive filtering, brightness adjustment, contrast enhancement, noise removal and edge sharpening operations, to improve the clarity and contrast of image data, thereby improving the quality of neural network input data and enhancing the accuracy of target position prediction.

4. The system of claim 1, wherein, The feedback module includes: A position detection module is used to detect the actual illumination position of the light in real time, which includes an angle sensor and a position sensor, capable of accurately measuring the horizontal and vertical rotation angles of the light; An error calculation module is used to calculate the error between the light illumination position and the target position, and generate a feedback signal, which is fed back to the neural network module to optimize subsequent light directional adjustment.

5. A neural network-based automatic light orientation control method, characterized by, The method includes: Real-time acquisition of target position data, including image data, video stream or environmental sensor data, through image acquisition unit to obtain image information of target object or area; Through image processing unit to process the captured image data, including edge detection, target recognition, target tracking and image enhancement operations, and input the processed data into the neural network module; Through the neural network module to predict the light rotation angle corresponding to the target position, generate light control instruction, adjust the horizontal rotation angle and vertical rotation angle of the light; Real-time acquisition of feedback information after light adjustment, calculation of the error between the actual illumination position of the light and the target position, and return of the error information to the neural network module to optimize the light directional control strategy.

6. The method of claim 5, wherein, The image processing step includes: Real-time tracking of the position of the target in the image through a target tracking algorithm to obtain the coordinate data of the target; Using image enhancement techniques to optimize image quality, adjusting the brightness, contrast and clarity of the image, and improving the input data quality of the neural network module.

7. The method of claim 5, wherein, The neural network module is trained through a deep learning algorithm, and historical data of target position and light control data are used for model training during the training process, so that the neural network can quickly and accurately generate light directional control instructions when receiving new target position data.

8. The method of claim 5, wherein, The feedback step includes: Real-time detection of the error between the light illumination position and the target position, and generation of a feedback signal through an error calculation module for the neural network module to adjust network parameters and optimize subsequent light directional control.

Images