A desktop lighting robot control system

Abstract

The present application relates to a kind of desktop lighting robot control systems, comprising: integrated camera, voice module, information processing module, attitude sensing module, motion control module and spectrum detection and light source control module;User image is collected by camera, voice module captures user voice signal, information processing module solves user face position and gesture control instruction, attitude sensing module obtains user head posture information and visual point distance information;Motion control module generates control instruction according to the above information, drives robot to adjust position and adjusts light source;Spectrum detection and light source control module utilize deep learning technology to predict restore ambient spectrum information, and intelligently adjust light source, to adapt to different scene needs.The present application realizes intelligent following and dimming by posture solution and spectrum prediction technology, provides comfortable stable lighting environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robots, in particular to a tabletop lighting robot control system. BACKGROUND

[0002] In recent years, deep learning technology has made significant progress in object detection, mainly by simulating the learning mechanism of the human brain to learn features from massive data and use them for target detection. The R-CNN algorithm is the first algorithm in deep learning to be applied to object detection, which generates candidate regions through methods such as selective search, and uses convolutional neural networks and support vector machines for feature extraction and classification. Fast R-CNN, as an optimized version of R-CNN, avoids repeated feature calculation by extracting image feature maps once and extracting features for candidate regions, while improving speed and performance. Faster R-CNN further introduces a region proposal network that predicts target positions and bounding boxes by sliding windows on feature maps. The generated candidate regions are then used for classification and regression in Fast R-CNN. Thanks to this efficient candidate region generation and accurate target recognition, Faster R-CNN can divide object detection into two subtasks, enabling end-to-end training and inference, thus becoming a benchmark in the field and being widely used in security monitoring, autonomous driving, and other fields.

[0003] FOC technology, or Field-Oriented Control, is an advanced motor control technology. It uses a variable frequency drive (VFD) to precisely control the direction and magnitude of the magnetic field of a three-phase AC motor, achieving efficient and accurate torque and speed control. FOC technology converts the three-phase motor model into a two-phase model through Clarke and Park transformations, and then controls the d-axis and q-axis currents to orient the motor magnetic field. This technology makes the motor torque smooth, low noise, high efficiency, and has high dynamic response. FOC technology is widely used in industrial drives, household appliances, electric vehicles, wind power generation, etc., improving production efficiency and equipment performance. It is one of the important technologies in modern motor control field.

[0004] Pose solving technology is a technology that obtains the pose information of an object through sensors and processes and calculates it through algorithms. It is widely used in robots, drones, virtual reality, etc. Pose solving technology includes pose acquisition and pose processing. Pose acquisition obtains the pose information of an object through sensors, cameras, etc. such as accelerometers, gyroscopes, etc. Pose processing processes and calculates the obtained pose information through algorithms to determine the position, direction and pose of the object. This technology has wide application in various fields. In the field of robots, it can help robots to navigate and perform tasks autonomously. To improve accuracy, researchers have proposed various algorithms and models such as Kalman filtering, particle filtering, etc. to handle sensor noise and uncertainty. The continuous improvement of these algorithms and models makes the pose solving technology play an increasingly important role in various fields.

[0005] Digital spectrum sensors are commonly used to detect spectral information based on photoelectric effect and photodetection technology. It integrates multiple photodiodes inside, each of which is sensitive to different wavelengths of light, forming multiple channels. When light shines on the sensor, different wavelengths of light are detected by the photodiodes in each channel, producing a current proportional to the light intensity. These currents are converted into digital signals, i.e. spectral information, by the built-in analog-to-digital converter (ADC). By reading these digital signals, the intensity of different wavelengths of light can be obtained, and the spectral information of the object can be obtained. Digital spectrum sensors have high sensitivity, high resolution and high stability, etc. and are widely used in color detection, material analysis, environmental monitoring, etc. Through the analysis of spectral information, the color temperature, irradiance, etc. of the environmental spectrum can be understood, providing important information for scientific research, industry, medical, etc.

[0006] Traditional lighting methods often only focus on the brightness and color of light, ignoring the impact of light on human health. Modern lighting systems pay more attention to the quality, distribution and change of light, providing a comfortable and harmless lighting environment for people through scientific design and reasonable layout. This lighting method not only meets the visual needs of people, but also adjusts people's biological clock, improves mood, improves work efficiency, etc. In homes, schools, offices, etc. appropriate light sources can play an important role. By choosing the right lamps, adjusting light brightness and color temperature, etc. a healthy and comfortable lighting environment can be created, which is crucial for protecting our physical and mental health. SUMMARY

[0007] To solve the above problems, the present application provides a desktop lighting robot control system, which realizes intelligent following and dimming through pose solving and spectral prediction technology, provides a stable and comfortable lighting environment, and is suitable for office and education scenarios.

[0008] Specific schemes are as follows:

[0009] A desktop lighting robot control system comprises a camera, a voice module, an information processing module, a posture sensing module, a motion control module, a plurality of motors, a spectrum sensor, and a spectrum detection and light source control module.

[0010] The camera is configured to capture user images.

[0011] The voice module is configured to capture user voice signals.

[0012] The information processing module is configured to receive user images and user voice signals, calculate user facial positions and gesture control instructions based on the user images, and calculate user voices based on the user voice signals.

[0013] The posture sensing module is configured to capture posture signals and distance signals, and calculate user head posture information and visual point distance information based on the posture signals and the distance signals.

[0014] The motion control module is configured to receive and analyze user facial positions, gesture control instructions, user voices, user head posture information, and visual point distance information, and generate motor control instructions and spectrum detection and light source control module control instructions.

[0015] The plurality of motors are configured to receive and analyze the motor control instructions to drive the desktop lighting robot.

[0016] The spectrum sensor is configured to capture spectrum information.

[0017] The spectrum detection and light source control module is configured to receive and analyze the spectrum detection and light source control module control instructions, predict and restore the spectrum information through deep learning technology, obtain ambient light information, and control the light source of the desktop lighting robot based on the ambient light information.

[0018] Further, the desktop lighting robot control system further comprises a horizontal motion track, a power module, a horizontal motion motor, a camera holder, a light source holder, a horizontal motor synchronous wheel, a synchronous belt idler, an idler fixing plate, a main body trolley and a pulley, a sensor mounting bracket, an installation fixing buckle, a limit switch, and a cooling fan.

[0019] The power module comprises a boost, a buck, a power detection, and a heat dissipation control part.

[0020] The horizontal motion motor comprises a stepper motor, a synchronous wheel, and a stepper motor drive module with a closed-loop control function.

[0021] The camera holder and the light source holder have the same structure, comprising a yaw shaft brushless motor, a pitch shaft brushless motor, and a brushless motor FOC drive board.

[0022] The power module and the transverse movement motor are respectively fixed at two ends of the transverse movement track, and the transverse movement motor pulls the robot body to move through a synchronous belt;

[0023] The transverse motor synchronous wheel is fixed to the output shaft of the transverse movement motor;

[0024] The synchronous belt idler is fixed on the idler fixing plate through bolts;

[0025] The idler fixing plate is fixed on the transverse movement track close to the power module through bolts;

[0026] The robot body is connected with the track through the main body trolley and the pulley, the information processing module is located above the pulley plate, and the motion control module is fixed on the front side of the pulley plate through a tenon and screws;

[0027] The cooling fan is installed on the idler fixing plate through screws and blows towards the information processing module;

[0028] The installation fixing buckle is fixed on the transverse movement track through bolts;

[0029] The limit switch is installed on the installation fixing buckle on both sides;

[0030] The camera holder and the light source holder are connected to the installation support of the motion control module through the same support.

[0031] Further, the generation motor control instruction specifically includes:

[0032] The motion control module calculates the movement angle of the camera holder corresponding to two axes and the rotation angle of the transverse stepping motor according to the user head posture information and the coordinate information of the face in the user image, generates a first control instruction, and simultaneously fuses the visual point distance information, calculates the movement angle of the pitch axis brushless motor and the yaw axis brushless motor of the light source holder, generates a second control instruction, and transmits the first control instruction and the second control instruction to each motor through a control bus.

[0033] Further, the spectrum information is predicted and restored through a deep learning technology, specifically including:

[0034] S1, mixing the spectrum power distribution data containing red light, green light, blue light and white light, and taking the mixed sample data as a training data set;

[0035] S2, sampling the training data set, obtaining the spectrum power distribution data of the digital spectrum sensor on the specified channel as the input of the prediction and restoration model, and obtaining the spectrum power distribution data of the specified multiple of the specified channel number of the digital spectrum sensor as the output of the prediction and restoration model at equal intervals;

[0036] S3, normalize the spectral power distribution data used to predict the restoration model input and output, obtain the spectral power distribution data scaled to the 0 to 1 interval;

[0037] S4, input the spectral power distribution data scaled to the 0 to 1 interval into the prediction restoration model for training, obtain the trained prediction restoration model;

[0038] S5, input the test data into the trained prediction restoration model, obtain the prediction data, compare the prediction data with the true data to obtain the final calibration coefficient, and obtain the final prediction restoration model;

[0039] S6, deploy the final prediction restoration model in the spectral detection and light source control module.

[0040] Further, the normalization process is specifically as follows:

[0041]

[0042] Wherein, x represents the original data obtained by the digital spectrum sensor, x max And x min Respectively represent the maximum and minimum values in the prediction data.

[0043] Further, the prediction restoration model is trained, specifically including:

[0044] The prediction restoration model is trained using MLP, RBF, LSTM or 1D-CNN algorithm, and the spectral power distribution data is reconstructed.

[0045] The present application adopts the above technical scheme and has beneficial effects:

[0046] (1) The present application accurately obtains the position information of the user through the wireless attitude sensing module and the information processing module, realizes intelligent following, and ensures that the lighting is always focused on the position of the user;

[0047] (2) The present application uses spectral data prediction restoration technology to accurately predict and restore ambient light information, ensures the stability and comfort of the lighting environment, reduces light pollution, and protects the user's vision;

[0048] (3) The present application automatically adjusts the light source brightness according to the ambient light information, provides suitable lighting conditions, improves the user experience, especially in office and education scenarios, and ensures that the user works and learns under the best light. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 The attitude sensing module of the embodiment of the present application is shown in the figure;

[0050] Figure 2A schematic diagram of a desktop lighting robot structure according to an embodiment of the present application;

[0051] Figure 3 A framework diagram of interaction of each module according to an embodiment of the present application;

[0052] Figure 4 A schematic diagram of mixing of 60% white light SPD and 40% red light SPD according to an embodiment of the present application;

[0053] Figure 5 An example diagram of a neural network model according to an embodiment of the present application;

[0054] Figure 6 A prediction effect diagram of different neural network models according to an embodiment of the present application;

[0055] Figure 7 A comparison diagram of real data, predicted data and calibrated data according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The present application will be further described in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present application are not limited thereto.

[0057] As shown in Figure 1 , Figure 2 and Figure 3 , a desktop lighting robot control system according to the present application comprises:

[0058] comprises a camera, a voice module, an information processing module, a posture sensing module, a motion control module, a plurality of motors, a spectrum sensor and a spectrum detection and light source control module;

[0059] The camera is configured to capture user images;

[0060] The voice module is configured to capture user voice signals;

[0061] The information processing module is configured to receive user images and user voice signals, calculate user face positions and gesture control instructions based on the user images, and calculate user voices based on the user voice signals;

[0062] The posture sensing module is configured to capture posture signals and distance signals, and calculate user head posture information and visual point distance information based on the posture signals and the distance signals;

[0063] The motion control module is configured to receive and analyze user face positions, gesture control instructions, user voices, user head posture information and visual point distance information, generate motor control instructions and spectrum detection and light source control module control instructions, and drive the desktop lighting robot by receiving and analyzing the motor control instructions by each motor;

[0064] The plurality of motors are configured to receive and analyze motor control instructions to drive the tabletop lighting robot.

[0065] The spectrum sensor is configured to collect spectrum information.

[0066] The spectrum detection and light source control module is configured to receive and analyze spectrum detection and light source control module control instructions, predict and restore the spectrum information by using a deep learning technology, obtain ambient light information, and control the light source of the tabletop lighting robot based on the ambient light information.

[0067] Specifically, the tabletop lighting robot further comprises:

[0068] The lateral movement track (18), the power module (15), the lateral movement motor (17), the camera holder (6), (7), the light source holder (8), (9), the image and attitude acquisition module (10), the audio acquisition module (2), the motion control module (3), the lateral motor synchronous wheel (16), the synchronous belt idler (19), the idler fixing plate (20), the main body trolley (4) and the pulley (5), the sensor mounting bracket (22), the mounting and fixing buckle (13), (14), the limit switch (11), (12) and the cooling fan (21);

[0069] The power module comprises a boost, a buck, a power detection and a heat dissipation control part;

[0070] The lateral movement motor comprises a stepper motor, a synchronous wheel and a stepper motor driving module with a closed-loop control function;

[0071] The camera holder and the light source holder have the same structure and comprise a yaw shaft brushless motor, a pitch shaft brushless motor and a brushless motor FOC driving board;

[0072] The power module and the lateral movement motor are respectively fixed at two ends of the lateral movement track, and the lateral movement motor drives the robot body to move through a synchronous belt;

[0073] The lateral motor synchronous wheel is fixed to the output shaft of the lateral movement motor;

[0074] The synchronous belt idler is fixed to the idler fixing plate by bolts;

[0075] The idler fixing plate is fixed to the lateral movement track near the power module by bolts;

[0076] The robot body is connected to the track through the main body trolley and the pulley, the information processing module is located above the pulley plate, and the motion control module is fixed to the front side of the pulley plate by a tenon and screws;

[0077] The cooling fan is installed on the idler fixing plate by screws and blows towards the information processing module.

[0078] The installation fixing buckle is fixed on the transverse movement track by bolts;

[0079] The limit switch is installed on the two installation fixing buckles;

[0080] The camera holder and the light source holder are connected to the installation support of the movement control module by the same support;

[0081] The movement control module comprises a main controller and a wireless transmission module, and the movement control module analyzes the position information and control instructions obtained by the attitude solution technology to generate motor control instructions, which are transmitted to the motors by the main controller and the wireless transmission module.

[0082] Specifically, the motor control instructions are generated, comprising:

[0083] The movement control module calculates the movement angles of the camera holder corresponding to two axes and the rotation angle of the transverse stepping motor according to the user head posture information and the coordinate information of the face in the user image to generate a first control instruction; meanwhile, the movement angles of the pitch axis brushless motor and the yaw axis brushless motor of the light source holder are calculated by fusing the visual point distance information to generate a second control instruction; the first control instruction and the second control instruction are transmitted to the holder motor 1, the holder motor 2, the holder motor 3, the holder motor 4, the transverse motor and the light source control module (spectrum detection and light source control module) through the control bus.

[0084] Specifically, the way to realize the desktop lighting robot which can intelligently follow and intelligently adjust the light is as follows:

[0085] Firstly, the user wears the wearable posture sensing module on the head; then, the camera captures the image, the voice module captures the user voice signal, and the image signal and the voice signal are transmitted to the information processing module to solve the user face position information and the user voice and gesture control instructions; next, the posture sensing module collects the posture signal and the distance signal to solve the user head posture information and the visual point distance information; then, the information processing module transmits the user face position information and the user voice and gesture control instructions to the movement control module through the communication bus, and the posture sensing module transmits the user head posture information and the visual point distance information to the movement control module through the wireless communication module; on this basis, the movement control module analyzes the above information and instructions to generate motor control instructions which are transmitted to the motors for execution through the control bus; at the same time, the spectrum detection and light source control module collects the spectrum signal by the digital spectrum sensor, restores the spectrum information by the deep learning technology, accurately calculates the ambient light information, and controls the light source to ensure that the light supplement process is gentle and non-sensing, and the lighting environment is stable.

[0086] Specifically, the user will move during the use of the table lighting robot. In order to realize the tracking function and the intelligent light compensation function, the tracking function of the camera holder and the light source holder is realized through the motion control of the motor. The motion control module calculates the motion angle of the two axes corresponding to the camera holder and the rotation angle of the horizontal stepping motor according to the head posture information (i.e. the angle information of the two axes of yaw and pitch) of the user and the coordinate information of the face in the collected image, and generates a control instruction. At the same time, the motion angle of the pitch motor and the yaw motor of the light source holder is calculated by fusing the aforementioned visual point distance information, and a control instruction is also generated. After the instruction is generated, it is transmitted to each motor for execution through the control bus.

[0087] Specifically, the spectral data prediction and restoration technology is used to predict and restore the collected spectral signals, specifically including:

[0088] S1, the spectral power distribution data containing red light, green light, blue light and white light is mixed, and the sample data obtained by mixing is used as a training data set;

[0089] S2, the training data set is sampled to obtain the spectral power distribution data of the digital spectral sensor on the specified channel as the input of the prediction and restoration model, and the spectral power distribution data of the specified multiple of the specified channel number of the digital spectral sensor is obtained as the output of the prediction and restoration model at equal intervals;

[0090] S3, the spectral power distribution data used for the input and output of the prediction and restoration model is normalized to obtain the spectral power distribution data scaled to the interval of 0 to 1;

[0091] S4, the spectral power distribution data scaled to the interval of 0 to 1 is input into the prediction and restoration model for training to obtain the trained prediction and restoration model;

[0092] S5, the test data is input into the trained prediction and restoration model to obtain prediction data, and the prediction data is compared with the true data to obtain the final calibration coefficient to obtain the final prediction and restoration model;

[0093] S6, the final prediction and restoration model is deployed in the spectral detection and light source control module.

[0094] Specifically, the normalization processing is as follows:

[0095]

[0096] wherein, x represents the original data obtained by the digital spectrum sensor, x max and x min respectively represent the maximum value and the minimum value in the predicted data.

[0097] Specifically, the prediction restoration model is trained, specifically including:

[0098] The prediction restoration model is trained using an MLP, RBF, LSTM or 1D-CNN algorithm, and the spectral power distribution data is reconstructed.

[0099] Specifically, the final prediction restoration model is run on the spectral detection and light source control module or other embedded devices, and the implementation is as follows:

[0100] First, the SPD of red light, green light, blue light and white light under room temperature conditions is measured using a spectrometer as initial data.

[0101] Next, any two or three different colors of light are combined in percentage increments of 2%, for example, 2% white light combined with 98% red light, 4% white light combined with 96% red light, 2% white light combined with 2% red light combined with 96% green light, 4% white light combined with 2% red light combined with 94% green light, and so on, Figure 4 for the illustration of 60% white light combined with 40% red light; finally, a total of 5006 sample training data sets are obtained.

[0102] Next, each sample is sampled to obtain the SPD at 8 wavelengths corresponding to the spectral sensor response as the input of the Jupiter training data; then, 81 points of SPD data are obtained at intervals of 5 nm as the output of the model training; on this basis, the obtained SPD is normalized, and the present application uses linear normalization, which can scale the data to between 0 and 1.

[0103] After that, the MLP, RBF, LSTM, 1D-CNN and other algorithms are used for model training and SPD reconstruction work, and the neural network model is shown as Figure 5 According to the reconstruction effects of the models shown in Figure 6 , the most suitable neural network is selected; wherein the input of the neural network model is the value of the corresponding spectral sensor visible light range channel wavelength extracted from the real measured SPD, Figure 6 (a) indicates that the SPD is reconstructed using MLP; (b) indicates that the SPD is reconstructed using RBF; (c) indicates that the SPD is reconstructed using LSTM; and (d) indicates that the SPD is reconstructed using 1D-CNN.

[0104] Finally, the original data obtained by the spectral sensor is used for prediction, the predicted 81 data is compared with the true data, the ratio is calculated as a calibration coefficient, the average value of the calibration coefficients calculated by 50 groups of data measured in different environments is taken as the final calibration coefficient; the comparison of the true data, the predicted data and the calibrated predicted data is as shown in Figure 7

[0105] Although the present application is specifically shown and described with reference to preferred embodiments, it is pointed out that various modifications can be made in form and detail without departing from the spirit and scope of the application as defined by the appended claims.​

Claims

1. A desktop lighting robot control system, characterized in that, include: Camera, voice module, information processing module, posture sensing module, motion control module, several motors, spectral sensor and spectral detection and light source control module; The camera is used to capture user images; The voice module is used to capture the user's voice signal; The information processing module is used to receive user images and user voice signals, calculate the user's face position and gesture control commands based on the user images, and calculate the user's voice based on the user voice signals. The posture sensing module is used to collect posture signals and distance signals, and calculate user head posture information and visual point distance information based on the posture signals and distance signals. The motion control module is used to receive and parse the user's face position, gesture control commands, user voice, user head posture information and visual point distance information, and generate motor control commands and spectral detection and light source control module control commands. The aforementioned motors are used to receive and parse motor control commands to drive the desktop lighting robot; The spectral sensor is used to collect spectral information; The spectral detection and light source control module is used to receive and parse the control commands from the spectral detection and light source control module, predict and reconstruct the spectral information through deep learning technology, obtain ambient light information, and control the light source of the desktop lighting robot based on the ambient light information.

2. The desktop lighting robot control system according to claim 1, characterized in that, Also includes: Lateral motion track, power module, lateral motion motor, camera gimbal, light source gimbal, lateral motor synchronous pulley, synchronous belt idler pulley and idler pulley fixing plate, main trolley and pulley, sensor mounting bracket, mounting and fixing buckle, limit switch and cooling fan; The power module includes boost, buck, power detection and heat dissipation control sections; The lateral motion motor includes a stepper motor, a synchronous pulley, and a stepper motor drive module with closed-loop control function; The camera gimbal and the light source gimbal have the same structure, including a yaw axis brushless motor, a pitch axis brushless motor and a brushless motor FOC drive board. The power module and the lateral motion motor are respectively fixed at both ends of the lateral motion track. The lateral motion motor pulls the robot body to move via a synchronous belt. The lateral motor synchronous pulley is fixed to the output shaft of the lateral motion motor; The timing belt idler pulley is fixed to the idler pulley fixing plate by bolts; The idler wheel fixing plate is fixed to the end of the transverse motion track near the power module by bolts. The robot body is connected to the track via a main trolley and pulleys. The information processing module is located above the pulley plate, and the motion control module is fixed to the front side of the pulley plate by latches and screws. The cooling fan is mounted on the idler wheel fixing plate by screws and blows air toward the information processing module; The mounting and fixing clips are fixed to the transverse movement track by bolts; The limit switches are mounted on the mounting and fixing buckles on both sides; The camera gimbal and the light source gimbal are connected to the mounting bracket of the motion control module via the same bracket.

3. The desktop lighting robot control system according to claim 2, characterized in that, The generated motor control commands specifically include: The motion control module calculates the motion angles of the camera gimbal on the corresponding two axes and the rotation angle of the lateral stepper motor based on the user's head posture information and the coordinate information of the face in the user's image, and generates the first control command; at the same time, it integrates the visual point distance information to calculate the motion angles of the light source gimbal's pitch axis brushless motor and yaw axis brushless motor, and generates the second control command; the first control command and the second control command are transmitted to each motor through the control bus.

4. The desktop lighting robot control system according to claim 1, characterized in that, The method of predicting and restoring spectral information using deep learning technology specifically includes: S1, mix the spectral power distribution data containing red, green, blue and white light, and use the mixed sample data as the training dataset; S2, sample the training dataset to obtain the spectral power distribution data of the digital spectral sensor on the specified channel, and use it as the input of the prediction and reconstruction model. At equal intervals, obtain the spectral power distribution data of the specified number of channels of the digital spectral sensor at a specified multiple, and use it as the output of the prediction and reconstruction model. S3, normalize the spectral power distribution data used for predicting the input and output of the reconstruction model to obtain spectral power distribution data scaled to the 0 to 1 interval; S4. Input the spectral power distribution data scaled to the 0 to 1 interval into the prediction and reconstruction model for training to obtain the trained prediction and reconstruction model. S5. Input the test data into the trained prediction and restoration model to obtain the prediction data. Compare the prediction data with the real data to obtain the final calibration coefficients and obtain the final prediction and restoration model. S6 deploys the final prediction and reconstruction model in the spectral detection and light source control module.

5. The desktop lighting robot control system according to claim 4, characterized in that, The normalization process is specifically formulated as follows: Where x represents the raw data obtained by the digital spectral sensor, x max and x min These represent the maximum and minimum values ​​in the predicted data, respectively.

6. The desktop lighting robot control system according to claim 4, characterized in that, Training the prediction and reconstruction model specifically includes: The predictive reconstruction model and the spectral power distribution data are reconstructed using MLP, RBF, LSTM or 1D-CNN algorithms.

Images