A lighting monitoring method and monitoring system based on machine learning

Abstract

The present invention proposes a light monitoring method based on machine learning, comprising: obtaining RGB data, IR grayscale data and Depth data of an image in a sample image set, and obtaining the luminous flux and color temperature of a lamp in the corresponding image; constructing a feature sequence according to the RGB data, IR grayscale data and Depth data of the image, and taking the luminous flux and color temperature of the lamp in the corresponding image as a first label and a second label; inputting the feature sequence and the first label and the second label into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model; fusing a model of the RGB data, IR grayscale data and Depth data of the extracted image with the trained many-to-many network model to obtain a light monitoring model; inputting an image to be monitored into the light monitoring model to obtain the luminous flux and color temperature of the lamp in the image; the method proposed by the present invention realizes automatic light monitoring of lamps, and the test process is fast, the result is accurate and the cost is low.

Description

Technical Field

[0001] The present invention relates to the field of lamp testing, and in particular to a lighting monitoring method and monitoring system based on machine learning. Background Art

[0002] When developing and testing lamps, it is necessary to identify the brightness and color temperature of the lamps. Traditionally, the brightness of lamps is usually identified by obtaining the brightness of the lamps through photosensors, while the color temperature of the lamps is mainly related to the current ratio parameters passing through the lamps. Therefore, the identification of the color temperature of the lamps requires first obtaining the current ratio parameters passing through the lamps, and then finding the corresponding color temperature based on the corresponding relationship between the preset current ratio parameters and the color temperature. The testing process is cumbersome and the testing cost is high. Summary of the invention

[0003] The main purpose of the present invention is to overcome the above-mentioned defects in the prior art and propose a lighting monitoring method based on machine learning, which combines image recognition and machine learning to realize automatic lighting monitoring of lamps in the research and development process and the production process without human intervention, and the testing process is fast, the results are accurate, and the cost is low.

[0004] The present invention adopts the following technical solution:

[0005] A light monitoring method based on machine learning, comprising:

[0006] Obtain the RGB data, IR grayscale data, and Depth data of the images in the sample image set, and obtain the luminous flux and color temperature of the lamps in the corresponding images;

[0007] According to the RGB data, IR grayscale data and Depth data of the image, a feature sequence is constructed, and the luminous flux and color temperature of the lamp in the corresponding image are used as the first label and the second label;

[0008] Inputting the feature sequence, the first label, and the second label into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model;

[0009] The models of the extracted image's RGB data, IR grayscale data, and Depth data are integrated with the trained many-to-many network model to obtain a light monitoring model.

[0010] The image to be monitored is input into the lighting monitoring model to obtain the luminous flux and color temperature of the lamps in the image.

[0011] Specifically, according to the RGB data, IR grayscale data and Depth data of the image, a feature sequence is constructed, which is as follows:

[0012] The average R value of pixels in the image is taken as the first eigenvalue;

[0013] The average G value of pixels in the image is taken as the second eigenvalue;

[0014] The average value of the B values ​​of the pixels in the image is taken as the third eigenvalue;

[0015] The average IR value of pixels in the image is taken as the fourth eigenvalue;

[0016] The average value of the depth value of the pixels in the image is taken as the fifth eigenvalue;

[0017] The first eigenvalue, the second eigenvalue, the third eigenvalue, the fourth eigenvalue and the fifth eigenvalue are constructed as a characteristic sequence.

[0018] Specifically, the many-to-many network model includes but is not limited to: a many-to-many recurrent neural network model and an LSTM model.

[0019] Specifically, it also includes:

[0020] The RGB data, IR grayscale data and Depth data of the images in the sample image set are obtained, and the lamp area is determined according to the RGB data, IR grayscale data and Depth data of the images in the sample image set, and a feature sequence is constructed according to the RGB data, IR grayscale data and Depth data in the lamp area.

[0021] Another aspect of the present invention provides a lighting monitoring system based on machine learning, comprising:

[0022] Sample data acquisition unit: obtains RGB data, IR grayscale data and Depth data of the image in the sample image set, and obtains the luminous flux and color temperature of the lamp in the corresponding image;

[0023] Feature sequence and label acquisition unit: constructs a feature sequence according to the RGB data, IR grayscale data and Depth data of the image, and uses the luminous flux and color temperature of the lamp in the corresponding image as the first label and the second label;

[0024] Many-to-many network model training unit: inputting the feature sequence and the first label and the second label into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model;

[0025] Light monitoring model acquisition unit: The model of RGB data, IR grayscale data and Depth data of the extracted image is integrated with the trained many-to-many network model to obtain the light monitoring model;

[0026] Lamp monitoring unit: Input the image to be monitored into the light monitoring model to obtain the luminous flux and color temperature of the lamp in the image.

[0027] Specifically, according to the RGB data, IR grayscale data and Depth data of the image, a feature sequence is constructed, which is as follows:

[0028] The average R value of pixels in the image is taken as the first eigenvalue;

[0029] The average G value of pixels in the image is taken as the second eigenvalue;

[0030] The average value of the B values ​​of the pixels in the image is taken as the third eigenvalue;

[0031] The average IR value of pixels in the image is taken as the fourth eigenvalue;

[0032] The average value of the depth value of the pixels in the image is taken as the fifth eigenvalue;

[0033] The first eigenvalue, the second eigenvalue, the third eigenvalue, the fourth eigenvalue and the fifth eigenvalue are constructed as a characteristic sequence.

[0034] Specifically, the many-to-many network model includes but is not limited to: a many-to-many recurrent neural network model and an LSTM model.

[0035] Specifically, it also includes:

[0036] The RGB data, IR grayscale data and Depth data of the images in the sample image set are obtained, and the lamp area is determined according to the RGB data, IR grayscale data and Depth data of the images in the sample image set, and a feature sequence is constructed according to the RGB data, IR grayscale data and Depth data in the lamp area.

[0037] According to another aspect of an embodiment of the present invention, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method steps for light monitoring based on machine learning when executing the computer program.

[0038] Yet another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned method steps for light monitoring based on machine learning.

[0039] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention proposes a light monitoring method based on machine learning. First, the RGB data, IR grayscale data and Depth data of an image in a sample image set are obtained, and the luminous flux and color temperature of the lamp in the corresponding image are obtained; a feature sequence is constructed according to the RGB data, IR grayscale data and Depth data of the image, and the luminous flux and color temperature of the lamp in the corresponding image are used as the first label and the second label; the feature sequence and the first label and the second label are input into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model; the model of the RGB data, IR grayscale data and Depth data of the extracted image is integrated with the trained many-to-many network model to obtain a light monitoring model; the image to be monitored is input into the light monitoring model to obtain the luminous flux and color temperature of the lamp in the image; the method proposed by the present invention combines image recognition and machine learning to realize automatic light monitoring of lamps in the research and development process and the production process without manual intervention, and the test process is fast, the results are accurate and the cost is low.

[0041] (2) The present invention uses the RGB data, IR grayscale data and Depth data of the image as input feature values, which comprehensively characterizes the properties of the image, making the training model accurate and thus the test results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of a lighting monitoring method based on machine learning provided by an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of different RNN structures provided by embodiments of the present invention;

[0044] Figure 3 A schematic diagram of lamp identification provided by an embodiment of the present invention;

[0045] Figure 4 A system structure diagram of a lighting monitoring system based on machine learning provided by an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention;

[0047] Figure 6 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention.

[0048] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0049] The present invention proposes a lighting monitoring method based on machine learning, which combines image recognition and machine learning to realize automatic lighting monitoring of lamps in the research and development process and the production process without manual intervention. The test process is fast, the results are accurate, and the cost is low.

[0050] like Figure 1 , is a flow chart of a lighting monitoring method based on machine learning provided by an embodiment of the present invention, specifically:

[0051] A light monitoring method based on machine learning, comprising:

[0052] S101: Obtain RGB data, IR grayscale data, and Depth data of an image in a sample image set, and obtain the luminous flux and color temperature of a lamp in the corresponding image;

[0053] First, obtain a sample image set containing lamps, and then obtain the RGB data, IR grayscale data, and Depth data of the images in the image set;

[0054] It is worth mentioning that 3D camera products currently include binocular structured light (RGB+IR) solutions and TOF (single IR camera) solutions. The embodiment of the present invention can adopt the structure of the TOF solution plus an RGB camera.

[0055] Specifically, a frame synchronization signal is added to the color RGB and infrared IR, and the RGB data and IR grayscale data are collected synchronously. The depth data is generated by the 3D depth generation algorithm processing the frame data. Specifically, the depth data can be generated based on the RGB data and the IR grayscale data, or based on the IR grayscale data. The TOF solution only needs the IR grayscale data to generate the depth data.

[0056] S102: construct a feature sequence according to the RGB data, IR grayscale data and Depth data of the image, and use the luminous flux and color temperature of the lamp in the corresponding image as the first label and the second label;

[0057] In fact, this step also includes obtaining the RGB data, IR grayscale data and Depth data of the images in the sample image set, and determining the lamp area based on the RGB data, IR grayscale data and Depth data of the images in the sample image set, and constructing a feature sequence based on the RGB data, IR grayscale data and Depth data in the lamp area.

[0058] The lamp area is determined based on the RGB data, IR grayscale data and Depth data of the images in the sample image set. Specifically, the change rate of each data is slower in the lamp area, and the change rate of each data will drop suddenly when exceeding the lamp area. The lamp area is determined based on the sudden drop in the data change rate.

[0059] The average R value of pixels in the image lamp area is used as the first eigenvalue;

[0060] The average value of the G value of the pixels in the image lamp area is used as the second eigenvalue;

[0061] The average value of the B values ​​of the pixels in the lamp area of ​​the image is taken as the third eigenvalue;

[0062] The average IR value of the pixels in the image lamp area is taken as the fourth eigenvalue;

[0063] The average value of the depth value of the pixels in the lamp area of ​​the image is taken as the fifth eigenvalue;

[0064] The first eigenvalue, the second eigenvalue, the third eigenvalue, the fourth eigenvalue and the fifth eigenvalue are constructed as a characteristic sequence.

[0065] The brightness of a lamp is characterized by luminous flux, the unit is lumen, or lm. The amount of light emitted by a light source per unit time is called the luminous flux of the light source. Similarly, this quantity is for the light source, describing the total amount of light emitted by the light source, which is equivalent to the optical power. The larger the luminous flux of a light source, the more light it emits.

[0066] Color temperature is an important indicator for considering lamps. Color temperature is a unit of measurement that indicates the color components contained in light. Theoretically, blackbody temperature refers to the color of an absolute blackbody after heating from absolute zero (-273℃). After being heated, the blackbody gradually changes from black to red, yellow, white, and finally emits blue light. When heated to a certain temperature, the spectral components contained in the light emitted by the blackbody are called the color temperature at this temperature, and the unit of measurement is "K".

[0067] S103: inputting the feature sequence, the first label, and the second label into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model;

[0068] The many-to-many network model includes but is not limited to: a many-to-many recurrent neural network model and an LSTM model.

[0069] The embodiment of the present invention adopts a many-to-many recurrent neural network model:

[0070] Recurrent Neural Network (RNN) adds horizontal connections between hidden layer units based on common multi-layer back propagation (BP) neural network. Through a weight matrix, the value of the neural unit in the previous time series can be transferred to the current neural unit, so that the neural network has a memory function. It has good applicability for natural language processing or time series machine learning problems with contextual connections. For a standard RNN structure, the main structure input of the RNN at time t comes from the input layer Xt, and there is also a loop edge to provide the hidden state transmitted from time t-1.

[0071] The applicability of RNN models varies depending on the number of output and input sequences, such as Figure 2 As shown in the figure, RNN can have many different structures, and the five structures are: one-to-one, one-to-many, many-to-one, interval many-to-many, and synchronous many-to-many. Different structures naturally have different application scenarios. These five RNN model structures can correspond to Vanilla neural network, picture title generation, sentiment analysis, machine translation, and context prediction application scenarios respectively.

[0072] The data input of the present invention is a feature sequence, and the required output is the luminous flux and color temperature of the lamp. Its natural sequentiality conforms to the two structures of the RNN model: interval many-to-many and synchronous many-to-many. The biggest difference between the two is that in the interval many-to-many mode, the model cannot utilize the correlation between features in the input feature sequence, while the synchronous many-to-many mode can. Therefore, the modeling of the embodiment of the present invention selects an RNN model with a synchronous many-to-many structure, namely a synchronous many-to-many recurrent neural network model, as the basic network structure.

[0073] S104: fusing the models of the extracted image's RGB data, IR grayscale data, and Depth data with the trained many-to-many network model to obtain a light monitoring model;

[0074] Combine the conventional algorithm for extracting RGB data, IR grayscale data, and Depth data of images with the trained many-to-many network model to obtain a light monitoring model.

[0075] In this embodiment, the RGB data and IR grayscale data of the image can be directly collected by the camera, and the Depth data is generated by processing the RGB data and the IR grayscale data through a conventional 3D depth generation algorithm. Therefore, it is only necessary to combine the conventional 3D depth generation algorithm with a trained many-to-many recurrent neural network model to obtain a light monitoring model.

[0076] S105: Input the image to be monitored into the light monitoring model to obtain the luminous flux and color temperature of the lamps in the image.

[0077] like Figure 3 , which is a schematic diagram of the identification of lamps during the test.

[0078] like Figure 4 Another aspect of the present invention provides a lighting monitoring system based on machine learning, including:

[0079] The sample data acquisition unit 401 acquires the RGB data, IR grayscale data and Depth data of the image in the sample image set, and acquires the luminous flux and color temperature of the lamp in the corresponding image;

[0080] First, obtain a sample image set containing lamps, and then obtain the RGB data, IR grayscale data, and Depth data of the images in the image set;

[0081] It is worth mentioning that 3D camera products currently include binocular structured light (RGB+IR) solutions and TOF (single IR camera) solutions. The embodiment of the present invention can adopt the structure of the TOF solution plus an RGB camera.

[0082] Specifically, a frame synchronization signal is added to the color RGB and infrared IR, and the RGB data and IR grayscale data are collected synchronously. The depth data is generated by the 3D depth generation algorithm processing the frame data. Specifically, the depth data can be generated based on the RGB data and the IR grayscale data, or based on the IR grayscale data. The TOF solution only needs the IR grayscale data to generate the depth data.

[0083] Feature sequence and label acquisition unit 402: constructs a feature sequence according to the RGB data, IR grayscale data and Depth data of the image, and uses the luminous flux and color temperature of the lamp in the corresponding image as the first label and the second label;

[0084] In fact, this step also includes obtaining the RGB data, IR grayscale data and Depth data of the images in the sample image set, and determining the lamp area based on the RGB data, IR grayscale data and Depth data of the images in the sample image set, and constructing a feature sequence based on the RGB data, IR grayscale data and Depth data in the lamp area.

[0085] The lamp area is determined based on the RGB data, IR grayscale data and Depth data of the images in the sample image set. Specifically, the change rate of each data is slower in the lamp area, and the change rate of each data will drop suddenly when exceeding the lamp area. The lamp area is determined based on the sudden drop in the data change rate.

[0086] The average R value of pixels in the image lamp area is used as the first eigenvalue;

[0087] The average value of the G value of the pixels in the image lamp area is used as the second eigenvalue;

[0088] The average value of the B values ​​of the pixels in the lamp area of ​​the image is taken as the third eigenvalue;

[0089] The average IR value of the pixels in the image lamp area is taken as the fourth eigenvalue;

[0090] The average value of the depth value of the pixels in the lamp area of ​​the image is taken as the fifth eigenvalue;

[0091] The first eigenvalue, the second eigenvalue, the third eigenvalue, the fourth eigenvalue and the fifth eigenvalue are constructed as a characteristic sequence.

[0092] The brightness of a lamp is characterized by luminous flux, the unit is lumen, or lm. The amount of light emitted by a light source per unit time is called the luminous flux of the light source. Similarly, this quantity is for the light source, describing the total amount of light emitted by the light source, which is equivalent to the optical power. The larger the luminous flux of a light source, the more light it emits.

[0093] Color temperature is an important indicator for considering lamps. Color temperature is a unit of measurement that indicates the color components contained in light. Theoretically, blackbody temperature refers to the color of an absolute blackbody after heating from absolute zero (-273℃). After being heated, the blackbody gradually changes from black to red, yellow, white, and finally emits blue light. When heated to a certain temperature, the spectral components contained in the light emitted by the blackbody are called the color temperature at this temperature, and the unit of measurement is "K".

[0094] Many-to-many network model training unit 403: inputting the feature sequence, the first label, and the second label into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model;

[0095] The many-to-many network model includes but is not limited to: a many-to-many recurrent neural network model and an LSTM model.

[0096] The embodiment of the present invention adopts a many-to-many recurrent neural network model:

[0097] Recurrent Neural Network (RNN) adds horizontal connections between hidden layer units based on common multi-layer back propagation (BP) neural network. Through a weight matrix, the value of the neural unit in the previous time series can be transferred to the current neural unit, so that the neural network has a memory function. It has good applicability for natural language processing or time series machine learning problems with contextual connections. For a standard RNN structure, the main structure input of the RNN at time t comes from the input layer Xt, and there is also a loop edge to provide the hidden state transmitted from time t-1.

[0098] The applicability of RNN models varies depending on the number of output and input sequences, such as Figure 2 As shown in the figure, RNN can have many different structures, and the five structures are: one-to-one, one-to-many, many-to-one, interval many-to-many, and synchronous many-to-many. Different structures naturally have different application scenarios. These five RNN model structures can correspond to Vanilla neural network, picture title generation, sentiment analysis, machine translation, and context prediction application scenarios respectively.

[0099] The data input of the present invention is a feature sequence, and the required output is the luminous flux and color temperature of the lamp. Its natural sequentiality conforms to the two structures of the RNN model: interval many-to-many and synchronous many-to-many. The biggest difference between the two is that in the interval many-to-many mode, the model cannot utilize the correlation between features in the input feature sequence, while the synchronous many-to-many mode can. Therefore, the modeling of the embodiment of the present invention selects an RNN model with a synchronous many-to-many structure, namely a synchronous many-to-many recurrent neural network model, as the basic network structure.

[0100] Light monitoring model acquisition unit 404: fusing the models of RGB data, IR grayscale data and Depth data of the extracted image with the trained many-to-many network model to obtain a light monitoring model;

[0101] Combine the conventional algorithm for extracting RGB data, IR grayscale data, and Depth data of images with the trained many-to-many network model to obtain a light monitoring model.

[0102] In this embodiment, the RGB data and IR grayscale data of the image can be directly collected by the camera, and the Depth data is generated by processing the RGB data and the IR grayscale data through the conventional 3D depth generation algorithm. Therefore, it is only necessary to combine the conventional 3D depth generation algorithm with the trained many-to-many recurrent neural network model to obtain the light monitoring model.

[0103] The lamp monitoring unit 405 inputs the image to be monitored into the light monitoring model to obtain the luminous flux and color temperature of the lamp in the image.

[0104] Deploy the intelligent terminal equipped with the method of the present invention to the site to be monitored, install the target equipment to be monitored, start the test, and wait for the output result.

[0105] like Figure 5 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 520 and executable on the processor 520. When the processor 520 executes the computer program 511, a light monitoring method based on machine learning provided in an embodiment of the present invention is implemented.

[0106] In a specific implementation process, when the processor 520 executes the computer program 511, it can achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0107] Since the electronic device introduced in this embodiment is a device used to implement a data processing device in the embodiment of the present invention, based on the method introduced in the embodiment of the present invention, technical personnel in this field can understand the specific implementation method of the electronic device of this embodiment and its various variations. Therefore, how the electronic device implements the method in the embodiment of the present invention is not introduced in detail here. As long as the equipment used by technical personnel in this field to implement the method in the embodiment of the present invention falls within the scope of protection of the present invention.

[0108] See also Figure 6 , Figure 6 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention.

[0109] like Figure 6 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, a light monitoring method based on machine learning provided by an embodiment of the present invention is implemented;

[0110] In the specific implementation process, when the computer program 611 is executed by the processor, it can achieve Figure 1 Any implementation manner in the corresponding embodiments.

[0111] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0112] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] The present invention proposes a light monitoring method based on machine learning. First, RGB data, IR grayscale data and Depth data of an image in a sample image set are obtained, and the luminous flux and color temperature of the lamp in the corresponding image are obtained; a feature sequence is constructed according to the RGB data, IR grayscale data and Depth data of the image, and the luminous flux and color temperature of the lamp in the corresponding image are used as the first label and the second label; the feature sequence and the first label and the second label are input into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model; a model of the RGB data, IR grayscale data and Depth data of the extracted image is fused with the trained many-to-many network model to obtain a light monitoring model; the image to be monitored is input into the light monitoring model to obtain the luminous flux and color temperature of the lamp in the image; the method proposed by the present invention combines image recognition and machine learning to realize automatic light monitoring of lamps in the research and development process and the production process without manual intervention, and the test process is fast, the results are accurate and the cost is low.

[0114] The present invention uses the RGB data, IR grayscale data and Depth data of the image as input feature values, comprehensively characterizing the attributes of the image, making the training model accurate, and thus the test results more accurate.

[0115] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. The various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied for herein.

[0116] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A lighting monitoring method based on machine learning, characterized in that: include: Obtain the RGB data, IR grayscale data, and Depth data of the images in the sample image set, and obtain the luminous flux and color temperature of the lamps in the corresponding images; According to the RGB data, IR grayscale data and Depth data of the image, a feature sequence is constructed, and the luminous flux and color temperature of the lamp in the corresponding image are used as the first label and the second label; Inputting the feature sequence, the first label, and the second label into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model; The models of the extracted image's RGB data, IR grayscale data, and Depth data are integrated with the trained many-to-many network model to obtain a light monitoring model. The image to be monitored is input into the lighting monitoring model to obtain the luminous flux and color temperature of the lamps in the image; Also includes: Determine the lamp area according to the RGB data, IR grayscale data and Depth data of the image in the sample image set, and construct a feature sequence according to the RGB data, IR grayscale data and Depth data in the lamp area; The lamp area is determined specifically based on the change rate of the RGB data, IR grayscale data and Depth data of the image, and the lamp area is determined based on a sudden drop in the change rate of the data.

2. The method for lighting monitoring based on machine learning according to claim 1, characterized in that: According to the RGB data, IR grayscale data and Depth data of the image, a feature sequence is constructed, specifically: The average R value of pixels in the image is taken as the first eigenvalue; The average G value of pixels in the image is taken as the second eigenvalue; The average value of the B values ​​of the pixels in the image is taken as the third eigenvalue; The average IR value of pixels in the image is taken as the fourth eigenvalue; The average value of the depth value of the pixels in the image is taken as the fifth eigenvalue; The first eigenvalue, the second eigenvalue, the third eigenvalue, the fourth eigenvalue and the fifth eigenvalue are constructed as a characteristic sequence.

3. The method for lighting monitoring based on machine learning according to claim 1, characterized in that: The many-to-many network model includes but is not limited to: a many-to-many recurrent neural network model and an LSTM model.

4. A lighting monitoring system based on machine learning, characterized in that: include: Sample data acquisition unit: obtains RGB data, IR grayscale data and Depth data of the image in the sample image set, and obtains the luminous flux and color temperature of the lamp in the corresponding image; Feature sequence and label acquisition unit: constructs a feature sequence according to the RGB data, IR grayscale data and Depth data of the image, and uses the luminous flux and color temperature of the lamp in the corresponding image as the first label and the second label; Many-to-many network model training unit: inputting the feature sequence, the first label, and the second label into a pre-trained many-to-many regression network model for training to obtain a trained many-to-many network model; Light monitoring model acquisition unit: The model of RGB data, IR grayscale data and Depth data of the extracted image is integrated with the trained many-to-many network model to obtain the light monitoring model; Lamp monitoring unit: input the image to be monitored into the light monitoring model to obtain the luminous flux and color temperature of the lamp in the image; Also includes: Determine the lamp area according to the RGB data, IR grayscale data and Depth data of the image in the sample image set, and construct a feature sequence according to the RGB data, IR grayscale data and Depth data in the lamp area; The lamp area is determined specifically based on the change rate of the RGB data, IR grayscale data and Depth data of the image, and the lamp area is determined based on a sudden drop in the change rate of the data.

5. The lighting monitoring system based on machine learning according to claim 4, characterized in that: According to the RGB data, IR grayscale data and Depth data of the image, a feature sequence is constructed, specifically: The average R value of pixels in the image is taken as the first eigenvalue; The average G value of pixels in the image is taken as the second eigenvalue; The average value of the B values ​​of the pixels in the image is taken as the third eigenvalue; The average IR value of pixels in the image is taken as the fourth eigenvalue; The average value of the depth value of the pixels in the image is taken as the fifth eigenvalue; The first eigenvalue, the second eigenvalue, the third eigenvalue, the fourth eigenvalue and the fifth eigenvalue are constructed as a characteristic sequence.

6. The lighting monitoring system based on machine learning according to claim 4, characterized in that: The many-to-many network model includes but is not limited to: a many-to-many recurrent neural network model and an LSTM model.

7. An electronic device, characterized in that: include: A memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method steps described in any one of claims 1 to 3 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 3 are implemented.

Images