Model training method, illuminance determination method, device, program product

Abstract

The model training method, the illumination determination method, the device and the program product provided by the present disclosure relate to the technical field of image processing, and include the following steps: acquiring low dynamic range images of a preset environment acquired based on different exposure values and actual illumination values of the preset environment when the low dynamic range images are acquired; and training a preset model according to each low dynamic range image and the actual illumination value corresponding to the low dynamic range image to obtain an illumination prediction model. In the model training method, the illumination determination method, the device and the program product provided by the present disclosure, a predicted illumination value can be obtained according to the acquired low dynamic range image in a store, data does not need to be collected by a dedicated person, and the brightness of a lighting lamp in the store can be adjusted in real time according to the predicted illumination value.

Description

Technical Field

[0001] This disclosure relates to image processing technology, and more particularly to a model training method, an illumination determination method, an apparatus, and a program product. Background Technology

[0002] Currently, many stores are equipped with lighting to provide a suitable lighting environment. Since changes in the brightness of the external environment can affect the lighting environment inside the store, it is necessary to adjust the brightness of the lighting to create a suitable lighting environment for customers to linger.

[0003] In existing technology, a specialist uses an illuminance meter to collect the ambient illuminance in the store, and then adjusts the brightness of the lighting based on the actual illuminance in the store.

[0004] However, this method requires dedicated personnel to collect the store's illuminance, which is inefficient and cannot adjust the brightness of the store's lights in real time. Summary of the Invention

[0005] This disclosure provides a model training method, an illuminance determination method, an equipment, and a program product to solve the problem in the prior art that requires dedicated personnel to collect store illuminance and cannot adjust the brightness of store lighting in real time.

[0006] According to a first aspect of this application, a model training method is provided, comprising:

[0007] Acquire low dynamic range images of a preset environment based on different exposure values, and the actual illuminance value of the preset environment when acquiring the low dynamic range images;

[0008] Based on each of the low dynamic range images and the actual illuminance values ​​corresponding to the low dynamic range images, a preset model is trained to obtain an illuminance prediction model; the illuminance prediction model is used to obtain a high dynamic range image corresponding to the low dynamic range image, and the high dynamic range image is used to obtain the predicted illuminance value corresponding to the low dynamic range image.

[0009] According to a second aspect of this application, a method for determining ambient illuminance is provided, comprising:

[0010] Acquire a single-frame low dynamic range image obtained from a preset shooting environment;

[0011] The single-frame low dynamic range image is input into the illumination prediction model to obtain the predicted high dynamic range image corresponding to the single-frame low dynamic range image.

[0012] Based on the predicted high dynamic range image, determine the predicted illuminance value corresponding to the preset environment;

[0013] The illumination prediction model is trained using low dynamic range images of the training environment acquired based on different exposure values, and the actual illumination values ​​of the training environment when the low dynamic range images were acquired.

[0014] According to a third aspect of this application, a model training apparatus is provided, comprising:

[0015] The acquisition unit is used to acquire low dynamic range images of a preset environment based on different exposure values, and the actual illuminance value of the preset environment when the low dynamic range images are acquired.

[0016] The processing unit is used to train a preset model based on each of the low dynamic range images and the actual illuminance value corresponding to the low dynamic range image to obtain an illuminance prediction model; the illuminance prediction model is used to obtain a high dynamic range image corresponding to the low dynamic range image, and the high dynamic range image is used to obtain the predicted illuminance value corresponding to the low dynamic range image.

[0017] According to a fourth aspect of this application, an ambient light determination device is provided, comprising:

[0018] The acquisition unit is used to acquire a single-frame low dynamic range image obtained from a preset shooting environment.

[0019] The identification unit is used to input the single-frame low dynamic range image into the illumination prediction model to obtain a predicted high dynamic range image corresponding to the single-frame low dynamic range image.

[0020] The illuminance value determination unit is used to obtain the predicted illuminance value corresponding to the preset environment based on the predicted high dynamic range image.

[0021] The illumination prediction model is trained using low dynamic range images of the training environment acquired based on different exposure values, and the actual illumination values ​​of the training environment when the low dynamic range images were acquired.

[0022] According to a fifth aspect of this application, an electronic device is provided, including a memory and a processor; wherein,

[0023] The memory is used to store computer programs;

[0024] The processor is configured to read the computer program stored in the memory and execute the model training method as described in the first aspect or the environmental illumination determination method as described in the second aspect according to the computer program in the memory.

[0025] According to a sixth aspect of this application, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the model training method as described in the first aspect or the environmental illumination determination method as described in the second aspect is implemented.

[0026] According to a seventh aspect of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the model training method as described in the first aspect or the environmental illumination determination method as described in the second aspect.

[0027] The model training method, illuminance determination method, device, and program product disclosed herein include: acquiring low dynamic range (LVR) images of a preset environment collected based on different exposure values, and the actual illuminance value of the preset environment when the LVR images were collected; training a preset model based on each LVR image and the corresponding actual illuminance value to obtain an illuminance prediction model; the illuminance prediction model is used to acquire a high dynamic range (HVR) image corresponding to the LVR image, and the HVR image is used to acquire the predicted illuminance value corresponding to the LVR image. The model training method, illuminance determination method, device, and program product provided in this solution can obtain predicted illuminance values ​​based on the acquired LVR images within the store, eliminating the need for dedicated personnel to collect data, and can adjust the brightness of store lighting in real time based on the predicted illuminance values. Attached Figure Description

[0028] Figure 1 A schematic flowchart illustrating a model training method as shown in an exemplary embodiment of this application;

[0029] Figure 2 A schematic flowchart illustrating a model training method as shown in another exemplary embodiment of this application;

[0030] Figure 3 This is a schematic diagram illustrating the model training process of an exemplary embodiment of this application;

[0031] Figure 4 This is a flowchart illustrating an exemplary embodiment of the method for determining ambient illuminance according to this application;

[0032] Figure 5 A flowchart illustrating an environmental illuminance determination method as another exemplary embodiment of this application;

[0033] Figure 6 This is a structural diagram of a model training apparatus shown in an exemplary embodiment of this application;

[0034] Figure 7 A structural diagram of a model training apparatus shown in another exemplary embodiment of this application;

[0035] Figure 8 This is a structural diagram of an ambient illuminance determination device shown in an exemplary embodiment of this application;

[0036] Figure 9 This is a structural diagram of an electronic device illustrated in an exemplary embodiment of this application. Detailed Implementation

[0037] Currently, the brightness of lighting can be adjusted based on the ambient illuminance inside the store, thereby providing a suitable lighting environment. Specifically, a specialist can use an illuminance meter to collect the ambient illuminance inside the store, and then adjust the brightness of the lighting based on the actual indoor illuminance.

[0038] However, this requires equipping each store with an illuminance meter and having a dedicated person use the meter to collect illuminance data, which is inefficient. Moreover, indoor ambient illuminance changes in real time, and the method of having a dedicated person collect illuminance data cannot achieve real-time monitoring, thus making it impossible to adjust the brightness of the lighting based on the changing ambient illuminance.

[0039] To address the aforementioned technical problems, this application provides a model training method that uses low dynamic range (LR) images of a preset environment and actual illuminance values ​​for data training to obtain a high dynamic range (HDR) image. The HDR image is then used to obtain a predicted illuminance value. In use, simply acquiring the LR image of the preset environment is sufficient to obtain the HDR image from the model, which in turn provides the predicted illuminance value. This predicted illuminance value is then used to adjust the brightness of the lighting fixtures. The method provided in this application allows for real-time adjustment of lighting brightness by adjusting the frequency of acquiring LR images of the preset environment, without requiring dedicated personnel to collect environmental illuminance data.

[0040] Figure 1 This is a schematic flowchart illustrating a model training method as an exemplary embodiment of this application.

[0041] like Figure 1 As shown, the model training method provided in this embodiment includes:

[0042] Step 101: Obtain low dynamic range (LDR) images of a preset environment based on different exposure values, and the actual illuminance value of the preset environment when acquiring the LDR images.

[0043] The preset environment can be a space whose lighting is affected by changes in the brightness of the external environment, such as a room, a store, a residence, or a factory. There can be multiple preset environments, such as the environments of multiple stores.

[0044] Specifically, illuminance, or illuminance intensity, refers to the luminous flux of visible light received per unit area.

[0045] Specifically, dynamic range refers to the relative ratio of the highest and lowest values ​​of an electrical signal. In a photograph, this translates to the detail that can be displayed in highlight and shadow areas; a larger dynamic range results in richer detail. LDR images are low dynamic range images. In the real world, dynamic range can span 10⁶ to 10⁹ orders of magnitude, while the human eye can only perceive a dynamic range of around 10⁵ orders of magnitude. LDR images typically only reach a dynamic range of around 10² orders of magnitude.

[0046] Furthermore, an image acquisition device can be used to acquire multiple LDR images at the same image acquisition location using different exposure values. For example, 20 LDR images with different exposure values ​​can be acquired for a single image acquisition location.

[0047] In practical applications, when using an image acquisition device to acquire multiple LDR images with different exposure values ​​at the same image acquisition location, the illuminance value can also be measured simultaneously using an illuminance measuring instrument. The result obtained is the actual illuminance value corresponding to the acquired LDR image.

[0048] The method provided in this application can be executed by an electronic device with computing capabilities, such as a computer. This electronic device can acquire low dynamic range (LMR) images of a preset environment based on different exposure values, as well as the actual illuminance value of the preset environment when acquiring the LMR images.

[0049] For example, an image acquisition device can capture LDR images and send them to the electronic device, enabling the electronic device to acquire multiple LDR images. While acquiring LDR images, an illuminance measuring instrument can also collect illuminance data. The illuminance measuring instrument can then send the collected actual illuminance values ​​to the electronic device, allowing the electronic device to obtain the actual illuminance values ​​of the preset environment at the time of LDR image acquisition. Alternatively, a control unit can be set up that can simultaneously control both the image acquisition device and the illuminance measuring instrument, thus enabling the acquisition of actual illuminance values ​​while acquiring LDR images of the preset environment.

[0050] For example, LDR images captured by an image acquisition device can be imported into an electronic device, and the actual illuminance values ​​collected by an illuminance measuring instrument can be imported into the electronic device. The electronic device can determine the actual illuminance value corresponding to each LDR image based on the acquisition time of the LDR image and the acquisition time of the actual illuminance value.

[0051] Step 102: Train the preset model based on each low dynamic range image and the actual illumination value corresponding to the low dynamic range image to obtain the illumination prediction model.

[0052] The illuminance prediction model is used to obtain the high dynamic range image corresponding to the low dynamic range image, and the high dynamic range image is used to obtain the predicted illuminance value corresponding to the low dynamic range image.

[0053] The preset model can be a pre-built model, such as a neural network model.

[0054] Specifically, the electronic device can train this neural network model based on each low dynamic range image and the actual illuminance value corresponding to the low dynamic range image to obtain the target model, namely the illuminance prediction model.

[0055] Furthermore, each LDR image can be used as training data, and the actual illuminance value corresponding to the LDR image can be used as a data label, so that electronic devices can use the LDR image with the actual illuminance value result to train the preset model.

[0056] In practical applications, the preset model can process the input LDR image and identify the predicted illuminance value corresponding to that LDR image. It can also compare the predicted illuminance value of the LDR image with the actual illuminance value, and then adjust the parameters in the preset model based on the comparison results. Through multiple iterations, the difference between the predicted illuminance value determined by the preset model and the actual illuminance value can be made to meet the requirements, thus obtaining an illuminance prediction model that meets the requirements.

[0057] When processing the input LDR image, the preset model can convert the LDR image into a High Dynamic Range (HDR) image based on the model's internal parameters. The electronic device then determines the predicted illumination value based on the HDR image obtained by the preset model.

[0058] In this application, the HDR image refers to a high dynamic range image, and the dynamic range of an HDR image can typically reach 105 orders of magnitude.

[0059] Specifically, in this implementation, standard HDR images corresponding to multiple LDR images from the same image acquisition location can be generated in advance. These standard HDR images can then be used as data labels to train the model.

[0060] Furthermore, in this implementation, after converting the LDR image to an HDR image using a preset model, the electronic device can compare the identified HDR image with a standard HDR image of the LDR image, and adjust the parameters in the preset model based on the comparison results. Through multiple iterations, the difference between the HDR image identified by the preset model and the standard HDR image can be made to meet the requirements, thereby obtaining a satisfactory illumination prediction model.

[0061] In this implementation, a training data set can have two labels, which allows the model to be trained under two constraints, making the prediction results of the resulting illumination prediction model more accurate.

[0062] In practical applications, the trained illuminance prediction model can be incorporated into an illuminance recognition device, which can be, for example, a computer. Alternatively, an image acquisition device can be used to capture indoor LDR images, which can then be sent to the illuminance recognition device. This device can process the received LDR images using the trained illuminance prediction model to obtain predicted HDR images. The illuminance recognition device can then determine the predicted illuminance value based on these predicted HDR images. For instance, an LDR image can be input into the illuminance prediction model, which can output a predicted HDR image corresponding to the input LDR image. The illuminance recognition device can then determine the predicted illuminance value based on the model's output predicted HDR image. In this implementation, an illuminance prediction value can be obtained from a single LDR image, thereby improving the efficiency of illuminance value recognition.

[0063] The image acquisition device can be connected to the illuminance recognition device via wired or wireless means, enabling the automatic acquisition of LDR images and the generation of illuminance prediction values ​​without manual intervention. Furthermore, this method allows for real-time acquisition of LDR images and real-time determination of the corresponding illuminance values, enabling real-time adjustment of indoor lighting brightness based on these values.

[0064] The methods provided in this application are all executed by a device equipped with the methods provided in this application, which is typically implemented in hardware and / or software.

[0065] The model training method provided in this application includes acquiring low dynamic range (LDR) images of a preset environment based on different exposure values, and the actual illuminance values ​​of the preset environment when acquiring the LDR images; training a preset model based on each LDR image and the corresponding actual illuminance value to obtain an illuminance prediction model. This application can obtain illuminance prediction values ​​from a single LDR image, and both the acquisition of the LDR image and the illuminance prediction value can be automated, eliminating the need for manual acquisition, thus improving the efficiency of illuminance value recognition; furthermore, it can acquire LDR images in real time and determine the corresponding illuminance value in real time, thereby enabling real-time adjustment of indoor lighting brightness based on the illuminance value.

[0066] Figure 2 This is a schematic flowchart illustrating a model training method as another exemplary embodiment of this application.

[0067] like Figure 2 As shown, the model training method provided in this embodiment includes:

[0068] Step 201: Obtain low dynamic range images of a preset environment based on different exposure values, and the actual illuminance value of the preset environment when acquiring the low dynamic range images.

[0069] The implementation method and principle of step 201 are similar to those of step 101, and will not be repeated here.

[0070] Step 202: Fuse the low dynamic range images with different exposure values ​​to obtain a standard high dynamic range image.

[0071] Among them, after acquiring LDR images of a preset environment based on different exposure values, these LDR images can be fused to obtain an HDR image.

[0072] During fusion, multiple LDR images acquired at the same location can be fused.

[0073] Specifically, in this embodiment, the low dynamic range image of the preset environment may include: low dynamic range images of each point in the preset environment acquired based on different exposure values.

[0074] For example, eight data acquisition devices can be used to collect datasets from eight retail stores. These data acquisition devices can be equipped with 360-degree panoramic cameras and lux meters.

[0075] In one embodiment, both the 360-degree panoramic camera and the lux meter can be mounted at a preset height on the data acquisition device, such as eye level when a user is standing. In another embodiment, the lux meter can be mounted facing upwards. The 360-degree panoramic camera can be a dual fisheye lens camera calibrated through camera calibration.

[0076] This data acquisition device can be programmed with 20 different camera shutter speeds ranging from 5 milliseconds to 100 milliseconds. Multiple locations can be set, and the device can be controlled to acquire LDR images of these locations within a preset environment based on different exposure values. For example, the aforementioned eight retail stores can serve as a preset environment, and 8345 locations can be selected within this environment as data points. The device can then collect LDR images and actual illuminance values ​​for these locations daily.

[0077] The data acquisition device is capable of sending the acquired data to an electronic device used to perform the method provided in this embodiment.

[0078] Correspondingly, the low dynamic range image of the preset environment includes low dynamic range images of each point in the preset environment acquired based on different exposure values. When the standard high dynamic range image is obtained by fusing the low dynamic range images with different exposure values, the electronic device can perform preset fusion processing on the low dynamic range images with different exposure values ​​corresponding to the same point to obtain the standard high dynamic range image corresponding to the point.

[0079] For example, a preset fusion processing method can be set in the electronic device, allowing it to fuse LDR images with different exposure values ​​corresponding to the same location to obtain an HDR image corresponding to each location. For instance, location 1 could include 20 LDR images with different exposure values, and location 2 could also include 20 LDR images with different exposure values. The electronic device can fuse the LDR images of location 1 to obtain the HDR image of location 1, and it can also fuse the LDR images of location 2 to obtain the HDR image of location 2.

[0080] The aforementioned preset fusion process may include:

[0081] The first high dynamic range image is obtained by fusing the low dynamic range images with different exposure values ​​corresponding to the same point.

[0082] Specifically, Photosphere software can be used to fuse single-frame LDR images with different exposure values ​​at the same location to generate a first HDR image. For example, if 8345 locations are set, 8345 first HDR images can be generated.

[0083] Furthermore, a preset factor can be used to calibrate the first high dynamic range image to obtain a standard high dynamic range image.

[0084] Due to manufacturing differences in image acquisition devices, such as cameras, the response curves of different cameras are not identical. Therefore, the first HDR image generated by directly fusing LDR images needs to be calibrated using a scaling factor to obtain a standard HDR image. This calibrated standard HDR image is then used for model training, resulting in more accurate recognition results.

[0085] Step 203: Input the low dynamic range image into the preset model to obtain the predicted high dynamic range image corresponding to the low dynamic range image.

[0086] During model training, electronic devices can input LDR images into a preset model, thereby generating predicted HDR images corresponding to each LDR image based on the internal parameters of the preset model.

[0087] For example, an electronic device can extract image features of a single-frame LDR image based on parameters within a preset model, and then generate a predicted HDR image of the single-frame LDR image based on these image features.

[0088] Step 204: Determine the predicted illuminance value based on the predicted high dynamic range image.

[0089] Specifically, after the electronic device generates an HDR image corresponding to the LDR image based on a preset model, it can also determine the predicted illuminance value based on the predicted HDR image.

[0090] In one implementation, a preset model can be used to convert an LDR image into a predicted HDR image, and the electronic device can determine a predicted illuminance value based on the predicted HDR image determined by the preset model. In this implementation, the electronic device can train the model based on the determined predicted illuminance value and the actual illuminance value.

[0091] Furthermore, when determining the predicted illuminance value based on the predicted HDR image, the electronic device can determine the brightness value corresponding to each pixel based on the pixel information in the predicted HDR image.

[0092] In practical applications, after generating the predicted HDR image, the three-channel values ​​of each pixel in the HDR image can be obtained, specifically the R, G, and B channel values. Then, the luminance value L of each pixel is determined based on its three-channel values, using the following formula:

[0093] L=179×(0.2126·R+0.7152·G+0.0722·B)

[0094] Where R is the value of the pixel in the R channel, G is the value of the pixel in the G channel, and B is the value of the pixel in the B channel.

[0095] Specifically, the predicted illumination value for a high dynamic range image can be determined based on the brightness value of each pixel.

[0096] For example, the sum of the brightness values ​​of each pixel can be calculated as the predicted illuminance value for predicting HDR. Alternatively, the average value of each pixel can be calculated as the predicted illuminance value for predicting HDR.

[0097] Furthermore, the brightness of each pixel in a specified region of the predicted high dynamic range image can be integrated to obtain the predicted illuminance value of the predicted high dynamic range image.

[0098] For example, if the illuminance meter is placed vertically upwards, the Northern Hemisphere region in the predicted HDR image can be used as the designated area. Therefore, the brightness of each pixel in the Northern Hemisphere region of the predicted HDR image can be integrated. Specifically, the predicted illuminance value can be determined using the following formula:

[0099]

[0100] Among them, the predicted illuminance value This is used to characterize the lighting conditions at the location of the image acquisition device in a preset environment. Among them, It is to predict the location in the HDR image as The brightness values ​​of the pixels, where θ and It is a parameter that characterizes the polar coordinate system coordinates in a panoramic image.

[0101] Step 205: Based on the standard high dynamic range image, the predicted high dynamic range image, the actual illuminance value corresponding to the low dynamic range image, and the predicted illuminance value, optimize the parameters in the preset model to obtain the illuminance prediction model.

[0102] The standard HDR image is obtained by fusing multiple LDR images with different exposure values ​​at the same location; the predicted HDR image is obtained by predicting a single frame LDR image at the same location using a preset model; the actual illuminance value is the illuminance value measured by a lux meter when the LDR image is acquired at that location; and the predicted illuminance value is the predicted illuminance value determined based on the predicted HDR.

[0103] Specifically, the standard high dynamic range image and the actual illuminance value of the low dynamic range image can be used as labels for the low dynamic range image. Based on the labels of the low dynamic range image, the predicted high dynamic range image of the low dynamic range image determined by the preset model, and the predicted illuminance value determined by the predicted high dynamic range image, the parameters in the preset model are optimized to obtain the illuminance prediction model.

[0104] The predicted HDR image is obtained by identifying a single frame of LDR image using a preset model, while the standard HDR image is obtained by fusing multiple frames of LDR image. Therefore, the standard HDR image can be considered the accurate image. The standard HDR image can be used as a label for the LDR image to specifically constrain the predicted HDR image and train the preset model.

[0105] For example, if 20 LDR images with different exposure values ​​are collected for point A, these LDR images can be fused to obtain a standard HDR image, which can then be used as a label for these 20 LDR images.

[0106] Furthermore, the predicted illuminance value is determined based on the predicted HDR image, while the actual illuminance value is measured using a lux meter. Therefore, the actual illuminance value can be considered accurate. The actual illuminance value can be used as a label for the LDR image to constrain the predicted illuminance value and train the preset model.

[0107] For example, if 20 LDR images with different exposure values ​​are collected for point A, and the actual illuminance value can also be collected during image acquisition, then the actual illuminance value can be used as a label for these 20 LDR images.

[0108] The specific method for optimizing the preset model is to use standard high dynamic range images as labels for predicting high dynamic range images, and actual illuminance values ​​as labels for predicting illuminance values ​​to train and optimize the preset model.

[0109] In practical applications, the first loss can be determined based on the standard high dynamic range image and the predicted high dynamic range image.

[0110] The second loss is determined based on the actual illuminance value and the predicted illuminance value corresponding to the low dynamic range image;

[0111] Then, based on the first loss and the second loss, the parameters in the preset model are optimized to obtain the illuminance prediction model.

[0112] Furthermore, a loss function can be pre-set, which, along with a standard high dynamic range image and a predicted high dynamic range image, can be used to determine the first loss. The same loss function, along with the actual illuminance value and predicted illuminance value corresponding to the low dynamic range image, can also be used to determine the second loss.

[0113] Specifically, gradient backpropagation can be performed based on the first loss and the second loss to optimize the parameters in the preset model. Through multiple iterations, the first loss and / or the second loss can meet the preset requirements. Once the preset requirements are met, the model can be considered to have been trained and the illumination prediction model has been obtained.

[0114] Figure 3 This is a schematic diagram illustrating the model training process of an exemplary embodiment of this application.

[0115] This embodiment uses a multi-frame LDR image of a point in a preset environment for illustration.

[0116] like Figure 3 As shown, multiple LDR images 31 can be acquired at point A in a preset environment based on different exposure values. A standard HDR image 32 can be generated from the multiple LDR images 31. Actual illuminance values ​​33 can also be acquired while acquiring the multiple LDR images 31.

[0117] By inputting any frame 311 from the multi-frame LDR images 31 into the preset model 34, a predicted HDR image 35 can be obtained. The electronic device can determine the predicted illuminance value 36 based on the predicted HDR image 35. Subsequently, the electronic device can compare the standard HDR image 32 and the predicted HDR image 35, and also compare the actual illuminance value 33 and the predicted illuminance value 36, thereby training the preset model 34 based on two constraints.

[0118] The model training method in this application also includes:

[0119] Obtain the exposure value when acquiring low dynamic range images.

[0120] When acquiring a single-frame LDR image, the exposure value used to acquire that LDR image can also be recorded. In fact, for most image acquisition devices, the exposure value is usually part of the Exchangeable Image File Format (EXIF), so it can be obtained through EXIF.

[0121] A standard high dynamic range image is obtained by fusing various low dynamic range images with different exposure values. This step is similar in implementation and principle to step 202, and will not be described again.

[0122] Input the low dynamic range image and the corresponding exposure value into the preset model to obtain the predicted high dynamic range image corresponding to the low dynamic range image.

[0123] During model training, the electronic device can input LDR images and their corresponding exposure values ​​into a preset model, thereby generating a predicted HDR image corresponding to each LDR image based on the internal parameters of the preset model.

[0124] For example, an electronic device can extract the image features of a single-frame LDR image based on the parameters inside a preset model, and then combine the exposure value and image features of the LDR image to generate a predicted HDR image of the single-frame LDR image.

[0125] Predicted illuminance values ​​are determined based on the predicted high dynamic range image.

[0126] Based on the standard high dynamic range image, the predicted high dynamic range image, the actual illuminance value corresponding to the low dynamic range image, and the predicted illuminance value, the parameters in the preset model are optimized to obtain the illuminance prediction model.

[0127] Using the exposure values ​​of LDR images as training data can potentially help the pre-defined model better handle overexposed and underexposed areas, thus making more accurate HDR predictions.

[0128] The above two steps are similar in implementation and principle to steps 204 and 205. The exposure value training information is added on the basis of the above embodiment, and will not be described again.

[0129] Figure 4 This is a schematic flowchart illustrating an exemplary embodiment of the present application for determining environmental illuminance.

[0130] like Figure 4 As shown, the environmental illuminance determination method provided in this embodiment includes:

[0131] Step 401: Obtain a single-frame low dynamic range image obtained from the shooting preset environment.

[0132] The method provided in this application can be an electronic device with computing capabilities, such as a computer.

[0133] Specifically, an LDR image can be captured by an image acquisition device and sent to an electronic device that performs the method provided in this application, thereby enabling the electronic device to acquire a single-frame LDR image of a preset environment.

[0134] A single-frame LDR image refers to a still image. A frame is the smallest unit of video animation, consisting of a single image frame.

[0135] Step 402: Input a single-frame low dynamic range image into the illumination prediction model to obtain a predicted HDR image corresponding to a preset environment; wherein, the illumination prediction model is trained using low dynamic range images of the training environment collected based on different exposure values, and the actual illumination values ​​of the training environment when collecting low dynamic range images.

[0136] Specifically, a pre-trained illuminance prediction model can be set in electronic devices, and this illuminance prediction model can be achieved through... Figure 1 , Figure 2 The training was obtained from any of the embodiments shown.

[0137] The acquired single-frame LDR image can be input into the illumination prediction model, which can generate a predicted HDR image corresponding to the LDR image.

[0138] Step 403: Determine the predicted illuminance value corresponding to the preset environment based on the predicted HDR image.

[0139] Furthermore, electronic devices can process the predicted HDR image to determine the predicted illumination value. For example, the brightness value corresponding to each pixel can be determined based on the pixel information of the predicted high dynamic range image; then, the predicted illumination value of the predicted high dynamic range image can be determined based on the brightness value of each pixel.

[0140] Figure 5This is a flowchart illustrating an environmental illumination determination method as another exemplary embodiment of this application.

[0141] like Figure 5 As shown, the environmental illuminance determination method provided in this embodiment includes:

[0142] Step 501: Obtain a single-frame low dynamic range image obtained from shooting in a preset environment, and the exposure value when acquiring the single-frame low dynamic range image.

[0143] In this application, an LDR image of a preset environment is acquired using an image acquisition device. This can be achieved using a panoramic camera, specifically a dual fisheye lens camera, to acquire a single-frame LDR image of the store. The image acquisition device can then send the captured LDR image to an electronic device that performs the method provided in this embodiment, enabling the electronic device to acquire the single-frame LDR image.

[0144] Specifically, the image acquisition device can also send the exposure value used when capturing the LDR image to the electronic device. This exposure value can be obtained from the camera's EXIF ​​file.

[0145] Step 502: Input the single-frame low dynamic range image and the exposure value when acquiring the single-frame low dynamic range image into the illumination prediction model to obtain the predicted HDR image corresponding to the preset environment; wherein, the illumination prediction model is trained using low dynamic range images of the training environment acquired based on different exposure values, as well as the actual illumination value and exposure value of the training environment when acquiring the low dynamic range image.

[0146] Specifically, a pre-trained illuminance prediction model can be set in electronic devices, and this illuminance prediction model can be achieved through... Figure 2 The training was performed using the example shown. This training environment could be, for example, [the following text is incomplete and requires further context: "the example shown is "trained using "]. Figure 2 The default environment in the program.

[0147] For example, predicted high dynamic range images can be displayed on image display devices, such as computers or cameras, allowing users to understand the preset environment in real time.

[0148] Step 503: Determine the predicted illuminance value corresponding to the preset environment based on the predicted HDR image.

[0149] Optionally, the method provided in this application may also include:

[0150] Step 504: Adjust the brightness of the lighting lamps set in the preset environment according to the predicted illuminance value.

[0151] This method involves transmitting the predicted illuminance value to an automatic lighting brightness adjustment module for brightness adjustment. This can be implemented using a computer, chip, or program. The predicted illuminance value is transmitted to the automatic lighting brightness adjustment module within the computer or chip. The program then obtains the lighting brightness adjustment signal and transmits it to the lighting fixture to adjust its brightness.

[0152] Figure 6 This is a structural diagram of a model training apparatus shown in an exemplary embodiment of this application.

[0153] like Figure 6 As shown, the model training device 600 provided in this application includes:

[0154] The acquisition unit 610 is used to acquire low dynamic range images of a preset environment based on different exposure values, and the actual illuminance value of the preset environment when acquiring the low dynamic range images.

[0155] The processing unit 620 is used to train a preset model based on each low dynamic range image and the actual illuminance value corresponding to the low dynamic range image to obtain an illuminance prediction model; the illuminance prediction model is used to obtain a high dynamic range image corresponding to the low dynamic range image, and the high dynamic range image is used to obtain the predicted illuminance value corresponding to the low dynamic range image.

[0156] The principle, implementation method, and technical effects of the model training device provided in this application are similar to those of the model training device provided in this application. Figure 1 Similarities, no further explanation needed.

[0157] Figure 7 This is a structural diagram of a model training apparatus shown as another exemplary embodiment of this application.

[0158] like Figure 7 As shown, based on the above embodiments, the processing unit 620 in the model training apparatus 700 provided in this application includes:

[0159] The fusion module 621 is used to fuse various low dynamic range images with different exposure values ​​to obtain a standard high dynamic range image;

[0160] The recognition module 622 is used to input the low dynamic range image into the preset model to obtain the predicted high dynamic range image corresponding to the low dynamic range image;

[0161] Determining module 623 is used to determine the predicted illuminance value based on the predicted high dynamic range image;

[0162] Training module 624 is used to optimize the parameters in the preset model based on the standard high dynamic range image, the predicted high dynamic range image, the actual illuminance value corresponding to the low dynamic range image, and the predicted illuminance value, to obtain the illuminance prediction model.

[0163] In the model training device 700 provided in this application, the low dynamic range images of a preset environment acquired by the acquisition unit 610 based on different exposure values ​​include: low dynamic range images of each point in the preset environment acquired based on different exposure values.

[0164] Accordingly, the fusion module 621 is specifically used to perform preset fusion processing on low dynamic range images with different exposure values ​​corresponding to the same point to obtain a standard high dynamic range image corresponding to the point.

[0165] The fusion module 621 is specifically used to fuse low dynamic range images with different exposure values ​​corresponding to the same point to obtain a first high dynamic range image; and to calibrate the first high dynamic range image using a preset factor to obtain a standard high dynamic range image.

[0166] Module 623 is specifically used for:

[0167] Based on the pixel information of the predicted high dynamic range image, determine the brightness value corresponding to each pixel.

[0168] The predicted illumination value for the high dynamic range image is determined based on the brightness value of each pixel.

[0169] Module 623 is specifically used for:

[0170] The brightness of each pixel in a specified region of the predicted high dynamic range image is integrated to obtain the predicted illuminance value of the predicted high dynamic range image.

[0171] Training module 624 is specifically used for:

[0172] The first loss is determined based on the standard high dynamic range image and the predicted high dynamic range image; the second loss is determined based on the actual illuminance value and the predicted illuminance value corresponding to the low dynamic range image; and the parameters in the preset model are optimized based on the first loss and the second loss to obtain the illuminance prediction model.

[0173] The model training device 700 provided in this application,

[0174] The acquisition unit 610 is also used to acquire the exposure value when acquiring low dynamic range images;

[0175] Processing unit 620 is specifically used for:

[0176] A standard high dynamic range image is obtained by fusing various low dynamic range images with different exposure values.

[0177] Input the low dynamic range image and the exposure value corresponding to the low dynamic range image into the preset model to obtain the predicted high dynamic range image corresponding to the low dynamic range image.

[0178] Determine the predicted illuminance value based on the predicted high dynamic range image;

[0179] Based on the standard high dynamic range image, the predicted high dynamic range image, the actual illuminance value corresponding to the low dynamic range image, and the predicted illuminance value, the parameters in the preset model are optimized to obtain the illuminance prediction model.

[0180] Figure 8 This is a structural diagram of an ambient illuminance determination device shown in an exemplary embodiment of this application.

[0181] like Figure 8 As shown, the ambient illuminance determining device 800 provided in this application includes:

[0182] The acquisition unit 810 is used to acquire a single-frame low dynamic range image obtained from a preset shooting environment.

[0183] The recognition unit 820 is used to input a single-frame low dynamic range image into the illumination prediction model to obtain a predicted HDR image corresponding to a preset environment. The illumination prediction model is trained using low dynamic range images of the training environment acquired based on different exposure values, and the actual illumination values ​​of the training environment when acquiring the low dynamic range images.

[0184] The illuminance value determination unit 830 is used to determine the predicted illuminance value corresponding to the preset environment based on the predicted HDR image.

[0185] The acquisition unit 810 is also used to acquire the exposure value when acquiring a single frame of low dynamic range image.

[0186] The recognition unit 820 is specifically used to input a single-frame low dynamic range image and the exposure value when acquiring the single-frame low dynamic range image into the illumination prediction model to obtain a predicted HDR image corresponding to a preset environment. The exposure value when acquiring the low dynamic range image is also used when training the illumination prediction model.

[0187] Optionally, the device also includes:

[0188] The adjustment unit 840 is used to adjust the brightness of the lighting lamps set in the preset environment according to the predicted illuminance value.

[0189] Figure 9 This is a structural diagram of an electronic device illustrated in an exemplary embodiment of this application.

[0190] like Figure 9 As shown, the electronic device provided in this embodiment includes:

[0191] Memory 901;

[0192] Processor 902; and

[0193] Computer programs;

[0194] The computer program is stored in memory 901 and configured to be executed by processor 902 to implement any of the above-mentioned model training methods or environmental illumination determination methods.

[0195] This embodiment also provides a computer-readable storage medium on which a computer program is stored.

[0196] The computer program is executed by the processor to implement any of the above-mentioned model training methods or environmental illumination determination methods.

[0197] This embodiment also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described model training methods or environmental illumination determination methods.

[0198] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0199] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model training method, characterized in that, include: Acquire low dynamic range images of a preset environment based on different exposure values, and the actual illuminance value of the preset environment when acquiring the low dynamic range images; Based on each of the low dynamic range images, the actual illuminance value corresponding to the low dynamic range image, the predicted high dynamic range image, the predicted illuminance value, and the standard high dynamic range image, a preset model is trained through two constraints, namely comparing the standard high dynamic range image and the predicted high dynamic range image, and comparing the actual illuminance value and the predicted illuminance value, to obtain an illuminance prediction model. The step of training the preset model to obtain the illuminance prediction model includes: A preset fusion process is performed on low dynamic range images with different exposure values ​​corresponding to the same point to obtain a standard high dynamic range image corresponding to the point. The preset fusion process includes fusing low dynamic range images with different exposure values ​​corresponding to the same point to obtain a first high dynamic range image; and calibrating the first high dynamic range image using a preset factor to obtain a standard high dynamic range image. The low dynamic range image is input into a preset model to obtain a predicted high dynamic range image corresponding to the low dynamic range image. The predicted illuminance value is determined based on the predicted high dynamic range image; The illuminance prediction model is obtained by optimizing the parameters in the preset model based on the standard high dynamic range image, the predicted high dynamic range image, the actual illuminance value corresponding to the low dynamic range image, and the predicted illuminance value. The step of determining the illuminance prediction model includes: determining a first loss based on the standard high dynamic range image and the predicted high dynamic range image; determining a second loss based on the actual illuminance value corresponding to the low dynamic range image and the predicted illuminance value; and optimizing the parameters in the preset model based on the first loss and the second loss to obtain the illuminance prediction model. A low dynamic range image is input to an illuminance recognition device, which uses the illuminance prediction model to obtain a predicted high dynamic range image corresponding to the input low dynamic range image, and determines the predicted illuminance value based on the predicted high dynamic range image.

2. The method according to claim 1, characterized in that, Determining the predicted illuminance value based on the predicted high dynamic range image includes: Based on the pixel information of the predicted high dynamic range image, determine the brightness value corresponding to each pixel. The predicted illumination value of the predicted high dynamic range image is determined based on the brightness value of each pixel.

3. The method according to claim 2, characterized in that, Determining the predicted illumination value of the predicted high dynamic range image based on the brightness value of each pixel includes: The brightness of each pixel in a specified region of the predicted high dynamic range image is integrated to obtain the predicted illuminance value of the predicted high dynamic range image.

4. The method according to any one of claims 1-3, characterized in that, The step of optimizing the parameters in the preset model based on the standard high dynamic range image, the predicted high dynamic range image, the actual illuminance value corresponding to the low dynamic range image, and the predicted illuminance value to obtain the illuminance prediction model includes: The standard high dynamic range image and the actual illuminance value of the low dynamic range image are used as the labels of the low dynamic range image; Based on the label of the low dynamic range image, the predicted high dynamic range image of the low dynamic range image determined by the preset model, and the predicted illuminance value determined by the predicted high dynamic range image, the parameters in the preset model are optimized to obtain the illuminance prediction model.

5. The method according to any one of claims 1-3, characterized in that, Also includes: Obtain the exposure value when acquiring the low dynamic range image; The step of inputting the low dynamic range image into a preset model to obtain a predicted high dynamic range image corresponding to the low dynamic range image includes: The low dynamic range image and the exposure value corresponding to the low dynamic range image are input into a preset model to obtain a predicted high dynamic range image corresponding to the low dynamic range image.

6. A method for determining ambient illuminance, characterized in that, include: Acquire a single-frame low dynamic range image obtained from a preset shooting environment; Obtain the exposure value when acquiring the single-frame low dynamic range image; The single-frame low dynamic range image and the exposure value when the single-frame low dynamic range image was acquired are input into the illumination prediction model to obtain the predicted high dynamic range image corresponding to the single-frame low dynamic range image. Based on the predicted high dynamic range image, determine the predicted illuminance value corresponding to the preset environment; The illumination prediction model is obtained by the model training method according to any one of claims 1-5.

7. The method according to claim 6, characterized in that, Also includes: The brightness of the lighting fixtures set in the preset environment is adjusted according to the predicted illuminance value.

8. A model training device, characterized in that, include: The acquisition unit is used to acquire low dynamic range images of a preset environment based on different exposure values, and the actual illuminance value of the preset environment when the low dynamic range images are acquired. The processing unit is configured to train a preset model based on each of the low dynamic range (LVR) images, the actual illuminance value corresponding to the LVR images, the predicted high dynamic range (HVR) image, the predicted illuminance value, and the standard HVR image, through two constraints: comparing the standard HVR image and the predicted HVR image, and comparing the actual illuminance value and the predicted illuminance value, to obtain an illuminance prediction model. The training of the preset model to obtain the illuminance prediction model includes: performing a preset fusion process on LVR images with different exposure values ​​corresponding to the same location to obtain the standard HVR image corresponding to the location; the preset fusion process includes fusing the LVR images with different exposure values ​​corresponding to the same location to obtain a first HVR image; calibrating the first HVR image using a preset factor to obtain the standard HVR image; and inputting the LVR image into the preset model. The process involves obtaining a predicted high dynamic range (HMR) image corresponding to the low dynamic range (LMR) image; determining the predicted illuminance value based on the predicted HMR image; optimizing the parameters in the preset model based on the standard HMR image, the predicted HMR image, the actual illuminance value corresponding to the LMR image, and the predicted illuminance value to obtain the illuminance prediction model; and determining the illuminance prediction model by: determining a first loss based on the standard HMR image and the predicted HMR image; determining a second loss based on the actual illuminance value corresponding to the LMR image and the predicted illuminance value; optimizing the parameters in the preset model based on the first loss and the second loss to obtain the illuminance prediction model; inputting the LMR image to an illuminance recognition device, which uses the illuminance prediction model to obtain a predicted HMR image corresponding to the input LMR image and determines the predicted illuminance value based on the predicted HMR image. The illumination prediction model is used to obtain a predicted high dynamic range image corresponding to the low dynamic range image, and the predicted high dynamic range image is used to obtain a predicted illumination value corresponding to the low dynamic range image.

9. An ambient illumination determination device, characterized in that, include: The acquisition unit is used to acquire a single-frame low dynamic range image obtained by shooting in a preset environment, and to acquire the exposure value when acquiring the single-frame low dynamic range image. The identification unit is used to input the single-frame low dynamic range image and the exposure value when the single-frame low dynamic range image was acquired into the illumination prediction model to obtain a predicted high dynamic range image corresponding to the single-frame low dynamic range image. An illuminance value determination unit is used to obtain a predicted illuminance value corresponding to the preset environment based on the predicted high dynamic range image. The illumination prediction model is obtained using the model training device according to claim 8.

10. An electronic device, characterized in that, It includes a memory and a processor; wherein the memory is used to store computer programs; The processor is configured to read the computer program stored in the memory and execute the method described in any one of claims 1-5 or 6-7 according to the computer program in the memory.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method described in any one of claims 1-5 or 6-7.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-5 or 6-7.

Images