Indoor lighting recognition method and device, electronic equipment and storage medium

Abstract

The present application relates to the technical field of image processing, and provides an indoor lighting recognition method and device, electronic equipment and storage medium, the method comprising: obtaining an indoor photo of a house to be recognized; dividing the indoor photo into multiple grids; analyzing the image features of the multiple grids to determine lighting features representing the lighting degree of the house to be recognized; inputting the lighting features into a pre-trained lighting recognition model to obtain a recognition result of the lighting degree of the house to be recognized. The present application can accurately recognize indoor lighting.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an indoor lighting recognition method and device, electronic equipment and storage medium. BACKGROUND

[0002] The indoor lighting degree is an important dimension to judge the quality of a house. Good lighting means that the house is bright and the quality of the house is high. On the contrary, the quality of a house with poor lighting is also low. Lighting is affected by many dimensions, such as the orientation of the house, the floor-to-floor distance, the floor, the depth and the width, etc. At the same time, the shooting exposure, the weather, whether the light is on or not, and the curtain, etc. will affect the judgment of the indoor lighting degree of the house. Therefore, how to accurately and objectively evaluate the indoor lighting degree is a problem to be solved by those skilled in the art. SUMMARY

[0003] The present application provides an indoor lighting recognition method, device, electronic equipment and storage medium, which can accurately recognize indoor lighting.

[0004] Embodiments of the present application can be implemented as follows:

[0005] In a first aspect, the present application provides an indoor lighting recognition method, which comprises:

[0006] obtaining an indoor photo of a house to be recognized;

[0007] dividing the indoor photo into a plurality of grids;

[0008] analyzing the image features of the plurality of grids to determine lighting features representing the lighting degree of the house to be recognized;

[0009] inputting the lighting features into a pre-trained lighting recognition model to obtain a recognition result of the lighting degree of the house to be recognized.

[0010] In an optional implementation, the step of analyzing the image features of the plurality of grids to determine the lighting features representing the lighting degree of the house to be recognized comprises:

[0011] statistically analyzing the image features of the plurality of grids to obtain overall features of the indoor photo;

[0012] analyzing the brightness values of the plurality of grids to obtain brightness features of the indoor photo;

[0013] taking the overall features and the brightness features as the lighting features.

[0014] In an optional embodiment, the step of analyzing the luminance values of the plurality of grids to obtain the luminance feature of the indoor photo comprises:

[0015] performing luminance distribution analysis on the luminance values of the plurality of grids to obtain a grid feature of the indoor photo;

[0016] performing contrast analysis on the luminance values of the plurality of grids to obtain a contrast feature of the indoor photo;

[0017] performing luminance change analysis on the luminance values of the plurality of grids to obtain a change feature of the indoor photo;

[0018] determining a service feature of the indoor photo according to a plurality of grids with luminance values greater than a preset value and continuous positions;

[0019] taking the grid feature, the contrast feature, the change feature and the service feature as the lighting feature.

[0020] In an optional embodiment, the training process of the lighting recognition model is as follows:

[0021] obtaining a plurality of sample images, the sample images comprising labeled sample images and unlabeled sample images, the real label of the labeled sample image representing the lighting degree of the labeled sample image;

[0022] determining a virtual label of the unlabeled sample image according to the labeled sample image and its real label;

[0023] training the lighting recognition model using the labeled sample image and its real label and the unlabeled sample image and its virtual label to obtain a trained lighting recognition model.

[0024] In an optional embodiment, the step of determining a virtual label of the unlabeled sample image according to the labeled sample image and its real label comprises:

[0025] determining, from the unlabeled sample image, a preliminary selected unlabeled sample image belonging to the same category as the labeled sample image using a label propagation algorithm according to the labeled sample image and its real label;

[0026] taking the real label of the labeled sample image as a virtual label of the preliminary selected unlabeled sample image belonging to the same type as the labeled sample image;

[0027] if there is still a selected unlabeled sample image without a determined virtual label in the unlabeled sample image, determining a virtual label of the selected unlabeled sample image according to the lighting feature of the selected unlabeled sample image.

[0028] In an optional embodiment, the step of determining the virtual label of the candidate unlabeled sample image according to the lighting feature of the candidate unlabeled sample image comprises:

[0029] acquiring a target feature characterized by a lighting degree consistent with the true label from the lighting feature of the candidate unlabeled sample image;

[0030] selecting a preset proportion of images from the candidate unlabeled sample image as target unlabeled sample images according to the value of the target feature of the candidate unlabeled sample image and a preset threshold, the virtual label of the target unlabeled sample image being a true label consistent with the lighting degree characterized by the target feature.

[0031] In an optional embodiment, the step of determining the virtual label of the candidate unlabeled sample image according to the lighting feature of the candidate unlabeled sample image further comprises:

[0032] if there are still remaining images in the unlabeled sample images for which the virtual labels have not been determined, all the remaining images are classified as one category, and a virtual label characterizing the category is set for all the remaining images.

[0033] In a second aspect, the present application provides an indoor lighting recognition device, the device comprising:

[0034] an acquisition module configured to acquire an indoor photo of a house to be identified;

[0035] an analysis module configured to divide the indoor photo into a plurality of grids;

[0036] the analysis module is further configured to analyze image features of the plurality of grids to determine a lighting feature characterizing a lighting degree of the house to be identified;

[0037] a recognition module configured to input the lighting feature into a pre-trained lighting recognition model to obtain a recognition result of the lighting degree of the house to be identified.

[0038] In a third aspect, the present application provides an electronic device comprising a processor and a memory, the memory being configured to store a program, and the processor being configured to implement the indoor lighting recognition method of any one of the preceding embodiments when executing the program.

[0039] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, the computer program being configured to implement the indoor lighting recognition method of any one of the preceding embodiments when executed by a processor.

[0040] The embodiment of the present application divides the indoor photo of the house to be identified into multiple grids, analyzes the image features of the multiple grids, determines the lighting features representing the lighting degree of the house to be identified, and then inputs the analyzed lighting features into the pre-trained lighting identification model to finally obtain the accurate identification result of the lighting degree of the house to be identified. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0042] Figure 1 The flowchart of the indoor lighting identification method provided by the embodiment of the present application is shown.

[0043] Figure 2 The comparative example diagram of the exposure adjustment before and after the underexposed picture provided by the embodiment of the present application is shown.

[0044] Figure 3 The example diagram of the division of different grid sizes provided by the embodiment of the present application is shown.

[0045] Figure 4 The example diagram of the heat map of the brightness map provided by the embodiment of the present application is shown.

[0046] Fig. 5(a) is an example diagram of the brightness value analysis of the grid under the average brightness dimension provided by the embodiment of the present application.

[0047] Fig. 5(b) is an example diagram of the brightness value analysis of the grid under the grid bright part ratio dimension provided by the embodiment of the present application.

[0048] Fig. 5(c) is an example diagram of the brightness value analysis of the grid under the dark part ratio dimension provided by the embodiment of the present application.

[0049] Figure 6 The example diagram of the feature dimension sorting result provided by the embodiment of the present application is shown.

[0050] Figure 7 The block diagram of the indoor lighting device provided by the embodiment of the present application is shown.

[0051] Figure 8 The block diagram of the electronic device provided by the embodiment of the present application is shown.

[0052] Icon: 10 - electronic device; 11 - processor; 12 - memory; 13 - bus; 100 - indoor lighting recognition device; 110 - acquisition module; 120 - analysis module; 130 - recognition module; 140 - training module. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0055] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0056] In the description of the present application, it should be noted that if the terms "upper", "lower", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the present application is usually placed, which is only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0057] In addition, if the terms "first", "second" and the like appear, they are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0058] It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0059] The degree of indoor lighting is an important dimension for judging the quality of a house, and accurate and objective evaluation of the indoor lighting of a house has a wide range of application scenarios in house engineering and related derivative fields. In house mortgage, the loan institution will evaluate the house of the house buyer, and calculating the house value is an important means to measure the value and quality of the house. House value, also known as house value, is an estimate of the value of a house based on factors such as structure, standard, location, material, area, and degree of newness.

[0060] In the calculation of house property valuation, the degree of indoor lighting is an important dimension to judge the quality of the house. Good lighting means that the house is bright, the light is a little brighter, and the value of the house will be higher. On the contrary, living in a poorly lit house will make people feel depressed, and the value of the house will be lower. Lighting will be affected by many dimensions, such as the orientation of the house, the floor-to-floor distance, the floor, the depth and the width, etc. At the same time, the shooting exposure, the weather, whether the light is on or not, and the curtain, etc. will affect the judgment of the house. However, good lighting or poor lighting is not so well defined, and different people may have their own criteria for lighting. Therefore, how to judge the degree of lighting through indoor photos has become a technical problem to be solved by those skilled in the art.

[0061] The degree of indoor lighting is usually identified by artificial judgment, and artificial judgment involves inconsistent judgment criteria and inconsistent understanding of the degree of lighting, which is difficult to define, resulting in inaccurate and subjective judgment of the degree of lighting.

[0062] Therefore, the embodiments of the present application provide an indoor lighting identification method, device, electronic equipment and storage medium. By dividing the indoor photo into a grid, analyzing the image features of the grid, obtaining a lighting feature that accurately represents the degree of lighting, and using the lighting feature to accurately identify the degree of lighting of the house using a lighting identification model, the lighting result provided by the embodiments of the present application can verify the authenticity of the information filled by the customer manager or the customer. On the other hand, the lighting identification result provided by the embodiments of the present application can also be used to preliminarily judge the indoor lighting condition. The following will be described in detail.

[0063] Please refer to Figure 1 , Figure 1 The flowchart of the indoor lighting identification method provided by the embodiments of the present application is shown in the figure. The method comprises the following steps:

[0064] Step S101, obtaining an indoor photo of a house to be identified.

[0065] In this embodiment, since the image exposure has a great influence on the identification of lighting, the indoor photo is a photo obtained by exposure correction processing on the original photo of the house to be identified according to the exposure quality of the original photo. For the original photo with good exposure quality, the original photo can be used as the indoor photo without processing. For the original photo with poor exposure quality, the original photo needs to be corrected, and the corrected original photo is used as the indoor photo.

[0066] In this embodiment, in the camera imaging, the wrong exposure in capturing the photo is still the main source of error, and overexposure and underexposure are brightness problems caused by improper image exposure. Specifically, overexposure means that the image is overexposed, that is, some areas in the image are too bright and exceed the range that the camera sensor can capture; and underexposure means that the image is underexposed, that is, some areas in the image are too dark and cannot show the real details. When shooting the lighting of a house, overexposure and underexposure may occur due to the shooting environment and camera settings, such as failure to properly control the camera exposure time in a strong light environment, etc. At this time, the captured picture is not real, but a "false lighting" judgment caused by shooting, and at this time, the picture needs to be corrected for exposure.

[0067] In image processing, the method of histogram equalization is generally used to process overexposed and underexposed images. Histogram equalization is a method of enhancing image contrast by stretching the pixel intensity distribution range. For example, a brighter image limits all pixels to a high value. However, a good image will have pixels from all areas of the image. Therefore, the histogram needs to be expanded to both ends. The basic principle of histogram equalization is to expand the gray value of the pixel with a large number in the image (i.e. the gray value that mainly affects the picture), and to merge the gray value of the pixel with a small number (i.e. the gray value that does not mainly affect the picture), thereby increasing the contrast, making the image clear, and achieving the purpose of enhancement. To achieve the effect of image histogram equalization, a distribution function mapping needs to be performed on the histogram of the original image, and this mapping function is the cumulative distribution function. In the histogram equalization process, the mapping method is:

[0068]

[0069] k = 0, 1, 2, …, L-1

[0070] where n is the total number of pixels in the image, k is the gray level of the pixel, that is, the gray level of the pixel in the image, n j is the number of pixels of the current gray level j, and L is the total number of possible gray levels in the image. It should be noted that histogram equalization is usually performed on a single channel image, and if it is a multi-channel image, equalization can be performed on each channel respectively.

[0071] Please refer to Figure 2 , Figure 2 the contrast example graph before and after the exposure adjustment of the underexposed picture provided by the embodiment of the application, Figure 2 the entire underexposed picture in the Figure 2 the picture after exposure correction can more truly reflect the lighting condition of the house.

[0072] In addition to the histogram equalization method, the exposure correction can also be performed by a neural network method. By training an artificial neural network, the function relationship between the brightness distribution law of each region in the picture and the image exposure value is learned to predict and adjust the image exposure value, thereby realizing the exposure correction of the image. A realization process is as follows: a group of original images and corresponding exposure values are input into a training set, the mapping relationship between the brightness distribution characteristics of different regions in the image and the exposure value is learned by an artificial neural network, and a model capable of performing exposure correction on different images is trained. The trained neural network is used to perform exposure correction on a new image, the new image is input into the trained neural network, the new exposure value is calculated by the neural network, and the image is subjected to corresponding exposure correction.

[0073] In step S102, the indoor photo is divided into a plurality of grids.

[0074] In the embodiment, on the one hand, in order to accurately analyze the image features of the indoor photo, so as to obtain the lighting features accurately reflecting the lighting degree, on the other hand, in order to improve the analysis efficiency, the indoor photo is divided into a plurality of grids, and the lighting features are obtained by analyzing the image features of the grids.

[0075] The indoor photo is divided into a plurality of grids, that is, the indoor photo is divided into a plurality of small grid regions, each small region has a specific width, height and brightness, and represents the pixels of the corresponding region in the indoor image.

[0076] Please refer to Figure 3 , Figure 3 The example diagrams of different grid sizes provided by the embodiment of the present application are shown in Figure 3 The original diagram, the diagram after 3*3 grid division, the diagram after 6*6 grid division and the diagram after 12*12 grid division are shown in

[0077] When the indoor photo is gridized, different grid sizes will have certain influence on the analysis result. Generally speaking, the smaller the grid is, the more detailed the information of the indoor photo is extracted, and the smaller range of brightness change and feature change can be detected; on the contrary, the larger the grid is, the wider the covered area is, and the more macro image features can be extracted. In the analysis of the lighting capacity of the house, different grid sizes will also lead to different analysis results. If a smaller grid size is used, the brightness and color distribution of each position in the room can be detected more finely, and the lighting effect of the window can be analyzed better. However, because the grid is relatively small, the calculation amount is large, and more time and computing resources are needed. On the contrary, a larger grid size can quickly obtain the overall brightness and color distribution, but some local details may be ignored. Therefore, when the gridization is performed in the training of the lighting recognition model, a suitable grid size can be selected according to the data set used for model training, so as to improve the calculation efficiency on the basis of ensuring the result accuracy. In addition, a more suitable grid size can be selected according to the actual application scenario, combined with the visual habits of people and the understanding of image features.

[0078] In step S103, the image features of the plurality of grids are analyzed to determine the lighting feature representing the lighting degree of the house to be identified.

[0079] In the embodiment, the analysis of the image features of the plurality of grids includes, but is not limited to, calculating the average brightness value of the pixels in different grids to understand the brightness of the entire picture, and using the feature extraction method to statistically analyze the brightness, color and other information of the pixels in different grids to derive the lighting feature representing the lighting degree of the house to be identified.

[0080] In step S104, the lighting feature is input into the pre-trained lighting recognition model to obtain the recognition result of the lighting degree of the house to be identified.

[0081] In the embodiment, the training method of the lighting recognition model can be run on the same device as the indoor lighting recognition method provided in the embodiment, the method for training the lighting recognition model is run on the device first, and then the indoor lighting recognition method is run, or the two can be run on different devices, which is not limited in the embodiment.

[0082] The above method provided in the embodiment can divide the indoor photo into grids, analyze the image features of the grids, obtain the lighting feature representing the lighting degree more accurately, and accurately identify the lighting degree of the house by using the lighting recognition model according to the lighting feature.

[0083] In an optional implementation, an implementation of determining the lighting feature representing the lighting degree of the house to be identified is:

[0084] First, the image features of the plurality of grids are statistically analyzed to obtain overall features of the indoor photo;

[0085] In this embodiment, the overall features include, but are not limited to, overall brightness of the indoor photo, ratio of width to height of the indoor photo, size of the indoor photo, contrast of the indoor photo, color tone, depth, overall bright part ratio, overall dark part ratio, and the like. Note that the brightness range is 0 to 255, 0 represents full black (dark), 255 represents full white (bright), the larger the number, the brighter the brightness, and the smaller the number, the darker the brightness. The bright part refers to a grid with a grid gray value greater than a set brightness threshold (for example, the brightness threshold is 180), and the dark part refers to a grid with a grid gray value less than a set darkness threshold (for example, the darkness threshold is 80).

[0086] Second, the brightness values of the plurality of grids are analyzed to obtain brightness features of the indoor photo;

[0087] In this embodiment, the analysis of the brightness values of the plurality of grids includes, but is not limited to, brightness distribution analysis, contrast analysis, brightness change analysis, and the like.

[0088] In this embodiment, an implementation manner of obtaining the brightness features of the indoor photo is as follows:

[0089] The brightness values of the plurality of grids are subjected to brightness distribution analysis to obtain grid features of the indoor photo;

[0090] The brightness values of the plurality of grids are subjected to contrast analysis to obtain contrast features of the indoor photo;

[0091] The brightness values of the plurality of grids are subjected to brightness change analysis to obtain change features of the indoor photo;

[0092] According to a plurality of grids with brightness values greater than a preset value and continuous positions, service features of the indoor photo are determined;

[0093] The grid features, the contrast features, the change features, and the service features are taken as lighting features.

[0094] In this embodiment, the grid features include, but are not limited to, average brightness of different grids, grid bright part ratio, brightest grid brightness, grid dark part ratio, brightest grid darkness, bright part average brightness, and dark part average brightness. The grid bright part ratio refers to a proportion of the number of bright part grids to the total number of grids, and the grid dark part ratio refers to a proportion of the number of dark part grids to the total number of grids.

[0095] In this embodiment, the contrast features include, but are not limited to, brightness average difference between the bright part and the dark part, relative distance between the brightest grid and the darkest grid, distance between the center point of the bright part and the center point of the dark part, and the like. That is, the bright part and the dark part are calculated first, and then the contrast degree and direction features of different regions are calculated.

[0096] In the embodiment, the variation features include, but are not limited to, calculating the variation degree of longitudinal / lateral brightness and dark part, the shape of the edge of the brightness and dark part region, whether the bright part is at the top end, and the like.

[0097] In the embodiment, the service features can be calculated by the size, orientation, color intensity, and the like of the continuous bright part region simulation window, the window area, distance, whether the curtain is partially blocked, whether other objects are blocked, and the like.

[0098] Finally, the overall features and the brightness features are taken as the lighting features.

[0099] In the embodiment, the lighting features that can accurately reflect the indoor lighting are obtained by analyzing the statistical rules of the image features of the indoor photos and analyzing the brightness from multiple dimensions.

[0100] In the embodiment, since the brightness value of the grid is involved in determining the brightness features, the embodiment further provides two implementation manners of calculating the brightness value:

[0101] Manner one: RGB weighted average method

[0102] The brightness value can be embodied by the gray value, and the calculated gray value of the picture can be used as the brightness value of the picture. The method of calculating the gray value of the picture is used (the range is 0-255), 0 represents full black (dark), 255 represents full white (bright), the larger the number, the brighter the brightness, and the smaller the number, the darker the brightness. The calculation of the gray value of the picture is a method of converting a color image into a gray image. The gray image only contains one channel, and the gray value of each pixel represents the brightness of the pixel in the image, the higher the value, the brighter the image, and the lower the value, the darker the image. There are multiple methods for calculating the gray value, one of which is to use the psychological gray weighting formula. When converting the RGB image into a gray image, the gray value can be calculated by the following formula:

[0103] Gray = R x 0.299 + G x 0.587 + B x 0.114

[0104] Wherein, R, G, and B represent the red, green, and blue channels of the pixel respectively, and 0.299, 0.587, and 0.114 are the weighting coefficients of the perceived brightness in psychology, representing the different perception degrees of the human eye to different colors. Please refer to Figure 4 , Figure 4 The example diagram of the heat map of the brightness map provided by the embodiment of the present application, Figure 4 shows the gray map (brightness map), the brightness map after 9x9 grid, and the heat map of different brightness.

[0105] In addition, there is also a method of directly taking the average value:

[0106] Gray = (R + G + B) / 3

[0107] This method is simple and easy to understand, but it ignores the psychological differences in the perception of different colors.

[0108] Method 2: V-channel method of HSV color space

[0109] The HSV color space is a commonly used color space for describing colors. Here, H represents hue, S represents saturation, and V represents brightness. By converting an image to the HSV color space and extracting the values ​​from the V channel, the brightness value of the image can be directly obtained. In practical applications, a suitable method can be chosen to calculate the brightness value of an image based on the specific requirements.

[0110] In an optional implementation, when training the daylight recognition model, sample images and corresponding labels representing the degree of daylight are required. Since judging the degree of daylight is inherently difficult, labeling the sample images as having excellent / poor daylight is a very time-consuming task. To improve the training efficiency of the daylight recognition model, this embodiment also provides an implementation method that uses a small number of labeled sample images to train the model:

[0111] First, acquire multiple sample images, including labeled and unlabeled sample images. The actual labels of the labeled sample images represent the lighting conditions of the labeled sample images.

[0112] In this embodiment, since the labeled sample image only occupies a small portion of the entire sample image, the actual labels on the labeled sample image can be manually applied. Because more extreme cases are easier to identify accurately, the actual labels can include both those with excellent and poor lighting conditions.

[0113] Secondly, virtual labels for unlabeled sample images are determined based on labeled sample images and their real labels.

[0114] In this embodiment, the virtual label can be the same as the real label. For example, the real label includes excellent lighting and poor lighting, and the virtual label also includes excellent lighting and poor lighting. The virtual label can also include more categories than the real label. For example, the virtual label includes excellent lighting, poor lighting and moderate lighting.

[0115] Finally, the daylight recognition model is trained using labeled sample images and their real labels, and unlabeled sample images and their virtual labels, to obtain the trained daylight recognition model.

[0116] In an optional embodiment, by determining the virtual labels of the unlabeled sample images according to the labeled sample images and their real labels, the labels of the sample images can be effectively expanded, and the trained lighting recognition model is more accurate. Based on the idea of semi-supervised learning, the existing small amount of labeled sample images are used in combination with the method of single variable abnormal region analysis to infer the good or bad situation of indoor lighting, and the label library is further supplemented by adding part of the labeled sample. Thus, by combining a small amount of labeled data with a large amount of unlabeled data, the effect of improving the label library is achieved, and the labels of part of the excellent / bad samples are added to further supplement the label library.

[0117] One implementation is to determine the initial selected unlabeled sample images belonging to the same category as the labeled sample images from the unlabeled sample images according to the labeled sample images and their real labels by using a label propagation algorithm; the real label of the labeled sample image is used as the virtual label of the initial selected unlabeled sample image belonging to the same type as the labeled sample image; if there are still candidate unlabeled sample images in the unlabeled sample images whose virtual labels are not determined, then the virtual label of the candidate unlabeled sample image is determined according to the lighting characteristics of the candidate unlabeled sample image.

[0118] In this embodiment, the label propagation algorithm is based on the idea of transferring labels from similar samples to neighboring samples, so as to infer the label information in the unlabeled data. Its formula can be expressed as:

[0119]

[0120] wherein, is the label prediction result of sample i, ω ij is the influence weight of sample j on sample i, y(i) is the real label value of sample j, and n is the total number of samples. In the label propagation algorithm, the influence weight ω ij is usually calculated based on the similarity between data. Common similarity calculation methods include distance measurement, kernel function and graph model, etc. One way to calculate the similarity is to use the Euclidean distance to calculate the influence weight respectively. The Euclidean distance formula is as follows:

[0121] wherein, d(p,q) is the distance between two points, q and p are n-dimensional vectors of two points, q i and p i are the i-th components of q and p respectively.

[0122] In the embodiment, the preliminary selected unlabeled sample images determined by the label propagation algorithm can include all the unlabeled sample images, that is, all the unlabeled sample images have their virtual labels determined by the label propagation algorithm, or can include only part of the unlabeled sample images, and the other part of the unlabeled sample images still cannot have their virtual labels determined. In some application scenarios, the determined virtual labels still cannot meet the requirements. At this time, the embodiment further provides another implementation manner for determining the virtual labels of the remaining unlabeled sample images:

[0123] acquiring a target feature representing a lighting degree consistent with the real label from the lighting feature of the to-be-selected unlabeled sample image;

[0124] selecting images of a preset proportion from the to-be-selected unlabeled sample images as target unlabeled sample images according to the value of the target feature of the to-be-selected unlabeled sample image and a preset threshold, and the virtual label of the target unlabeled sample image is the real label consistent with the lighting degree represented by the target feature;

[0125] if there are still remaining images in the unlabeled sample images that have not determined the virtual labels, all the remaining images are taken as a category, and the virtual labels representing the category are set for all the remaining images.

[0126] In the embodiment, the to-be-selected unlabeled sample image is an unlabeled sample image that still cannot determine its virtual label by the label propagation algorithm. The lighting feature includes, but is not limited to, a grid feature, a contrast feature, a change feature and a business feature. The target feature is one of the lighting features. In different scenarios, the target feature can be different features in the lighting features. For example, in scenario 1, the contrast feature can fully represent the lighting degree, and the contrast feature is the target feature. In scenario 2, the change feature can fully represent the lighting degree, and the change feature is the target feature.

[0127] Please refer to Figures 5(a) to 5(c) , FIG. 5(a) is an example diagram of the brightness value analysis of the grid in the average brightness dimension provided by the embodiment of the application, FIG. 5(b) is an example diagram of the brightness value analysis of the grid in the grid light part proportion dimension provided by the embodiment of the application, and FIG. 5(c) is an example diagram of the brightness value analysis of the grid in the dark part proportion dimension provided by the embodiment of the application. Figures 5(a) to 5(c)The combination chart of the histogram and the line chart of the brightness value of the grid under each dimension, wherein the abscissa of the histogram of each dimension is the binning result of the dimension feature, the ordinate Bin count distribution is the ordinate of the histogram of the dimension, indicating the distribution of good and bad samples under different bins, and the ordinate Bad probability is the ordinate of the line chart of the dimension, indicating the proportion of bad samples in the bin samples, for example, in FIG. 5(a), 331, 17.8% appears on the first histogram in [-inf, 60] in the first row average brightness, indicating that the number of samples in the bin is 331 and the proportion of the bin in the total number of samples is 17.8% when the average brightness is in [-inf, 60]; the value of the first point in the line chart [-inf, 60] in the first row average brightness is 99.4%, indicating that the bad sample rate in the bin sample is 99.4%, and the other histograms in FIG. 5(a) are similar to the first histogram, the meaning of the other points in the line chart is similar to that of the first point in the first row average brightness, and the meaning of the histogram in the other dimensions and the meaning of the point in the line chart are also similar to those in FIG. 5(a), except that the features represented are different. Note that, Figures 5(a) to 5(c) In the figure in which the lighting is poor, bad indicates poor lighting, and good indicates general lighting; in the figure in which the lighting is excellent, bad indicates excellent lighting, and good indicates general lighting.

[0128] In this embodiment, the lighting degree represented by the target feature can be excellent or extremely poor, the preset threshold corresponding to the lighting degree represented by the target feature is different, and the way of determining the target unlabeled sample image is also different, for example, from the target feature, the target unlabeled sample image is selected as the target feature value of the selected unlabeled sample image in a preset proportion.

[0129] It should be noted that the preset threshold is used to determine whether the feature value of the target feature is abnormal, if the feature value of the target feature exceeds the threshold, it is considered that the data point is in the abnormal region. The process of setting the threshold needs to select appropriate methods and parameters. Some commonly used methods and parameters are:

[0130] Residual method: assuming that the data obeys normal distribution, the upper and lower limits of the abnormal region can be calculated according to the mean and standard deviation of the data, and 3σ or 6σ is usually taken as the threshold.

[0131] Box plot: through the box plot, the statistical information such as the median, quartile, maximum and minimum of the data can be directly displayed, and the abnormal data points can also be detected and distinguished.

[0132] Histogram and density plot: the data distribution can be visualized, and the appropriate threshold can be selected according to the distribution of the data and domain knowledge, etc.

[0133] The preset threshold set by the above method can also be adjusted and optimized based on the actual distribution of the feature values of the target features and domain knowledge to fully reflect the abnormal situation of the feature values of the target features. At the same time, a visualization tool is used to help determine the threshold, and then identify and label the abnormal data points. At least one of the above three methods can be used, or a combination of multiple of the above three methods can be used to select the abnormal region in different target features as the label of the bad sample.

[0134] In the present embodiment, the lighting condition can be judged based on a preset machine learning method using a joint dimension. The joint dimension refers to combining multiple related feature dimensions (for example, the overall feature, the brightness feature, and the grid feature, the contrast feature, the change feature, and the business feature included in the brightness feature in the above embodiment) together as a judgment standard, thereby improving the accuracy and robustness of the model. One way is to use a random forest method instead of joint dimension classification. First, the feature importance needs to be sorted to determine the most important feature dimension. To further improve the accuracy of the model, the weather condition on the day, the shooting time, and other dimensions can be added for auxiliary judgment. For example, when considering the indoor lighting condition, the weather condition on the day can be considered as an auxiliary factor. If the weather is sunny, the indoor lighting condition is likely to be better; if the weather is gloomy or rainy, the indoor lighting condition is likely to be worse. In addition, factors such as shooting time can also be considered, such as the lighting condition in photos taken at certain specific times may be better. Finally, using these judgment standards, the user portrait is summarized, that is, the overall house of poor lighting is dark, the dark part accounts for a large proportion, and the light and dark change is obvious; the overall house of good lighting is bright, the bright part accounts for a large proportion, and the light and dark change is not obvious. A more accurate lighting recognition result is obtained. Please refer to Figure 6 , Figure 6 An example diagram of the feature dimension sorting result provided by the embodiment of the present application is shown in Figure 6 , which only shows the top 12 features. From top to bottom, they are overall brightness, proportion of dark part in overall pixels, average brightness, proportion of dark grid, proportion of bright part in overall pixels, brightness of the darkest grid after gridding, proportion of bright grid, average brightness of bright part and dark part, distance between bright part and dark part, proportion of bright part in overall pixels, variance of brightness in overall pixels, brightness difference between the brightest grid and the darkest grid after gridding.

[0135] In order to perform the corresponding steps in the above embodiments and various possible implementation manners, an implementation manner of an indoor lighting recognition device is given as follows. Please refer toFigure 7 , Figure 7 A block schematic diagram of the indoor lighting recognition device 100 provided by the embodiment of the present application is shown. It should be noted that the basic principle and the technical effects of the indoor lighting recognition device 100 provided by the embodiment are the same as those of the above-mentioned embodiments, and for brief description, part of the embodiment is not mentioned.

[0136] The indoor lighting recognition device 100 comprises an acquisition module 110, an analysis module 120, a recognition module 130 and a training module 140.

[0137] The acquisition module 110 is configured to acquire an indoor photo of a house to be recognized.

[0138] The analysis module 120 is configured to divide the indoor photo into a plurality of grids.

[0139] The analysis module 120 is further configured to analyze image features of the plurality of grids to determine lighting features representing a lighting degree of the house to be recognized.

[0140] The recognition module 130 is configured to input the lighting features into a pre-trained lighting recognition model to obtain a recognition result of the lighting degree of the house to be recognized.

[0141] In an optional implementation, the analysis module 120 is specifically configured to: statistically analyze the image features of the plurality of grids to obtain overall features of the indoor photo; analyze the luminance values of the plurality of grids to obtain luminance features of the indoor photo; and take the overall features and the luminance features as the lighting features.

[0142] In an optional implementation, when the analysis module 120 is configured to analyze the luminance values of the plurality of grids to obtain the luminance features of the indoor photo, the analysis module 120 is specifically configured to: perform luminance distribution analysis on the luminance values of the plurality of grids to obtain grid features of the indoor photo; perform contrast analysis on the luminance values of the plurality of grids to obtain contrast features of the indoor photo; perform luminance change analysis on the luminance values of the plurality of grids to obtain change features of the indoor photo; determine service features of the indoor photo according to a plurality of grids with luminance values greater than a preset value and continuous positions; and take the grid features, the contrast features, the change features and the service features as the lighting features.

[0143] In an optional implementation, the training module 140 is configured to: acquire a plurality of sample images, the sample images comprising labeled sample images and unlabeled sample images, a real label of the labeled sample image representing a lighting degree of the labeled sample image; determine a virtual label of the unlabeled sample image according to the labeled sample image and the real label thereof; and train the lighting recognition model by using the labeled sample image, the real label thereof, and the unlabeled sample image and the virtual label thereof to obtain the trained lighting recognition model.

[0144] In an optional implementation, the training module 140 is specifically configured to determine the virtual label of the unlabeled sample image according to the labeled sample image and the real label thereof, by using a label propagation algorithm to determine, from the unlabeled sample image, a preliminary selected unlabeled sample image belonging to the same category as the labeled sample image according to the labeled sample image and the real label thereof; and taking the real label of the labeled sample image as the virtual label of the preliminary selected unlabeled sample image belonging to the same category as the labeled sample image, and determining the virtual label of the selected unlabeled sample image according to the lighting feature of the selected unlabeled sample image.

[0145] In an optional implementation, the training module 140 is specifically configured to determine the virtual label of the selected unlabeled sample image according to the lighting feature of the selected unlabeled sample image, by obtaining, from the lighting feature of the selected unlabeled sample image, a target feature representing a lighting degree consistent with the real label; and selecting, from the selected unlabeled sample image, a preset proportion of images as target unlabeled sample images according to the value of the target feature of the selected unlabeled sample image and a preset threshold, the virtual label of the target unlabeled sample image being the real label consistent with the lighting degree represented by the target feature.

[0146] In an optional implementation, the training module 140 is further configured to determine the virtual label of the selected unlabeled sample image according to the lighting feature of the selected unlabeled sample image, by taking all the remaining images as a category if there are still remaining images in the unlabeled sample image that have not yet determined the virtual label, and setting a virtual label representing the category for all the remaining images.

[0147] The embodiment also provides a block schematic diagram of an electronic device 10, which will be described below with reference to Figure 8 , Figure 8 The block schematic diagram of the electronic device 10 provided by the embodiment of the present application is shown in FIG. 1. The electronic device 10 includes a processor 11, a memory 12 and a bus 13. The processor 11 and the memory 12 are connected through the bus 13.

[0148] The processor 11 can be an integrated circuit chip with a signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 11. The processor 11 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP) and the like; or can be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0149] The memory 12 is configured to store programs, for example, the program of the indoor lighting recognition device 100 in the computer 10. Figure 7 The indoor lighting recognition device 100 in the computer 10 each comprises at least one software function module stored in the memory 12 in the form of software or firmware, and the processor 11 executes the program to implement the indoor lighting recognition method in the embodiments of the present application after receiving an execution instruction.

[0150] The memory 12 can comprise a high-speed random access memory (RAM) and can also comprise a non-volatile memory. Optionally, the memory 12 can be a storage device built in the processor 11 or a storage device independent of the processor 11.

[0151] The bus 13 can be an ISA bus, a PCI bus, an EISA bus, etc. Figure 8 Only one bidirectional arrow is used for representation, but it does not mean that there is only one bus or only one type of bus.

[0152] The embodiments of the present application provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the indoor lighting recognition method in the foregoing embodiments.

[0153] In summary, the embodiment of the present application provides an indoor lighting recognition method, device, electronic equipment and storage medium, the method comprising: obtaining an indoor photo of a house to be identified; dividing the indoor photo into multiple grids; analyzing the image features of the multiple grids to determine the lighting features representing the lighting degree of the house to be identified; inputting the lighting features into a pre-trained lighting recognition model to obtain the recognition result of the lighting degree of the house to be identified. Compared with the prior art, the embodiment of the present application has at least the following advantages: by dividing the grid and analyzing the image features of the multiple grids, the lighting features representing the lighting degree of the house to be identified are accurately determined, and the accuracy of indoor lighting recognition is improved; the average brightness value, color information and brightness change of the grid and other information are helpful to deeply understand the features and statistical rules of the image, and further analyze the performance of the image in the lighting capacity of the house; based on the analysis result of the features, the influence of factors such as the size, position and orientation of the window on the lighting capacity of the house can be simulated, so that more reliable improvement suggestions are provided; by using a small amount of labeled data and a large amount of unlabeled data, through a semi-supervised manner, combining the target features representing the lighting degree consistent with the real label and the preset threshold, the good and bad conditions of indoor lighting are inferred, the virtual label of the unlabeled sample is determined, and the label library is supplemented by adding part of the labeled sample; based on the lighting features and recognition results analyzed from the image, the user portrait is summarized, that is, the house with poor lighting is dark overall, the proportion of dark part is large, and the light and dark change is obvious; the house with good lighting is bright overall, the proportion of bright part is large, and the light and dark change is not obvious, which provides more effective reference for the use value judgment of the house; it has high flexibility and scalability, and the lighting recognition model can be adjusted and optimized according to different sample image data sets and application scenarios.

[0154] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for identifying indoor lighting, characterized in that, The method includes: Obtain interior photos of the house to be identified; The indoor photograph is divided into multiple grids; The image features of the multiple grids are analyzed to determine the lighting features that characterize the lighting level of the house to be identified; The lighting characteristics are input into a pre-trained lighting recognition model to obtain the recognition result of the lighting level of the house to be identified; The training process of the light-collecting recognition model is as follows: Multiple sample images are acquired, including labeled sample images and unlabeled sample images. The real labels of the labeled sample images represent the light intensity of the labeled sample images. Based on the labeled sample images and their real labels, a label propagation algorithm is used to determine the initial unlabeled sample images that belong to the same category as the labeled sample images from the unlabeled sample images. The real labels of the labeled sample images are used as virtual labels for the initial unlabeled sample images of the same type. If there are candidate unlabeled sample images among the unlabeled sample images for which no virtual label has been determined, then the target feature representing the lighting level consistent with the real label is obtained from the lighting features of the candidate unlabeled sample images; based on the value of the target feature of the candidate unlabeled sample images and a preset threshold, a preset proportion of images are selected from the candidate unlabeled sample images as target unlabeled sample images, and the virtual label of the target unlabeled sample image is the real label consistent with the lighting level represented by the target feature; The daylight recognition model is trained using the labeled sample images and their real labels, and the unlabeled sample images and their virtual labels, to obtain the trained daylight recognition model.

2. The indoor lighting recognition method as described in claim 1, characterized in that, The step of analyzing the image features of the multiple grids to determine the lighting features characterizing the lighting level of the house to be identified includes: Statistical analysis is performed on the image features of the multiple grids to obtain the overall features of the indoor photograph; The brightness values ​​of the multiple grids are analyzed to obtain the brightness characteristics of the indoor photograph; The overall features and the brightness features are used as the light-collecting features.

3. The indoor lighting recognition method as described in claim 2, characterized in that, The step of analyzing the brightness values ​​of the multiple grids to obtain the brightness characteristics of the indoor photograph includes: Brightness distribution analysis is performed on the brightness values ​​of the multiple grids to obtain the grid features of the indoor photograph; Contrast analysis is performed on the brightness values ​​of the multiple grids to obtain the contrast characteristics of the indoor photograph; Brightness variation analysis is performed on the brightness values ​​of the multiple grids to obtain the variation characteristics of the indoor photograph; The business characteristics of the indoor photos are determined based on multiple grids whose brightness values ​​are greater than a preset value and whose positions are consecutive. The grid feature, the contrast feature, the variation feature, and the service feature are used as the lighting feature.

4. The indoor lighting recognition method as described in claim 1, characterized in that, The method further includes: If there are remaining images among the unlabeled sample images that have not yet been assigned virtual labels, then all the remaining images are treated as one category, and virtual labels representing that category are set for all the remaining images.

5. An indoor lighting recognition device, characterized in that, The device includes: The acquisition module is used to acquire interior photos of the house to be identified; The analysis module is used to divide the indoor photographs into multiple grids; The analysis module is also used to analyze the image features of the multiple grids to determine the lighting features that characterize the lighting level of the house to be identified. The identification module is used to input the lighting characteristics into a pre-trained lighting identification model to obtain the identification result of the lighting level of the house to be identified; The training module acquires multiple sample images, including labeled and unlabeled sample images. The real labels of the labeled sample images represent their illumination levels. Based on the labeled sample images and their real labels, a label propagation algorithm is used to identify initial unlabeled sample images belonging to the same category as the labeled sample images from the unlabeled sample images. The real labels of the labeled sample images are used as virtual labels for the initial unlabeled sample images belonging to the same category. If there are still unlabeled sample images among the unlabeled sample images for which virtual labels have not been determined... In this image, the target feature representing the lighting level consistent with the real label is obtained from the lighting features of the candidate unlabeled sample images; based on the value of the target feature of the candidate unlabeled sample images and a preset threshold, a preset proportion of images are selected from the candidate unlabeled sample images as target unlabeled sample images, and the virtual label of the target unlabeled sample image is the real label consistent with the lighting level represented by the target feature; the lighting recognition model is trained using the labeled sample images and their real labels and the unlabeled sample images and their virtual labels to obtain the trained lighting recognition model.

6. An electronic device, characterized in that, It includes a processor and a memory, the memory being used to store a program, and the processor being used to implement the indoor lighting recognition method according to any one of claims 1-4 when executing the program.

7. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the indoor lighting recognition method according to any one of claims 1-4.

Images