An image processing method, apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to an image processing method, device, electronic equipment and storage medium, the image processing method comprising: acquiring a target image, processing the light scene of the target image through a pre-constructed mutation model to obtain a first classification result and a mutation value corresponding to the light sensed when the camera device shoots the target image, and a pre-trained neural network model also identifies the light of the target image to obtain a second classification result and a probability value corresponding to the target light, and then determining the classification result of the light in the target image according to the output results of the two models. The present disclosure is based on two pre-constructed models to identify the classification result of the light in the target image, determine the type of light according to the two classification results, has a relatively high classification accuracy, can accurately identify the light type in an image with high complexity, and is convenient for subsequent processing based on the classification result.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to an image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] During the driving of a vehicle, the image captured by the camera in the vehicle is easily affected by natural light and low light, and accurate identification of the light in the image is an important prerequisite for solving the problem of image light.

[0003] However, in the light environment, the video content captured by the camera is complex, for example, including surrounding environment, road and light, and the like, and the scene is rich, so that the type of light in the image cannot be accurately identified.

[0004] At present, when the light in the image is identified by using the prior art, the classification accuracy is low due to the influence of the richness of prior information, so that the related functions applied to the classification result are limited. SUMMARY

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides an image processing method, device, electronic equipment and storage medium.

[0006] In a first aspect, the embodiments of the present disclosure provide an image processing method, comprising:

[0007] obtaining a target image;

[0008] processing the target image by using a pre-trained mutation model to obtain a first classification result corresponding to the light sensed when the target image is captured and a target mutation value;

[0009] identifying the target image by using a pre-trained neural network model to obtain a second classification result corresponding to the light and a probability value of the light being a target light;

[0010] determining a target classification result of the light according to the first classification result, the target mutation value, the second classification result and the probability value.

[0011] Optionally, processing the target image by using the pre-trained mutation model to obtain the first classification result corresponding to the light sensed when the target image is captured and the target mutation value comprises:

[0012] performing gray processing on the target image, and determining a third classification result corresponding to the light sensed when the target image is captured according to the gray processing result;

[0013] dividing the target image into a light source candidate region and a reflection candidate region according to the third classification result;

[0014] calculate a first mutation value of the light source candidate region, and determine a light source sub-region in the light source candidate region according to the first mutation value;

[0015] obtain a first classification result corresponding to the light and a target mutation value according to the light source sub-region and the reflection candidate region.

[0016] Optionally, the gray scale processing result includes a gray scale mean value of the target image.

[0017] determine a third classification result corresponding to the light sensed when the target image is captured according to the gray scale processing result, including:

[0018] determine the third classification result corresponding to the light sensed when the target image is captured according to the gray scale mean value of the target image and a first preset threshold.

[0019] Optionally, the determining the third classification result corresponding to the light sensed when the target image is captured according to the gray scale mean value of the target image and the first preset threshold includes:

[0020] if the gray scale mean value of the target image is greater than the first preset threshold, determine that the third classification result is natural light;

[0021] if the gray scale mean value of the target image is less than or equal to the first preset threshold, determine that the third classification result is low light.

[0022] Optionally, the calculating the first mutation value of the light source candidate region and the determining the light source sub-region in the light source candidate region according to the first mutation value include:

[0023] divide the light source candidate region into a plurality of first sub-regions;

[0024] calculate a gray scale mean value of each first sub-region in the plurality of first sub-regions;

[0025] calculate a first mutation value corresponding to each first sub-region according to the gray scale mean value of each first sub-region and position information of each first sub-region in the target image;

[0026] take a first sub-region with a first mutation value greater than a second preset threshold in the plurality of first sub-regions as the light source sub-region.

[0027] Optionally, the obtaining the first classification result corresponding to the light and the target mutation value according to the light source sub-region and the reflection candidate region includes:

[0028] divide the reflection candidate region into a plurality of second sub-regions;

[0029] calculate a second mutation value according to the plurality of second sub-regions and the light source sub-region;

[0030] According to the second mutation value and a third preset threshold, a first classification result corresponding to the illumination and the target mutation value are determined.

[0031] Optionally, the second mutation value is calculated according to the plurality of second sub-regions and the light source sub-region, including:

[0032] According to position information of the light source sub-region in the target image, position information of each second sub-region in the target image is determined.

[0033] According to the position information of each second sub-region in the target image, a gray mean value of each second sub-region is calculated, and the second mutation value is calculated based on the gray mean value of each second sub-region.

[0034] Optionally, the target mutation value is the second mutation value.

[0035] According to the second mutation value and a third preset threshold, a first classification result corresponding to the illumination is determined, including:

[0036] If the third classification result is natural light, and the second mutation value is less than or equal to the third preset threshold, the first classification result corresponding to the illumination is determined as natural light.

[0037] If the third classification result is natural light, and the second mutation value is greater than the third preset threshold, the first classification result corresponding to the illumination is determined as backlight.

[0038] If the third classification result is low light, and the second mutation value is less than or equal to the third preset threshold, the first classification result corresponding to the illumination is determined as low light.

[0039] If the third classification result is low light, and the second mutation value is greater than the third preset threshold, the first classification result corresponding to the illumination is determined as lamp light.

[0040] Optionally, according to the first classification result, the target mutation value, the second classification result and a probability value, a target classification result of the illumination is determined, including:

[0041] A sum value of the target mutation value corresponding to the first classification result and the probability value corresponding to the first classification result is calculated.

[0042] If the probability value corresponding to other classification results in the second classification result except the first classification result is greater than the sum value, the other classification result with the maximum probability value is taken as the target classification result of the illumination.

[0043] If the probability value corresponding to other classification results in the second classification result except the first classification result is less than the sum value, the first classification result is taken as the target classification result of the illumination.

[0044] In a second aspect, the embodiments of the present disclosure provide an image processing device, including:

[0045] an acquisition unit configured to acquire a target image;

[0046] a first identification unit configured to process the target image by using a pre-trained mutation model to obtain a first classification result corresponding to the light sensed when the target image is captured and a target mutation value;

[0047] a second identification unit configured to identify the target image by using a pre-trained neural network model to obtain a second classification result corresponding to the light and a probability value of the light being target light;

[0048] a determination unit configured to determine a classification result of the light according to the first classification result, the target mutation value, the second classification result, and the probability value.

[0049] In a third aspect, an electronic device is provided, including:

[0050] a memory;

[0051] a processor; and

[0052] a computer program;

[0053] The computer program is stored in the memory and is configured to be executed by the processor to implement the image processing method as described above.

[0054] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the image processing method as described above.

[0055] The present disclosure relates to an image processing method, device, electronic device, and storage medium. The image processing method includes: acquiring a target image, processing a light scene of the target image by using a pre-constructed mutation model to obtain a first classification result corresponding to the light sensed when the target image is captured by a camera device and a mutation value, simultaneously identifying the light of the target image by using a pre-trained neural network model to obtain a second classification result and a probability value corresponding to target light, and then determining a classification result of the light in the target image according to output results of the two models. The present disclosure identifies the classification result of the light in the target image based on two models constructed from different angles, determines the type of the light according to the two classification results, has a relatively high classification accuracy, can accurately identify the type of the light in an image with a relatively high complexity, and is convenient for subsequent processing based on the classification result. BRIEF DESCRIPTION OF DRAWINGS

[0056] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the disclosure.

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0058] Figure 1 A schematic diagram of an application scenario provided by the embodiments of the present disclosure;

[0059] Figure 2 A flowchart of an image processing method provided by the embodiments of the present disclosure;

[0060] Figure 3 A structural schematic diagram of an image processing method provided by the embodiments of the present disclosure;

[0061] Figure 4 A flowchart of an image processing method provided by the embodiments of the present disclosure;

[0062] Figure 5 A flowchart of an image processing method provided by the embodiments of the present disclosure;

[0063] Figure 6 A flowchart of an image processing method provided by the embodiments of the present disclosure;

[0064] Figure 7 A flowchart of an image processing method provided by the embodiments of the present disclosure;

[0065] Figure 8 A structural schematic diagram of an image processing device provided by the embodiments of the present disclosure;

[0066] Figure 9 A structural schematic diagram of an electronic device provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0068] In the following description, many specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some of the embodiments of the present disclosure, not all the embodiments.

[0069] This disclosure provides an image processing method that acquires a target image, processes the illumination in the target image based on a grayscale model constructed from the perspective of grayscale abrupt changes, and obtains a first classification result and a target abrupt change value. The illumination in the target image is the illumination sensed by the imaging device when capturing the target image. Simultaneously, the illumination in the target image is identified based on a neural network model constructed from the perspective of deep learning, and a second classification result and a probability value that the illumination is the target light are obtained. Then, the two classification results are combined to obtain the final classification result of the illumination. The classification accuracy is relatively high, and it is suitable for illumination recognition of images with high complexity. It also facilitates subsequent image processing based on the illumination classification results.

[0070] Specifically, the image processing method can be executed by a terminal or a server. Specifically, the terminal or server can use a neural network model and a mutation model to identify the illumination in the target image, and determine the classification result of the illumination in the target image based on the two identification results. The entity executing the training method of the neural network model and the mutation model can be the same as or different from the entity executing the image processing method.

[0071] For example, in one application scenario, such as Figure 1 As shown, server 12 trains the neural network model and the mutation model. Terminal 11 obtains the trained neural network model and mutation model from server 12. Terminal 11 uses the trained neural network model and mutation model to identify the lighting type in the target image, and determines the lighting type in the target image based on the two identification results. The target image can be captured by terminal 11. Alternatively, the target image can be obtained by terminal 11 from other devices. Or, the target image can be obtained by terminal 11 after image processing of a preset image, which can be captured by terminal 11 or obtained by terminal 11 from other devices. Here, no specific limitation is made to the other devices.

[0072] In another application scenario, server 12 trains a neural network model and a mutation model. Further, server 12 uses the trained neural network model and mutation model to identify the type of illumination in the target image, and determines the type of illumination in the target image based on the two identification results. The method by which server 12 acquires the target image can be similar to the method by which terminal 11 acquires the target image as described above, and will not be repeated here.

[0073] In another application scenario, the terminal 11 trains the neural network model and the mutation model. Further, the terminal 11 identifies the type of light in the target image through the trained neural network model and the mutation model respectively, and determines the type of light in the target image according to the two identification results. It can be understood that the application scenarios of the present disclosure are not limited to the above several possibilities.

[0074] The following takes the server 12 training the neural network model and the mutation model and determining the light scene of the target image as an example, and introduces an image processing method through one or more of the following embodiments. It can be understood that the image processing method is also applicable to the application scenario of the terminal 11.

[0075] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure is shown in the figure, which is applied to a server and specifically includes the following steps S210 to S240 as shown in the figure: Figure 2

[0076] S210, obtaining a target image.

[0077] It can be understood that the server obtains a target image, which can be generated by a shooting device installed on a vehicle or generated by other shooting devices and transmitted to the server. It can be understood that the shooting device uses the straight-line propagation property of light and the refraction and reflection law of light to use photons as carriers to transfer the light information of the photographed object at a certain moment to the photosensitive material through the shooting lens in the form of energy to generate a visible image. The shooting device installed on the vehicle can be understood as a target image shot in the case of sensing light, and the generated target image will contain information related to light.

[0078] Optionally, the step of obtaining the target image specifically includes: obtaining video data recorded by the shooting device; and performing frame extraction processing on the video data according to a preset frame extraction frequency to determine the target image.

[0079] It can be understood that the video data recorded by the shooting device installed on the vehicle can be directly obtained, and the video data is frame extracted according to a preset frame extraction frequency, and the image after frame extraction is taken as the target image. Subsequently, the type of light in each frame of image obtained after frame extraction can be identified through the image processing method provided by the present disclosure, and then the type of light in the video data, that is, the type of light sensed by the shooting device or the video data shot by the shooting device in which light scene, is determined. The video data can be extracted at a frame rate of 5 seconds to generate a target image, and the frame extraction frequency can be set to 0.2.

[0080] It can be understood that the formula for frame extraction of the video data is shown in the following formula (1).

[0081] ​[I1, I2,..., I n ] = F(video, fps, resize(weight, height)) (1)

[0082] Wherein, [I1, I2,..., I n ] is the target image after frame extraction, video represents video data, fps is the frame extraction frequency, and resize(weight, height) is the size of the target image after frame extraction.

[0083] S220, processing the target image through the pre-trained mutation model to obtain a first classification result corresponding to the light sensed when the target image is captured and a target mutation value.

[0084] It can be understood that on the basis of the above S210, the target image is processed and classified through the pre-trained mutation model to obtain the first classification result corresponding to the light sensed by the shooting device when the shooting device captures the target image and the target mutation value. The first classification result is also the type of light in the target image, which can be one of low light, natural light, backlight and light. That is, the mutation model only outputs one type of light and the target mutation value corresponding to the light type.

[0085] It can be understood that the output of the mutation model can be represented by the following formula (2).

[0086] Φ a (I) = (tag_a, a) (2)

[0087] Wherein, I represents the data of the target image, Φ a (I) represents the output of the mutation model, tag_a represents the first classification result, and a represents the target mutation value.

[0088] S230, identifying the target image through the pre-trained neural network model to obtain a second classification result corresponding to the light and a probability value of the light being the target light.

[0089] It can be understood that, on the basis of S210, the target image is recognized by the neural network model trained in advance to obtain the second classification result corresponding to the light in the target image and the probability value of the light being the target light; the neural network model can be constructed based on a convolutional neural network, which is a model capable of automatically identifying the light type in the target image, and the training sample image can be an image captured by the shooting device under different light scenes, and the specific training process is not described herein; the second classification result can include natural light, low light, backlight and light, and the probability value of the light being the target light refers to the probability value of the light being natural light, low light, backlight and light, that is, the probability value of identifying the light as the above four types, and the second classification result and the related probability value can be directly output by the neural network model, that is, the neural network model will output four types of light and the probability value corresponding to each type of light.

[0090] It can be understood that the output of the neural network model can be represented as formula (3) as follows.

[0091] Φ d (I)=(tag_d,w s ) (3)

[0092] Wherein, I represents the data of the target image, Φ d (I) represents the output of the neural network model, tag_d represents the second classification result, w s represents the probability value corresponding to the target light.

[0093] S240, according to the first classification result, the target mutation value, the second classification result and the probability value, determining the target classification result of the light.

[0094] It can be understood that, on the basis of S220 and S230, according to the first classification result and the target mutation value output by the mutation model and the second classification result and the probability value output by the neural network model, the final classification result of the light in the target image is determined.

[0095] Optionally, the specific steps of determining the target classification result include: calculating the sum of the target mutation value corresponding to the first classification result and the probability value corresponding to the first classification result; if the probability value corresponding to the other classification result in the second classification result except the first classification result is greater than the sum, the other classification result with the maximum probability value is taken as the target classification result of the light; if the probability value corresponding to the other classification result in the second classification result except the first classification result is less than the sum, the first classification result is taken as the target classification result of the light.

[0096] It can be understood that first, the sum value between the target mutation value corresponding to the first classification result and the probability value corresponding to the first classification result is calculated, and the probability value corresponding to the first classification result refers to the probability value corresponding to the classification result same as the first classification result in the second classification result, for example, the first classification result is low light, and the sum value between the probability value corresponding to low light in the second classification result and the target mutation value corresponding to low light is calculated. After calculating the sum value, it is judged whether the probability value corresponding to the classification result other than the first classification result in the second classification result is greater than the sum value, that is, the size of the calculated sum value and the probability value corresponding to the other classification result is judged, if yes, the classification result with the largest probability value is taken as the target classification result of the light; if not, the first classification result is taken as the target classification result of the light.

[0097] It can be understood that the formula for determining the recognition result is shown in the following formula (4).

[0098]

[0099] Wherein, G(I) represents the recognition result, tag is the target classification result, w represents the maximum value between the sum value and the probability value, represents the sum value and the maximum value.

[0100] For example, the first classification result is low light, the target mutation value is 0.2, the probability value of low light in the second classification result is 0.6, the probability value of natural light is 0.1, the probability value of backlight is 0.1, and the probability value of light is 0.2, then the sum value of low light is 0.8, and the sum value 0.8 is greater than the probability value of other classification results, and it is determined that the first classification result low light is the final classification result of the light in the target image.

[0101] Optionally, the video data is frame extracted to obtain a plurality of target images; the type of light in each target image is recognized to obtain a target classification result and a value (probability value or sum value); the sum of the values corresponding to the same classification result in the plurality of target images is calculated to obtain a total sum; if the total sum is greater than a target threshold, the type of light corresponding to the video data is determined as the classification result corresponding to the total sum.

[0102] It can be understood that the recognition result of the type of light in the video data is shown in the following formula (5).

[0103]

[0104] Wherein, VideoLight represents the recognition result of the video data, I i represents the i-th target image, G(I i ) represents the recognition result of the i-th target image, and i ranges from (1, n-z), n is the number of target images obtained by frame extraction of the video data, and z is a fixed value.

[0105] As can be understood, referring to Figure 3 , Figure 3 A structural schematic diagram of an image processing method provided by an embodiment of the present disclosure is provided. Video data captured by shooting is obtained. The video data is frame-extracted to obtain a plurality of target images. Each target image is input into a mutation model and a neural network model respectively to obtain a first classification result and a target mutation value output by the mutation model and a second classification result and a probability value output by the neural network model. A target classification result and a value of the target image are determined according to the output results of the two models. The value refers to the probability value or the sum value. Finally, a classification result of the video data is determined according to the target classification results and values of the plurality of target images.

[0106] An image processing method provided by an embodiment of the present disclosure is provided. A target image is obtained. Illumination in the target image is processed based on a gray scale model constructed at a gray scale mutation angle to obtain a first classification result and a target mutation value. The illumination in the target image is also the illumination sensed when a shooting device shoots the target image. Meanwhile, the illumination in the target image is identified based on a neural network model constructed at a deep learning angle to obtain a second classification result and a probability value of the illumination being target light. Then, the two classification results are combined to obtain a classification result of the illumination. The classification accuracy is relatively high. The method is suitable for identifying illumination of an image with high complexity. Meanwhile, the method is convenient for subsequent processing of the image based on the classification result of the illumination.

[0107] On the basis of the above-described embodiments, Figure 4 A flowchart of an illumination scene classification method provided by an embodiment of the present disclosure is provided. Optionally, a target image is processed by a pre-trained mutation model to obtain a first classification result and a target mutation value corresponding to the illumination sensed when the target image is shot. The method specifically includes the following steps S410 to S440 as shown in Figure 4

[0108] S410, performing gray scale processing on the target image and determining a third classification result corresponding to the illumination sensed when the target image is shot according to a gray scale processing result.

[0109] Optionally, the gray scale processing result includes a gray scale mean value of the target image. The third classification result corresponding to the illumination sensed when the target image is shot is determined according to the gray scale mean value of the target image and a first preset threshold value.

[0110] Optionally, the third classification result corresponding to the illumination is determined according to the gray scale mean value of the target image and the first preset threshold value. Specifically, if the gray scale mean value of the target image is greater than the first preset threshold value, the third classification result is determined to be natural light. If the gray scale mean value of the target image is less than or equal to the first preset threshold value, the third classification result is determined to be low light. ​

[0111] It can be understood that the target image is grayed to obtain a gray image, and then the gray mean value and the gray dispersion of the grayed target image are calculated, and according to the gray mean value and the first preset threshold, a third classification result corresponding to the light sensed by the shooting device when the shooting device shoots the target image is determined, and the third classification result includes natural light and low light, that is, the light is preliminarily classified according to the gray mean value and the first preset threshold. If the gray mean value of the target image is greater than the first preset threshold, it is determined that the third classification result is natural light, that is, the light in the target image is natural light; if the gray mean value of the target image is less than or equal to the first preset threshold, it is determined that the third classification result is low light, that is, the light in the target image is low light. The first preset threshold can be determined according to actual needs. The gray dispersion can be used to assist in verifying the classification result after the third classification result is determined.

[0112] It can be understood that the formula for calculating the gray mean value of the target image is shown in the following formula (6).

[0113]

[0114] Wherein, m represents the gray mean value of the target image, i represents the gray value, H(i) represents the number of pixels with gray value i after image graying, and s represents the size of the target image.

[0115] S420, according to the third classification result, the target image is divided into a light source candidate area and a reflection candidate area.

[0116] It can be understood that on the basis of the above S410, according to the third classification result (natural light or low light), the target image is divided into a light source candidate area and a reflection candidate area, and the light source candidate area can be the upper half of the target image, that is, half of the size of the target image, and the reflection candidate area can be the lower half of the image.

[0117] S430, calculating the first mutation value of the light source candidate area, and determining the light source sub-area in the light source candidate area according to the first mutation value.

[0118] It can be understood that on the basis of the above S420, the first mutation value of the divided light source candidate area is calculated, and the light source sub-area in the light source candidate area is determined according to the first mutation value. The light source sub-area is in the light source candidate area, that is, the position of the light source in the light source candidate area is determined. The light source refers to the source of light, and the source of light and the area close to the source of light in the target image have relatively high brightness in the target image.

[0119] S440, according to the light source sub-area and the reflection candidate area, a first classification result corresponding to the light and a target mutation value are obtained.

[0120] It can be understood that, on the basis of the above S430, according to the determined light source sub-region and the reflection candidate region, the first classification result corresponding to the light in the target image and the target mutation value are obtained, that is, after determining the position of the light source, the specific light type is judged by combining the light source reflection candidate region.

[0121] It can be understood that, after the preliminary classification in the above S410, the result of the preliminary classification is further classified to determine the specific type of the light in the target image, for example, the third classification result determined by S410 is natural light, and the subsequent S420 to S440 continue to judge whether the first classification result is natural light or backlight; if the third classification result determined by S410 is low light, the subsequent S420 to S440 continue to judge whether the first classification result is low light or lamp light.

[0122] The image processing method provided by the embodiment of the present disclosure preliminarily determines the third classification result of the light in the target image by calculating the gray mean value of the target image, and then divides the target image according to the third classification result to obtain the light source candidate region and the reflection candidate region, and then determines the light source sub-region according to the light source candidate region and determines the first classification result according to the light source sub-region and the reflection candidate region. By dividing the region, determining the light source sub-region and the reflection candidate region, the type of the light in the target image can be accurately determined, the classification accuracy is relatively high, and the method is easy to implement and does not need a large amount of training.

[0123] On the basis of the above embodiment, Figure 5 The flowchart of the light scene classification method provided by the embodiment of the present disclosure, optionally, the first mutation value of the light source candidate region is calculated, and the light source sub-region in the light source candidate region is determined according to the first mutation value, and the determination of the light source sub-region specifically includes the following steps S510 to S540 as shown in the figure: Figure 5

[0124] S510, divide the light source candidate region into a plurality of first sub-regions.

[0125] It can be understood that the preset number of the first sub-regions is determined, that is, it is determined in advance that the light source candidate region is divided into how many first sub-regions, and then the light source candidate region is divided according to the preset number to obtain a predetermined number of first sub-regions.

[0126] S520, calculate the gray mean value of each first sub-region in the plurality of first sub-regions.

[0127] It can be understood that, on the basis of the above S510, the gray mean value of each first sub-region in the preset number of first sub-regions is calculated, and the calculation method of the gray mean value can refer to the above formula (6), which will not be repeated here.

[0128] ​S530, according to the gray mean value of each first sub-region and the position information of each first sub-region in the target image, a first mutation value corresponding to each first sub-region is calculated.

[0129] It can be understood that, on the basis of S520, while dividing the plurality of first sub-regions, the position information of each first sub-region in the light source candidate region also needs to be recorded, and the position information can be the row and column information of the first sub-region in the light source candidate region. Then, according to the gray mean value of each first sub-region and the position information of each first sub-region in the target image, a first mutation value corresponding to each first sub-region is calculated, that is, each first sub-region corresponds to a first mutation value.

[0130] It can be understood that the specific formula for calculating the first mutation value of the first sub-region is shown in the following formula (7).

[0131]

[0132] wherein a light (x,y) represents the first mutation value of the first sub-region, m(x,y) represents the gray mean value of the first sub-region, m(i,j) represents the gray mean value of the other sub-regions adjacent to the first sub-region, and (x,y) represents the position of the first sub-region in the light source candidate region.

[0133] S540, the first sub-region in which the first mutation value is greater than the second preset threshold in the plurality of first sub-regions is taken as a light source sub-region.

[0134] It can be understood that, on the basis of S530, the first sub-region in which the first mutation value is greater than the second preset threshold in the first sub-region of the preset number is taken as a light source sub-region, that is, the first sub-region is determined as a light source sub-region if the first mutation value of the first sub-region is greater than the second preset threshold. The first mutation value can be understood as the brightness value in the target image, and the region with greater brightness value is more likely to be a light source sub-region, and the light emitted by the light source can be called illumination. If there are multiple first mutation values greater than the second preset threshold, the first sub-region corresponding to the maximum value in the multiple first mutation values is taken as a light source sub-region.

[0135] The image processing method provided by the embodiment of the present disclosure divides the light source candidate region into a plurality of first sub-regions, calculates the gray mean value of each first sub-region, determines the first mutation value corresponding to each first sub-region according to the gray mean value of the first sub-region and the position information of the first sub-region in the target image, and then determines the light source sub-region according to the first mutation value and the second preset threshold, that is, determines the position of the light source in the target image. Accurate determination of the position of the light source facilitates subsequent determination of the type of illumination in the target image based on the position of the light source.

[0136] On the basis of the above embodiments, Figure 6 A flowchart of a light scene classification method provided by the embodiments of the present disclosure is shown in FIG. 6. Optionally, the first classification result corresponding to the light and the target mutation value are obtained according to the light source sub-region and the reflection candidate region, and the method specifically includes the following steps S610 to S630 as shown in FIG. 6: Figure 6

[0137] S610, the reflection candidate region is divided into a plurality of second sub-regions.

[0138] It can be understood that the first preset number of the second sub-regions is determined in advance, and the reflection candidate region is divided into the first preset number of second sub-regions according to the first preset number.

[0139] S620, the second mutation value is calculated according to the plurality of second sub-regions and the light source sub-region.

[0140] Optionally, the specific steps of calculating the second mutation value include: determining the position information of each second sub-region in the target image according to the position information of the light source sub-region in the target image; calculating the gray mean value of each second sub-region according to the position information of each second sub-region in the target image, and calculating the second mutation value based on the gray mean value of each second sub-region.

[0141] It can be understood that, on the basis of the above S610, the position information of each second sub-region in the target image is determined according to the position information of the light source sub-region in the target image. The position information can be the position coordinates of the light source sub-region in the target image. The longitudinal coordinates of each second sub-region are determined according to the longitudinal coordinates of the light source sub-region and the first preset number, that is, the association between the longitudinal coordinates of each second sub-region and the longitudinal coordinates of the light source sub-region is established, which is convenient for subsequent determination of the reflection sub-region of the light source.

[0142] It can be understood that after the position information of each second sub-region in the target image is determined, the gray mean value of each second sub-region is calculated according to the position information of the second sub-region and the first preset number, and the second mutation value of each second sub-region is calculated based on the gray mean value of each second sub-region. The maximum value of the preset number of second mutation values is taken as the final second mutation value.

[0143] It can be understood that the formula for calculating the second mutation value of each second sub-region is shown in formula (8).

[0144]

[0145] wherein q represents the qth second sub-region, y' is the longitudinal coordinate of the second sub-region, m(q, y') represents the gray mean value of the second sub-region, a road ​(q, y') represents a second mutation value of the second sub-region.

[0146] S630, according to the second mutation value and the third preset threshold, determining the first classification result corresponding to the light and the target mutation value.

[0147] Optionally, the target mutation value is the second mutation value; and the specific step of determining the first classification result corresponding to the light comprises: if the third classification result is natural light and the second mutation value is less than or equal to the third preset threshold, determining that the first classification result corresponding to the light is natural light; if the third classification result is natural light and the second mutation value is greater than the third preset threshold, determining that the first classification result corresponding to the light is backlight; if the third classification result is low light and the second mutation value is less than or equal to the third preset threshold, determining that the first classification result corresponding to the light is low light; and if the third classification result is low light and the second mutation value is greater than the third preset threshold, determining that the first classification result corresponding to the light is lamp light.

[0148] It can be understood that the target mutation value is the second mutation value, which is the mutation value corresponding to the determination of the first classification result when compared with the third preset threshold. If the third classification result determined according to the mean value of the target image is natural light, and the second mutation value calculated in S620 is less than or equal to the third preset threshold, it is determined that the first classification result corresponding to the light in the target image is natural light, that is, the light sensed by the camera device when the camera device captures the target image is natural light; if the third classification result is natural light and the second mutation value is greater than the third preset threshold, it is determined that the first classification result corresponding to the light is backlight, and at this time, the second mutation value greater than the third preset threshold can be regarded as the target mutation value, and the result output by the mutation model is the first classification result (natural light) and the target mutation value (second mutation value); if the third classification result is low light and the second mutation value is less than or equal to the third preset threshold, it is determined that the first classification result corresponding to the light is low light; and if the third classification result is low light and the second mutation value is greater than the third preset threshold, it is determined that the first classification result corresponding to the light is lamp light, and at this time, the result output by the mutation model is the first classification result (lamp light) and the target mutation value (second mutation value).

[0149] The image processing method provided by the embodiments of the present disclosure divides the reflection candidate region into a plurality of second sub-regions according to a first preset number, calculates a second mutation value of each second sub-region, then judges the relationship between the second mutation value and a third preset threshold, and determines the first classification result corresponding to the light and the target mutation value. After the preliminary classification according to the gray mean value of the target image, the first classification result corresponding to the light and the target mutation value are determined according to the light source sub-region and the reflection candidate region, so that the accuracy of the result output by the mutation model is relatively high, and the referenceability is also relatively high when combined with the recognition result of the neural network model.

[0150] On the basis of the above embodiments, Figure 7 The image processing method provided by the embodiments of the present disclosure is described below, including the following steps S710-S792 as shown in the figure: Figure 7

[0151] S710, performing gray processing on the target image.

[0152] It can be understood that the gray processing on the target image is to unify the RGB value into a gray value, and to convert the color image into a gray image.

[0153] S720, calculating the gray mean value of the target image after the gray processing.

[0154] It can be understood that the gray mean value of the target image after the gray processing can be calculated according to the above formula (6).

[0155] S730, determining the third classification result corresponding to the illumination according to the gray mean value.

[0156] It can be understood that the method for determining the third classification result is described above, and will not be repeated here.

[0157] S740, dividing the light source candidate region and the reflection candidate region according to the third classification result.

[0158] It can be understood that the method for dividing the light source candidate region and the reflection candidate region is described above, and will not be repeated here.

[0159] S750, determining the light source region according to the light source candidate region.

[0160] It can be understood that the method for determining the light source region is described above, and will not be repeated here.

[0161] S760, calculating the second mutation value according to the light source position and the reflection candidate region.

[0162] It can be understood that the method for calculating the second mutation value is described above, and will not be repeated here.

[0163] S770, judging whether the third classification result is natural light.

[0164] It can be understood that on the basis of the above S760, it is judged whether the third classification result determined in S730 is natural light, if yes, S780 is executed; if not, S790 is executed.

[0165] S780, judging whether the second mutation value is greater than a third preset threshold.

[0166] ​Understandably, on the basis of S770, it is judged whether the second mutation value is greater than the third preset threshold value, and the preset threshold value is the third preset threshold value; if yes, S781 is executed; if no, S782 is executed.

[0167] S781, determining that the first classification result is backlight.

[0168] Understandably, on the basis of S780, if the second mutation value is greater than the third preset threshold value, it is determined that the first classification result is backlight.

[0169] S782, determining that the first classification result is natural light.

[0170] Understandably, on the basis of S780, if the second mutation value is less than or equal to the third preset threshold value, it is determined that the first classification result is natural light.

[0171] S790, judging whether the second mutation value is greater than the third preset threshold value.

[0172] Understandably, on the basis of S770, it is judged whether the second mutation value is greater than the third preset threshold value, and the preset threshold value is the third preset threshold value; if yes, S791 is executed; if no, S792 is executed.

[0173] S791, determining that the first classification result is light.

[0174] Understandably, on the basis of S790, if the second mutation value is greater than the third preset threshold value, it is determined that the first classification result is light.

[0175] S792, determining that the first classification result is low light.

[0176] Understandably, on the basis of S790, if the second mutation value is less than or equal to the third preset threshold value, it is determined that the first classification result is low light.

[0177] Figure 8 A structural schematic diagram of an image processing device provided by an embodiment of the present disclosure. The image processing device provided by the embodiment of the present disclosure can execute the processing flow provided by the image processing method embodiment. As shown in Figure 8 The image processing device 800 includes:

[0178] An acquisition unit 810, configured to acquire a target image.

[0179] A first identification unit 820, configured to process the target image through a pre-trained mutation model to obtain a first classification result corresponding to the light sensed when the target image is photographed and a target mutation value.

[0180] The second identification unit 830 is configured to identify the target image by using the pre-trained neural network model to obtain a second classification result corresponding to the light and a probability value of the light being the target light.

[0181] The determination unit 840 is configured to determine a classification result of the light according to the first classification result, the target mutation value, the second classification result, and the probability value.

[0182] Optionally, the first identification unit 820 processes the target image by using the pre-trained mutation model to obtain a first classification result corresponding to the light and a target mutation value sensed when the target image is captured, and specifically for:

[0183] performing gray scale processing on the target image, and determining a third classification result corresponding to the light sensed when the target image is captured according to a gray scale processing result;

[0184] dividing the target image into a light source candidate region and a reflection candidate region according to the third classification result;

[0185] calculating a first mutation value of the light source candidate region, and determining a light source sub-region in the light source candidate region according to the first mutation value;

[0186] obtaining the first classification result corresponding to the light and the target mutation value according to the light source sub-region and the reflection candidate region.

[0187] Optionally, the gray scale processing result in the first identification unit 820 includes a gray scale mean value of the target image.

[0188] Optionally, the first identification unit 820 determines the third classification result corresponding to the light sensed when the target image is captured according to the gray scale processing result, and specifically for:

[0189] determining the third classification result corresponding to the light sensed when the target image is captured according to the gray scale mean value of the target image and a first preset threshold.

[0190] Optionally, the first identification unit 820 determines the third classification result corresponding to the light sensed when the target image is captured according to the gray scale mean value of the target image and a first preset threshold, and specifically for:

[0191] if the gray scale mean value of the target image is greater than the first preset threshold, determining that the third classification result is natural light;

[0192] if the gray scale mean value of the target image is less than or equal to the first preset threshold, determining that the third classification result is low light.

[0193] Optionally, the first identification unit 820 calculates the first mutation value of the light source candidate region, and determines the light source sub-region in the light source candidate region according to the first mutation value, and specifically for:

[0194] divide the light source candidate region into a plurality of first sub-regions;

[0195] calculate a mean gray value of each of the plurality of first sub-regions;

[0196] calculate a first mutation value corresponding to each of the plurality of first sub-regions according to the mean gray value of each of the plurality of first sub-regions and position information of each of the plurality of first sub-regions in the target image;

[0197] take, as a light source sub-region, a first sub-region of the plurality of first sub-regions whose first mutation value is greater than a second preset threshold value.

[0198] Optionally, the first identification unit 820 obtains a first classification result corresponding to the illumination and a target mutation value according to the light source sub-region and the reflection candidate region, and specifically for:

[0199] divide the reflection candidate region into a plurality of second sub-regions;

[0200] calculate a second mutation value according to the plurality of second sub-regions and the light source sub-region;

[0201] determine the first classification result corresponding to the illumination and the target mutation value according to the second mutation value and a third preset threshold value.

[0202] Optionally, the first identification unit 820 calculates the second mutation value according to the plurality of second sub-regions and the light source sub-region, and specifically for:

[0203] determine position information of each of the plurality of second sub-regions in the target image according to position information of the light source sub-region in the target image;

[0204] calculate a mean gray value of each of the plurality of second sub-regions according to the position information of each of the plurality of second sub-regions in the target image, and calculate the second mutation value based on the mean gray value of each of the plurality of second sub-regions.

[0205] Optionally, the target mutation value in the first identification unit 820 is the second mutation value.

[0206] Optionally, the first identification unit 820 determines the first classification result corresponding to the illumination according to the second mutation value and the third preset threshold value, and specifically for:

[0207] if the third classification result is natural light and the second mutation value is less than or equal to the third preset threshold value, determine that the first classification result corresponding to the illumination is natural light;

[0208] if the third classification result is natural light and the second mutation value is greater than the third preset threshold value, determine that the first classification result corresponding to the illumination is backlight;

[0209] If the third classification result is low light, and the second mutation value is less than or equal to the third preset threshold, it is determined that the first classification result corresponding to the light is low light.

[0210] If the third classification result is low light, and the second mutation value is greater than the third preset threshold, it is determined that the first classification result corresponding to the light is lamp light.

[0211] Optionally, the determining unit 840 determines a target classification result of the light according to the first classification result, the target mutation value, the second classification result and the probability value, and is specifically used for:

[0212] calculating a sum value of the target mutation value corresponding to the first classification result and the probability value corresponding to the first classification result;

[0213] If the probability value corresponding to the other classification result except the first classification result in the second classification result is greater than the sum value, the other classification result with the maximum probability value is taken as the target classification result of the light;

[0214] If the probability value corresponding to the other classification result except the first classification result in the second classification result is less than the sum value, the first classification result is taken as the target classification result of the light.

[0215] Figure 8 The image processing apparatus of the illustrated embodiment can be used to execute the technical solutions of the above-mentioned method embodiments, and has similar implementation principles and technical effects, which will not be described here in detail.

[0216] Figure 9 A structural schematic diagram of an electronic device provided by the embodiment of the present disclosure is provided. The electronic device can be a server or a terminal as described above. The electronic device provided by the embodiment of the present disclosure can execute the processing flow provided by the above-mentioned embodiments, such as Figure 9 As shown, the electronic device 900 includes a processor 910, a communication interface 920 and a memory 930; wherein the computer program is stored in the memory 930 and is configured to be executed by the processor 910 to execute the image processing method as described above.

[0217] In addition, the embodiment of the present disclosure also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image processing method of the above-mentioned embodiment.

[0218] In addition, the embodiment of the present disclosure also provides a computer program product, which includes a computer program or instructions, and the computer program or instructions are executed by a processor to implement the image processing method as described above.

[0219] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0220] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized by, The method comprises: acquiring a target image; processing the target image through a pre-trained mutation model to obtain a first classification result corresponding to the light sensed when the target image is captured and a target mutation value, comprising: dividing the target image into a light source candidate region and a reflection candidate region; dividing the light source candidate region into a plurality of first sub-regions; dividing the reflection candidate region into a plurality of second sub-regions; calculating the gray mean value of each second sub-region according to the plurality of second sub-regions and the light source sub-region, and determining the target mutation value based on the gray mean value of each second sub-region, wherein the light source sub-region is at least one first sub-region in the plurality of first sub-regions; identifying the target image through a pre-trained neural network model to obtain a second classification result corresponding to the light and a probability value of the light being target light; determining a target classification result of the light according to the first classification result, the target mutation value, the second classification result, and the probability value, comprising: calculating the sum of the target mutation value corresponding to the first classification result and the probability value corresponding to the first classification result, wherein the probability value corresponding to the first classification result refers to the probability value corresponding to the classification result identical to the first classification result in the second classification result; if the probability value corresponding to other classification results in the second classification result except the first classification result is greater than the sum, the classification result with the maximum probability value is taken as the target classification result of the light; if the probability value corresponding to other classification results in the second classification result except the first classification result is less than the sum, the first classification result is taken as the target classification result of the light.

2. The method of claim 1, wherein, processing the target image through a pre-trained mutation model to obtain a first classification result corresponding to the light sensed when the target image is captured and a target mutation value, comprising: performing gray processing on the target image, and determining a third classification result corresponding to the light sensed when the target image is captured according to the gray processing result; dividing the target image into a light source candidate region and a reflection candidate region according to the third classification result; calculating a first mutation value of the light source candidate region, and determining a light source sub-region in the light source candidate region according to the first mutation value; obtaining the first classification result corresponding to the light and the target mutation value according to the light source sub-region and the reflection candidate region.

3. The method of claim 2, wherein, The gray processing result comprises a gray mean value of the target image; determining a third classification result corresponding to the light sensed when the target image is captured according to the gray processing result, comprising: determining the third classification result corresponding to the light sensed when the target image is captured according to the gray mean value of the target image and a first preset threshold.

4. The method of claim 3, wherein, determining the third classification result corresponding to the light sensed when the target image is captured according to the gray mean value of the target image and a first preset threshold, comprising: if the gray mean value of the target image is greater than the first preset threshold, determining the third classification result as natural light; If the mean gray value of the target image is less than or equal to the first preset threshold, the third classification result is determined as low light.

5. The method of claim 2, wherein, The first mutation value of the light source candidate region is calculated, and a light source sub-region in the light source candidate region is determined according to the first mutation value, including: The light source candidate region is divided into a plurality of first sub-regions; The mean gray value of each first sub-region in the plurality of first sub-regions is calculated; According to the mean gray value of each first sub-region and the position information of each first sub-region in the target image, the first mutation value corresponding to each first sub-region is calculated; The first sub-region with the first mutation value greater than the second preset threshold in the plurality of first sub-regions is taken as the light source sub-region.

6. The method of claim 2, wherein, The first classification result and the target mutation value corresponding to the light are obtained according to the light source sub-region and the reflection candidate region, including: The reflection candidate region is divided into a plurality of second sub-regions; The second mutation value is calculated according to the plurality of second sub-regions and the light source sub-region; The first classification result and the target mutation value corresponding to the light are determined according to the second mutation value and a third preset threshold.

7. The method of claim 6, wherein, The second mutation value is calculated according to the plurality of second sub-regions and the light source sub-region, including: According to the position information of the light source sub-region in the target image, the position information of each second sub-region in the plurality of second sub-regions in the target image is determined; According to the position information of each second sub-region in the target image, the mean gray value of each second sub-region is calculated, and the second mutation value is calculated based on the mean gray value of each second sub-region.

8. The method of claim 6, wherein, The target mutation value is the second mutation value; The first classification result corresponding to the light is determined according to the second mutation value and a third preset threshold, including: If the third classification result is natural light, and the second mutation value is less than or equal to the third preset threshold, the first classification result corresponding to the light is determined as natural light; If the third classification result is natural light, and the second mutation value is greater than the third preset threshold, the first classification result corresponding to the light is determined as back light; If the third classification result is low light, and the second mutation value is less than or equal to the third preset threshold, the first classification result corresponding to the light is determined as low light; If the third classification result is low light, and the second mutation value is greater than the third preset threshold, the first classification result corresponding to the light is determined as lamp light.

9. An image processing apparatus characterized by comprising: including: An acquisition unit is configured to acquire a target image; The first identification unit is configured to process the target image by using a pre-trained mutation model to obtain a first classification result corresponding to the light sensed when the target image is captured and a target mutation value, including: dividing the target image into a light source candidate region and a reflection candidate region; dividing the light source candidate region into a plurality of first sub-regions; dividing the reflection candidate region into a plurality of second sub-regions; calculating the average gray value of each second sub-region according to the plurality of second sub-regions and light source sub-regions, and determining the target mutation value based on the average gray value of each second sub-region, wherein the light source sub-region is at least one of the plurality of first sub-regions; The second identification unit is configured to identify the target image by using a pre-trained neural network model to obtain a second classification result corresponding to the light and a probability value of the light being target light; The determination unit is configured to determine the classification result of the light according to the first classification result, the target mutation value, the second classification result, and the probability value, including: calculating the sum of the target mutation value corresponding to the first classification result and the probability value corresponding to the first classification result, wherein the probability value corresponding to the first classification result refers to the probability value corresponding to the classification result identical to the first classification result in the second classification result; if the probability value corresponding to the classification result other than the first classification result in the second classification result is greater than the sum, the classification result with the maximum probability value is taken as the target classification result of the light; if the probability value corresponding to the classification result other than the first classification result in the second classification result is less than the sum, the first classification result is taken as the target classification result of the light.

10. An electronic device, comprising: including: a memory; a processor; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the image processing method of any one of claims 1 to 8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the image processing method of any one of claims 1 to 8.

Images