Device and method for calculating visibility value based on image and electronic equipment

Abstract

The embodiment of the invention provides a device and a method for calculating a visibility value based on an image, and electronic equipment. The device comprises a recognition device which recognizes the weather category in the current image; and the calculation device is used for calculating the visibility value in the current image according to the similarity information of the overlapped part of the current image and the reference image when the weather category in the current image is a preset category. According to the invention, the atmospheric visibility can be accurately measured at low cost.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology. Background Art

[0002] Generally, there are two main ways to measure atmospheric visibility: optical sensor-based methods and visual observation-based methods.

[0003] Since many video surveillance cameras have been widely deployed, estimating atmospheric visibility from surveillance videos has high practical value and has attracted the attention of many researchers.

[0004] There are three methods for estimating atmospheric visibility based on surveillance videos: the first method requires specifying a reference target or using auxiliary equipment; the second method requires extracting simple image features from video images or using a deep learning model to extract features of video images, and then using a convolutional neural network (CNN) model to obtain the classification results of atmospheric visibility; the third method requires finding relative features from paired images through deep learning.

[0005] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present application and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because these solutions are described in the background technology part of the present application. Summary of the invention

[0006] The above methods for measuring atmospheric visibility all have some limitations. For example, the method based on optical sensors requires the installation of a large number of optical sensors, which leads to high costs and great difficulty in implementation; the measurement results of the method based on visual observation are relatively subjective and easily affected by experience and the observation environment. For another example, in the method of estimating atmospheric visibility based on surveillance video: the first method requires reference targets or auxiliary equipment, and is therefore more complicated; the second method gives the classification results of atmospheric visibility, but cannot estimate the exact visibility value; the third method relies on a large amount of continuous visibility data for training. Therefore, how to provide a low-cost and accurate method for measuring atmospheric visibility has become a problem that needs to be solved.

[0007] In response to at least one of the above technical problems, the embodiments of the present application provide a device, method and electronic equipment method for calculating visibility values ​​based on images, which can accurately measure atmospheric visibility at low cost.

[0008] According to one aspect of an embodiment of the present application, a device for calculating a visibility value based on an image is provided, the device comprising:

[0009] an identification device for identifying a weather category in a current image; and

[0010] A calculation device calculates the visibility value in the current image based on similarity information of overlapping parts of the current image and a reference image when the weather category in the current image is a predetermined category.

[0011] According to another aspect of an embodiment of the present application, a method for calculating a visibility value based on an image is provided, the method comprising:

[0012] Identify the weather category in the current image; and

[0013] When the weather category in the current image is a predetermined category, the visibility value in the current image is calculated according to similarity information of an overlapping portion of the current image and a reference image.

[0014] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the aforementioned method.

[0015] One of the beneficial effects of the embodiments of the present application is that the atmospheric visibility value is calculated based on the similarity information of the overlapping parts of the current image and the reference image, thereby being able to accurately measure the atmospheric visibility at a relatively low cost.

[0016] With reference to the following description and accompanying drawings, the specific implementation of the embodiment of the present application is disclosed in detail, indicating the way in which the principle of the embodiment of the present application can be adopted. It should be understood that the implementation of the present application is not limited in scope. Within the scope of the spirit and clauses of the attached claims, the implementation of the present application includes many changes, modifications and equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the implementation methods of the present application, and together with the text description, explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other implementation methods can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 is a schematic diagram of a method for calculating visibility values ​​based on an image according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of operation 102;

[0020] Figure 3It is a schematic diagram of segmenting the current image;

[0021] Figure 4 is a schematic diagram of operation 202;

[0022] Figure 5 is a schematic diagram of a device for calculating visibility values ​​based on an image;

[0023] Figure 6 It is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] With reference to the accompanying drawings, the foregoing and other features of the embodiments of the present application will become apparent through the following description. In the description and the accompanying drawings, specific embodiments of the present application are specifically disclosed, which show some embodiments in which the principles of the embodiments of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the embodiments of the present application include all modifications, variations and equivalents that fall within the scope of the attached claims.

[0025] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements in terms of title, but do not indicate the spatial arrangement or temporal order of these elements, etc., and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.

[0026] In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms and should be broadly understood as "a kind" or "a type" rather than being limited to the meaning of "one"; in addition, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. In addition, the term "according to" should be understood as "at least in part according to...", and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.

[0027] Features described and / or shown for one embodiment may be used in one or more other embodiments in the same or similar manner, combined with features in other embodiments, or replace features in other embodiments. The term "includes / comprising" when used herein refers to the presence of a feature, an entirety, a step, or a component, but does not exclude the presence or addition of one or more other features, entireties, steps, or components.

[0028] Embodiments of the first aspect

[0029] An embodiment of the present application provides a method for calculating a visibility value based on an image.

[0030] Figure 1 FIG. 1 is a schematic diagram of a method for calculating visibility values ​​based on an image according to an embodiment of the present application. Figure 1 As shown, the method for calculating the visibility value based on the image includes:

[0031] Operation 101: Identify the weather category in the current image; and

[0032] Operation 102: When the weather category in the current image is a predetermined category, calculate the visibility value in the current image according to similarity information of an overlapping portion of the current image and a reference image.

[0033] According to an embodiment of the present application, the visibility value in the current image is calculated based on the similarity information of the overlapping part of the current image and the reference image. Thus, it is possible to accurately measure the atmospheric visibility at a low cost without the need for complex training with a large amount of data.

[0034] In operation 101, the current image may be a real-time image obtained by a monitoring device. The monitoring device is, for example, a monitoring camera or other device. The monitoring device is, for example, a device for traffic monitoring, and the current image obtained by the monitoring device may be used by the method of the present application to calculate the visibility value. In addition, the current image obtained by the monitoring device may also be used to detect and manage traffic conditions. Thus, the method of the present application can make the most of existing monitoring device resources without the need to set up dedicated sensors, thereby reducing costs.

[0035] In operation 101, the weather category in the current image may be identified based on a classification model. The weather categories that may be identified by the classification model include, for example, cloudy, sunny, rainy, foggy, snowy, snowdrift, etc. The above weather categories are only examples and are not limited to the above examples.

[0036] The classification model used in operation 101 is, for example, YOLOV5-cls. In addition, the present application is not limited thereto, and the classification model may also be other models. The classification model may be obtained by training based on images acquired by the monitoring device. In addition, when the position or viewing angle of the monitoring device changes, the classification model may be retrained based on new images acquired by the monitoring device after the position or viewing angle changes to ensure the accuracy of the classification model in identifying the weather category.

[0037] like Figure 1As shown, in operation 101, if the weather category in the current image is the first category (ie, the predetermined category), operation 102 is performed, and if the weather category in the current image is the second category, operation 103 is performed (ie, Figure 1 As shown). The first category may be a weather category in which atmospheric particulate matter has an impact on visibility, and the second category may be a weather category in which atmospheric particulate matter has a small impact or no impact on visibility, wherein the atmospheric particulate matter is, for example, raindrops, fog, snow, etc. For example, the first category of weather categories may be rainy, foggy, snowy, or snowdrift, etc.; the second category of weather categories may be cloudy or sunny, etc.

[0038] If the weather category in the current image is the second category, the operation 103 performed may be, for example, to use the current image as a reference image to calculate the visibility values ​​in other images. Thus, the reference image can be updated in a timely manner to avoid errors in visibility value calculation due to the lag of the reference image, thereby improving the accuracy of visibility value calculation.

[0039] In addition, if the weather category in the current image is the second category, the visibility value is not calculated for the current image, thereby reducing the amount of calculation in the visibility value calculation process.

[0040] If the weather category in the current image is the first category, it indicates that the particulate matter in the atmosphere has a greater impact on visibility, so it is necessary to perform operation 102 to calculate the visibility value.

[0041] Figure 2 is a schematic diagram of operation 102. Figure 2 As shown, operation 102 includes the following operations:

[0042] Operation 201, segmenting the current image and generating a mask; and

[0043] Operation 202: Calculate similarity information of overlapping portions of the current image and the reference image based on the mask, and calculate the visibility value based on the similarity information.

[0044] In operation 201, different objects in the current image may be identified, and the region of the current image may be segmented based on the identification result, and the region where the predetermined object is located may be used as a mask. In some embodiments, different objects in the current image may have different categories, and each category of objects has a corresponding region, and the region in the current image where the qualified object or category is located (i.e., the region where the predetermined object is located) is used as a mask.

[0045] When the visibility value is calculated using the method based on the atmospheric scattering model, some areas in the image do not meet the conditions required for the calculation. For example, in the current image, objects or categories such as the sky and snow do not meet the calculation requirements of the atmospheric scattering model, and thus are objects or categories that do not meet the conditions; for another example, in the current image, there may be moving objects or categories such as pedestrians and moving vehicles, which will reduce the accuracy of the visibility value calculation and thus are also objects or categories that do not meet the conditions.

[0046] In operation 201, the area in the current image where the categories that meet the conditions are located is used as a mask to shield the objects or categories in the current image that do not meet the calculation conditions, so that the visibility value can be calculated for the area in the current image where the objects or categories that meet the calculation conditions are located, thereby improving the accuracy of the visibility value calculation; in addition, there is no need to calculate the area outside the mask, so the amount of calculation can be reduced.

[0047] In operation 201, the category corresponding to the region of the current image segmented may be, for example, at least one of the following categories: road, sidewalk, building, fence, pole, traffic sign, vegetation, sky, person, car, bicycle, truck, water, snow, terrain, dotted line, line, etc. Among them, the region other than sky, person, car, bicycle, truck, and snow may be used as a mask.

[0048] In at least some embodiments of operation 201, a semantic segmentation model may be used to identify objects in the current image, and the current image may be segmented based on the recognition result. For example, the semantic segmentation model may have a unified perceptual parsing network (UperNet) architecture, and the backbone network of the unified perceptual parsing network (UperNet) architecture may be replaced with Swin-S, where Swin represents a sliding window. In addition, the present application is not limited thereto. For example, the semantic segmentation model may have other forms of architecture, or, operation 201 may use other types of models to segment the current image.

[0049] Figure 3 This is a schematic diagram of segmenting the current image. Figure 3 As shown, 301 is a current image, which is, for example, a color image; 302 is a segmented image.

[0050] In addition, in the present application, the method of operation 201 may also be used to segment the reference image and generate a mask.

[0051] In operation 202 , similarity information of overlapping portions of the current image and the reference image may be calculated based on the mask obtained in operation 201 , and the visibility value may be calculated based on the similarity information.

[0052] Figure 4 is a schematic diagram of operation 202, such as Figure 4 As shown, operation 202 may include the following operations:

[0053] Operation 401: extracting a first region from a current image and a second region from a reference image according to a mask;

[0054] Operation 402: Calculate parameters related to image blur for the current image and the reference image; and

[0055] Operation 403: Calculate similarity information according to a parameter related to image blur in an area where the first area and the second area overlap, and calculate the visibility value based on the similarity information.

[0056] In operation 401, the mask corresponding to the current image and the mask corresponding to the reference image may be intersected to obtain a fused mask, whereby the category corresponding to the fused mask is a qualified category existing in both the current image and the reference image. Furthermore, in operation 401, the fused mask may be used to extract a masked area from the current image as a first area, and extract a masked area from the reference image as a second area.

[0057] In operation 402, the parameters related to the blurring of the image include transmittance and / or gradient modulus. In addition, the present application may not be limited thereto, and the parameters related to the blurring of the image may also be other parameters.

[0058] In some examples, the transmittance can be calculated based on an atmospheric scattering model and a dark channel prior. For example, the transmittance can be calculated based on the following equation (1) and equation (2):

[0059]

[0060]

[0061] Wherein, formula (1) defines the dark channel J of image J dark , where J c is the color channel of J, and Ω(X) is the local region centered at pixel x. The dark channel prior means that if J is a fog-free outdoor image, then except for the sky region, J dark The intensity is low and tends to zero.

[0062] In formula (2), Ω(X) is the local area of ​​the input image I, represents the transmittance of the local area; c∈{r,g,b} is the color channel index, where r, g, b represent the red channel, green channel, and blue channel, respectively; y represents a sub-region in Ω(X), for example, the sub-region includes 1 or more pixels; ω(0<ω≤1) is a constant parameter based on the application, for example, ω can be 0.95; A is the light of the atmosphere, for example, 0.1% of the brightest pixels can be picked up in the mask area of ​​the dark channel, and among the picked pixels, the pixel with the highest intensity is selected as the light representing the atmosphere.

[0063] For detailed description of transmittance calculation, please refer to related technologies, for example, reference document 1 (Kaiming, He, Jian, Sun, Xiaoou, Tang. Single Image Haze Removal Using Dark Channel Prior. [J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 33 (12): 2341-53. DOI: 10.1109 / TPAMI.2010.168.). In addition, other methods can also be used to calculate the transmittance.

[0064] In some examples, the gradient modulus of the pixel can be calculated based on the gradient of the intensity of the pixel in the image in different directions. For example, the gradient modulus G(x) can be calculated based on the following formula (3):

[0065]

[0066] In formula (3), G x (x) and G y (x) represents the partial derivatives of the pixel value of pixel x in the x direction and the y direction, respectively, wherein the x direction and the y direction are, for example, the height direction and the width direction of the image.

[0067] In addition, the gradient modulus G(x) is not limited to equation (3), and can also be calculated based on other formulas.

[0068] In operation 402 , parameters related to blurring of an image are calculated for the first region and the second region, respectively.

[0069] In operation 403, similarity information is calculated for the overlapping area of ​​the first area and the second area. In some embodiments, the similarity information includes first similarity information S T (x) and the second similarity information S G (x), where the first similarity information S T (x) can be calculated based on the transmittance, and the second similarity information S G (x) can be calculated based on the gradient modulus. In addition, the present application may not be limited thereto, for example, the similarity information may also include other information.

[0070] In some examples, the first similarity information S T (x) can be calculated by the following formula (4):

[0071]

[0072] In some examples, the second similarity information SG(x) can be calculated by the following formula (5):

[0073]

[0074] In formula (4), T 1 (x) represents the transmittance of pixel x in the overlapped area in the reference image; T 2 (x) represents the transmittance of pixel x in the overlapping area in the current image; C 1 is a given positive constant used to increase stability.

[0075] In formula (5), G 1 (x) represents the gradient modulus of pixel x in the overlapping area in the reference image; G 2 (x) represents the gradient modulus of pixel x in the overlapping area in the current image; C 2 is another given positive constant.

[0076] In some examples, the visibility value VI can be calculated by the following formula (6):

[0077]

[0078] In formula (6), Ω represents the overlapping area set according to the fusion mask; α is used to adjust the first similarity information ST (x) and the second similarity information S G (x) is a parameter of relative importance between m (x) is used to weight the importance of similarity information of different pixels to obtain the final VI score.

[0079] Among them, T m (x) = max(1-T 1 (x),1-T 2 (x)).

[0080] For detailed description of the above formulas (4), (5), and (6), please refer to related technologies, for example, reference document 2 (Shiyu Zhao, Lin Zhang, Shuaiyi Huang, Ying Shen, and Shengjie Zhao. Dehazing evaluation: Real-world benchmark datasets, criteria and baselines. IEEE Trans. Image Processing, vol. 29, pp. 6947-6962, 2020). In addition, other methods can also be used to calculate the visibility value.

[0081] Only the steps or processes related to the present application are described above, but the present application is not limited thereto. The method of the present application may also include other steps or processes, and reference may be made to the prior art for the specific contents of these steps or processes. In addition, the embodiments of the present application are exemplarily described above by taking only some structures of the model used by the method of the present application as examples, but the present application is not limited to these structures, and these structures may also be appropriately modified, and the implementation methods of these modifications shall all be included within the scope of the embodiments of the present application.

[0082] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0083] As can be seen from the above embodiments, according to the embodiments of the present application, the visibility value in the current image is calculated based on the similarity information of the overlapping parts of the current image and the reference image. Thus, it is possible to accurately measure the atmospheric visibility at a lower cost without the need for complex training using a large amount of data.

[0084] Embodiments of the second aspect

[0085] The embodiment of the present application provides a device for calculating visibility value based on an image, which corresponds to the method for calculating visibility value based on an image in the embodiment of the first aspect. The embodiment of the second aspect is the same as the embodiment of the first aspect and the contents are not repeated here.

[0086] Figure 5 is a schematic diagram of a device for calculating visibility values ​​based on images. Figure 5 As shown, the device 500 includes:

[0087] Identification means 501, which identifies the weather category in the current image; and

[0088] The calculation device 502 calculates the visibility value in the current image according to the similarity information of the overlapping part of the current image and the reference image when the weather category in the current image is a predetermined category.

[0089] like Figure 5 As shown, the device 500 also includes:

[0090] The reference image updating device 503 uses the current image as a reference image to calculate visibility values ​​in other images when the weather category in the current image is not a predetermined category.

[0091] In at least some embodiments, the computing device 502 calculates the visibility value in the current image according to the similarity information of the overlapping portion of the current image and the reference image, including:

[0092] Segmenting the current image to generate a mask; and

[0093] Based on the mask, similarity information of the overlapping portion of the current image and the reference image is calculated, and based on the similarity information, the visibility value is calculated.

[0094] The step of segmenting the current image includes:

[0095] Identify different objects in the current image, and segment the region of the current image based on the identification result; and

[0096] The area where the predetermined object is located is used as a mask.

[0097] In at least some embodiments, calculating similarity information of overlapping portions of the current image and the reference image, and calculating the visibility value based on the similarity information, includes:

[0098] extracting a first region from the current image and a second region from the reference image according to the mask;

[0099] For the current image and the reference image, calculating parameters related to image blur; and

[0100] In an area where the first area and the second area overlap, similarity information is calculated according to a parameter related to blur of an image, and the visibility value is calculated based on the similarity information.

[0101] The parameters related to the blur of the image include transmittance and / or gradient modulus. The transmittance is calculated based on the atmospheric scattering model and dark channel priori, and the gradient modulus is calculated based on the gradient of the pixel value in different directions.

[0102] In at least some embodiments, the similarity information includes first similarity information and second similarity information, wherein the first similarity information S T (x) is calculated based on the transmittance, the second similarity information S G (x) is calculated based on the gradient modulus, and the visibility value is based on the first similarity information S T (x) and the second similarity information S G (x) is calculated.

[0103] It is worth noting that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The device described in the present application may also include other components or modules, and the specific contents of these components or modules may refer to the relevant technology.

[0104] To keep it simple, Figure 1 The connection relationship or signal direction between various components or modules is only exemplified, but it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned components or modules can be implemented by hardware facilities such as processors and memories; the embodiments of the present application are not limited to this.

[0105] The above embodiments are merely exemplary of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0106] Embodiments of the third aspect

[0107] The embodiment of the present application provides an electronic device, including the apparatus 500 for calculating visibility value based on an image as described in the embodiment of the second aspect, the contents of which are incorporated herein. The electronic device may be, for example, a computer, a server, a workstation, a laptop computer, a smart phone, etc., but the embodiment of the present application is not limited thereto.

[0108] Figure 6 Schematic diagram of an electronic device according to an embodiment of the present application. Figure 6 As shown, the electronic device 600 may include: a processor (e.g., a central processing unit CPU) 610 and a memory 620; the memory 620 is coupled to the central processing unit 610. The memory 620 may store various data; in addition, it may store a program 621 for information processing, and execute the program 621 under the control of the processor 610.

[0109] In some embodiments, the functions of the apparatus 500 for calculating visibility values ​​based on images are integrated into the processor 610. The processor 610 is configured to implement the method as described in the embodiment of the first aspect.

[0110] In some embodiments, the device 500 for calculating visibility values ​​based on images is configured separately from the processor 610. For example, the device 500 for calculating visibility values ​​based on images can be configured as a chip connected to the processor 610, and the functions of the device 500 for calculating visibility values ​​based on images can be implemented through the control of the processor 610.

[0111] For example, the processor 610 is configured to perform the following control: identifying the weather category in the current image; and when the weather category in the current image is a predetermined category, calculating the visibility value in the current image based on the similarity information of the overlapping part of the current image and the reference image.

[0112] In some embodiments, calculating the visibility value in the current image according to the similarity information of the overlapping portion of the current image and the reference image includes:

[0113] Segmenting the current image to generate a mask; and

[0114] Based on the mask, similarity information of the overlapping portion of the current image and the reference image is calculated, and based on the similarity information, the visibility value is calculated.

[0115] In some embodiments, segmenting the current image includes:

[0116] Identify different objects in the current image, and segment the region of the current image based on the identification result; and

[0117] The area where the predetermined object is located is used as a mask.

[0118] In some embodiments, calculating similarity information of an overlapping portion of the current image and the reference image, and calculating the visibility value based on the similarity information, comprises:

[0119] extracting a first region from the current image and a second region from the reference image according to the mask;

[0120] For the current image and the reference image, calculating parameters related to image blur; and

[0121] In an area where the first area and the second area overlap, similarity information is calculated according to a parameter related to blur of an image, and the visibility value is calculated based on the similarity information.

[0122] In some embodiments, parameters related to image blur include transmittance and / or gradient modulus, wherein the transmittance is calculated based on an atmospheric scattering model and a dark channel prior, and the gradient modulus is calculated based on the gradient of pixel values ​​in different directions.

[0123] In some embodiments, the similarity information includes first similarity information and second similarity information, wherein the first similarity information S T (x) is calculated based on the transmittance, the second similarity information S G (x) is calculated based on the gradient modulus, and the visibility value is based on the first similarity information S T (x) and the second similarity information S G (x) is calculated.

[0124] In some embodiments, when the weather category in the current image is not a predetermined category, the current image is used as a reference image to calculate visibility values ​​in other images.

[0125] In addition, if Figure 6 As shown, the electronic device 600 may also include: an input / output (I / O) device 630 and a display 640, etc. The functions of the above components are similar to those of the prior art and are not described in detail here. Figure 6 In addition, the electronic device 600 may also include Figure 6 For components not shown in the figure, reference may be made to the related art.

[0126] An embodiment of the present application also provides a computer-readable program, wherein when the program is executed in an electronic device, the program enables a computer to execute the method as described in the embodiment of the first aspect in the electronic device.

[0127] An embodiment of the present application further provides a storage medium storing a computer-readable program, wherein the computer-readable program enables a computer to execute the method as described in the embodiment of the first aspect in an electronic device.

[0128] The above devices and methods of the present application can be implemented by hardware, or by hardware combined with software. The present application relates to such a computer-readable program, which, when executed by a logic component, enables the logic component to implement the above-mentioned devices or components, or enables the logic component to implement the various methods or steps described above. The present application also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0129] The method / device described in conjunction with the embodiments of the present application may be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams shown in the figure and / or one or more combinations of the functional block diagrams may correspond to various software modules of the computer program flow or to various hardware modules. These software modules may correspond to the various steps shown in the figure, respectively. These hardware modules may be implemented by solidifying these software modules, for example, using a field programmable gate array (FPGA).

[0130] The software module may be located in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in a memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if a device (such as a mobile terminal) uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0131] For one or more of the functional blocks described in the drawings and / or one or more combinations of functional blocks, it can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component or any appropriate combination thereof for performing the functions described in the present application. For one or more of the functional blocks described in the drawings and / or one or more combinations of functional blocks, it can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0132] The present application is described above in conjunction with specific implementation methods, but it should be clear to those skilled in the art that these descriptions are exemplary and are not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles of the present application, and these modifications and variations are also within the scope of the present application.

[0133] This application also provides the following notes:

[0134] 1. A method for calculating visibility value based on an image, characterized in that the method comprises:

[0135] Identify the weather category in the current image; and

[0136] When the weather category in the current image is a predetermined category, the visibility value in the current image is calculated according to similarity information of an overlapping portion of the current image and a reference image.

[0137] 2. The method as described in Note 1, wherein the visibility value in the current image is calculated based on the similarity information of the overlapping portion of the current image and the reference image, comprising:

[0138] Segmenting the current image to generate a mask; and

[0139] Based on the mask, similarity information of the overlapping portion of the current image and the reference image is calculated, and based on the similarity information, the visibility value is calculated.

[0140] 3. The method as described in Note 2, wherein:

[0141] Segmenting the current image includes:

[0142] Identify different objects in the current image, and segment the region of the current image based on the identification result; and

[0143] The area where the predetermined object is located is used as a mask.

[0144] 4. The method as described in Note 3, wherein:

[0145] Calculating similarity information of an overlapping portion of the current image and the reference image, and calculating the visibility value based on the similarity information, comprising:

[0146] extracting a first region from the current image and a second region from the reference image according to the mask;

[0147] For the current image and the reference image, calculating parameters related to image blur; and

[0148] In an area where the first area and the second area overlap, similarity information is calculated according to a parameter related to blur of an image, and the visibility value is calculated based on the similarity information.

[0149] 5. The method as described in Note 4, wherein:

[0150] Parameters related to image blur include transmittance and / or gradient modulus.

[0151] in,

[0152] The transmittance is calculated based on the atmospheric scattering model and dark channel priori.

[0153] The gradient modulus is calculated based on the gradient of the pixel value in different directions.

[0154] 6. The method as described in Note 5, wherein:

[0155] The similarity information includes first similarity information and second similarity information,

[0156] The first similarity information S T (x) is calculated based on the transmittance,

[0157] The second similarity information S G (x) is calculated based on the gradient modulus,

[0158] The visibility value is based on the first similarity information S T (x) and the second similarity information S G (x) is calculated.

[0159] 7. The method as described in Note 1, wherein:

[0160] When the weather category in the current image is not a predetermined category, the current image is used as a reference image to calculate visibility values ​​in other images.

[0161] 8. A storage medium storing a computer-readable program, wherein the computer-readable program causes a processor coupled to the storage medium to execute the following method:

[0162] Identify the weather category in the current image; and

[0163] When the weather category in the current image is a predetermined category, the visibility value in the current image is calculated according to similarity information of an overlapping portion of the current image and a reference image.

[0164] 9. The storage medium according to supplementary note 8, wherein calculating the visibility value in the current image according to similarity information of overlapping parts of the current image and the reference image comprises:

[0165] Segmenting the current image to generate a mask; and

[0166] Based on the mask, similarity information of the overlapping portion of the current image and the reference image is calculated, and based on the similarity information, the visibility value is calculated.

[0167] 10. The storage medium according to Supplement 9, wherein:

[0168] Segmenting the current image includes:

[0169] Identify different objects in the current image, and segment the region of the current image based on the identification result; and

[0170] The area where the predetermined object is located is used as a mask.

[0171] 11. The storage medium according to Supplement 10, wherein:

[0172] Calculating similarity information of an overlapping portion of the current image and the reference image, and calculating the visibility value based on the similarity information, comprising:

[0173] extracting a first region from the current image and a second region from the reference image according to the mask;

[0174] For the current image and the reference image, calculating parameters related to image blur; and

[0175] In an area where the first area and the second area overlap, similarity information is calculated according to a parameter related to blur of an image, and the visibility value is calculated based on the similarity information.

[0176] 12. The storage medium according to Supplement 11, wherein:

[0177] Parameters related to image blur include transmittance and / or gradient modulus.

[0178] The transmittance is calculated based on the atmospheric scattering model and dark channel priori.

[0179] The gradient modulus is calculated based on the gradient of the pixel value in different directions.

[0180] 13. The storage medium according to Supplement 12, wherein:

[0181] The similarity information includes first similarity information and second similarity information,

[0182] The first similarity information S T (x) is calculated based on the transmittance,

[0183] The second similarity information S G (x) is calculated based on the gradient modulus,

[0184] The visibility value is based on the first similarity information S T (x) and the second similarity information S G (x) is calculated.

[0185] 14. The storage medium according to Supplement 8, wherein:

[0186] When the weather category in the current image is not a predetermined category, the current image is used as a reference image to calculate visibility values ​​in other images.

Claims

1. A device for calculating visibility value based on an image, It is characterized in that The device comprises: an identification device for identifying a weather category in a current image; and A calculation device calculates the visibility value in the current image based on similarity information of overlapping parts of the current image and a reference image when the weather category in the current image is a predetermined category.

2. The device according to claim 1, in, Calculating the visibility value in the current image according to similarity information of the overlapping portion of the current image and the reference image, comprising: Segmenting the current image to generate a mask; and Based on the mask, similarity information of the overlapping portion of the current image and the reference image is calculated, and based on the similarity information, the visibility value is calculated.

3. The device as claimed in claim 2, in, Segmenting the current image includes: Identify different objects in the current image, and segment the region of the current image based on the identification result; and The area where the predetermined object is located is used as a mask.

4. The device as claimed in claim 3, in, Calculating similarity information of an overlapping portion of the current image and the reference image, and calculating the visibility value based on the similarity information, comprising: extracting a first region from the current image and a second region from the reference image according to the mask; For the current image and the reference image, calculating parameters related to image blur; and In an area where the first area and the second area overlap, similarity information is calculated according to a parameter related to blur of an image, and the visibility value is calculated based on the similarity information.

5. The device as claimed in claim 4, in, Parameters related to image blur include transmittance and / or gradient modulus, in, The transmittance is calculated based on the atmospheric scattering model and dark channel priori. The gradient modulus is calculated based on the gradient of the pixel value in different directions.

6. The device as claimed in claim 5, in, The similarity information includes first similarity information and second similarity information, The first similarity information is calculated based on the transmittance, The second similarity information is calculated based on the gradient modulus, The visibility value is calculated based on the first similarity information and the second similarity information.

7. The device according to claim 1, in, The device also includes: The reference image updating device uses the current image as a reference image to calculate visibility values ​​in other images when the weather category in the current image is not a predetermined category.

8. An electronic device, comprising the device for calculating visibility value based on an image as claimed in any one of claims 1 to 7.

9. A method for calculating visibility value based on an image, It is characterized in that The method comprises: Identify the weather category in the current image; and When the weather category in the current image is a predetermined category, the visibility value in the current image is calculated according to similarity information of an overlapping portion of the current image and a reference image.

10. The method according to claim 9, in, Calculating the visibility value in the current image according to similarity information of the overlapping portion of the current image and the reference image, comprising: Segmenting the current image to generate a mask; and Based on the mask, similarity information of the overlapping portion of the current image and the reference image is calculated, and based on the similarity information, the visibility value is calculated.

Images