Surveillance camera and meteorological station fused surface precipitation high-resolution observation method

Abstract

The invention provides a monitoring camera and weather station fused surface precipitation high-resolution observation method, which comprises the following steps of: extracting rain lines from videos shot by monitoring cameras in an area, and determining the sizes and the quantity of raindrops corresponding to the rain lines; determining a rainfall intensity value corresponding to the monitoring camera according to the size and the number of the raindrops; and fusing the rainfall intensity value corresponding to the monitoring camera and the rainfall intensity value observed by the meteorological station in the area to obtain a space-time distribution diagram of the rainfall intensity value of the area. According to the invention, synchronous observation results of a monitoring camera and a (weather station) rain gauge are fused to form an encrypted earth surface rainfall observation network of the monitoring camera; the method is realized on the basis of existing urban monitoring resources and surface weather stations, and extra hardware investment and addition are not needed, so that the spatial resolution of surface rainfall observation is improved with low cost and high efficiency.

Description

Technical Field

[0001] The present invention relates to the field of meteorological monitoring technology, and in particular to a high-resolution observation method for surface precipitation integrating a monitoring camera with a meteorological station. Background Art

[0002] Currently, precipitation observation methods can be divided into two categories: indirect precipitation observation based on remote sensing means such as satellites and radars, and direct precipitation observation based on surface meteorological stations.

[0003] Among them, satellites and radars indirectly retrieve precipitation information by acquiring satellite cloud images, electromagnetic echoes and other signals. The advantages of this method are wide observation area and high spatial resolution, but the satellite revisit period is long (2 to 24 hours), and radar data processing and interpretation still require a period of time (5 to 6 minutes), resulting in the need to further improve the temporal resolution of the existing satellite radar precipitation range, so as to meet the urgent need for high temporal and spatial resolution precipitation data for rapid response hydrological events such as waterlogging and floods.

[0004] Surface meteorological stations include tipping bucket, weighing, siphon and other types of rain gauges, as well as one-dimensional laser raindrop spectrometers, two-dimensional video raindrop spectrometers and other precipitation microphysical characteristics measurement equipment. These devices can provide real-time and accurate point precipitation information, but the number of rain gauges in urban areas is sparse and the maintenance cost is high. This makes the spatial representation of precipitation information obtained by rain gauges insufficient, and the inability to obtain high-spatial-resolution meteorological information has become the biggest shortcoming of surface meteorological stations. Summary of the invention

[0005] The present invention provides a high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station, so as to solve the defect of low resolution of surface precipitation information in the prior art and improve the resolution of surface precipitation information.

[0006] The present invention provides a high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station, comprising:

[0007] Extract rain lines from the video captured by the surveillance camera in the area, and determine the size and number of raindrops corresponding to the rain lines;

[0008] Determining a rainfall intensity value corresponding to the monitoring camera according to the size and number of the raindrops;

[0009] The rainfall intensity value corresponding to the monitoring camera and the rainfall intensity value observed by the meteorological station in the area are fused to obtain a spatiotemporal distribution map of the rainfall intensity value in the area.

[0010] According to a high-resolution observation method for surface precipitation by integrating a surveillance camera with a meteorological station provided by the present invention, the method of extracting rain lines from a video captured by a surveillance camera in a region comprises:

[0011] Use the rain line description model to describe each rain-containing image:

[0012] ;

[0013] Wherein, O is a rain-containing image composed of a rain-free background image and rain lines corresponding to the video, B is the rain-free background image, and R is an area in the background image blocked or affected by rain lines. is a rain line image, (i, j) is the coordinate of the pixel in the corresponding image, and t is the frame number in the video;

[0014] Use deep learning regression models to fit different monitoring scenarios The changing law of can be used to describe the relationship between the rain line and the monitoring background, so as to achieve the separation between the rain line and the monitoring background.

[0015] According to a high-resolution observation method for surface precipitation by integrating a surveillance camera with a meteorological station provided by the present invention, after extracting rain lines from a video captured by a surveillance camera in an area, the method further comprises:

[0016] Extracting visual features of rain lines from a rain line visual knowledge base, and counting the value ranges of the visual features of the rain lines;

[0017] When the visual features of the rain lines extracted from the video are not within the visual range, the rain lines are removed.

[0018] According to a high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station provided by the present invention, the visual features include one or more of shape, texture, size and brightness.

[0019] According to a high-resolution observation method for surface precipitation by integrating a surveillance camera with a meteorological station provided by the present invention, before extracting rain lines from a video captured by a surveillance camera in an area, the method further includes:

[0020] Each rain-containing image in the video is converted into an HSV image.

[0021] According to a high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station provided by the present invention, the determining of the size and number of raindrops corresponding to the rain line comprises:

[0022] Determine the length and width of the rain line according to the camera imaging principle;

[0023] Calculating the ratio between the length and width of the rain line;

[0024] When the ratio is greater than a preset threshold, the size and number of raindrops corresponding to the rain line are determined.

[0025] According to a high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station provided by the present invention, the length L and width D of the rain line are determined according to the camera imaging principle by the following formula:

[0026]

[0027]

[0028] in, is the focal length, which refers to the distance between the lens and the focus where parallel light converges; Refers to the focusing distance, which is the distance from the mirror plane when the object image is clearest; is the object distance of the raindrop, that is, the distance from the raindrop to the lens; and Respectively represent the vertical and horizontal dimensions of the camera target surface; and Respectively represent the vertical and horizontal dimensions of the displayed video; and Respectively represent the vertical and horizontal dimensions of the rain line.

[0029] According to a high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station provided by the present invention, the determining of the size and number of raindrops corresponding to the rain line comprises:

[0030] According to the camera imaging principle, the relationship between the length L of the raindrop and the falling speed v is:

[0031]

[0032] Physics provides the following formula for calculating the falling speed of raindrops near the ground:

[0033]

[0034] in, is the exposure time of the monitoring camera, D is the width of the rain line, and the above formulas are solved to obtain the size of the raindrops corresponding to the rain line, and the number of the raindrops is obtained according to the size of the rain line and the size of the raindrops.

[0035] According to a high-resolution observation method for surface precipitation by integrating a monitoring camera and a meteorological station provided by the present invention, the rainfall intensity value corresponding to the monitoring camera and the rainfall intensity value observed by the meteorological station in the area are integrated to obtain a spatiotemporal distribution diagram of the rainfall intensity value in the area, including:

[0036] Aligning the observation time of the monitoring camera and the weather station, and unifying the resolution of the observation time of the monitoring camera and the weather station;

[0037] The Kriging interpolation model is used to interpolate the rainfall intensity values ​​observed by the monitoring camera and the meteorological station to obtain a spatiotemporal distribution map of the rainfall intensity values ​​in the area.

[0038] The present invention also provides a high-resolution surface precipitation observation device integrating a monitoring camera and a meteorological station, comprising:

[0039] An extraction module, used to extract rain lines from the video captured by the surveillance camera in the area, and determine the size and number of raindrops corresponding to the rain lines;

[0040] A prediction module, used to determine the rainfall intensity value corresponding to the monitoring camera according to the size and number of the raindrops;

[0041] The fusion module is used to fuse the rainfall intensity value corresponding to the monitoring camera with the rainfall intensity value observed by the meteorological station in the area to obtain a spatiotemporal distribution map of the rainfall intensity value in the area.

[0042] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a high-resolution surface precipitation observation method integrating a monitoring camera with a meteorological station as described above is implemented.

[0043] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a high-resolution surface precipitation observation method integrating a monitoring camera with a meteorological station as described in any one of the above.

[0044] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements a high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station as described in any one of the above.

[0045] The high-resolution observation method for surface precipitation by integrating monitoring cameras and meteorological stations provided by the present invention utilizes the existing urban monitoring camera resources in the city on the basis of the existing urban monitoring sensor network, gives full play to its observation advantages of large number, high density and fast data transmission, establishes a quantitative estimation method for precipitation by monitoring cameras, and integrates it with the existing meteorological station observation, thereby improving the spatial resolution of surface precipitation observation and effectively improving the high-resolution observation capability of urban precipitation. At the same time, no additional hardware equipment investment is required, the cost is low, and the utilization rate of urban resources is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is a flow chart of a high-resolution observation method for surface precipitation by integrating a monitoring camera and a weather station provided by the present invention;

[0048] Figure 2 It is a schematic diagram of a video rain line in the surface precipitation high-resolution observation method of the integrated monitoring camera and meteorological station provided by the present invention;

[0049] Figure 3 It is a schematic diagram of the structure of a deep learning model for rainfall intensity regression calculation in the surface precipitation high-resolution observation method integrating a monitoring camera and a meteorological station provided by the present invention;

[0050] Figure 4 It is a structural schematic diagram of a deep learning model for semi-supervised monitoring rain line extraction in a high-resolution surface precipitation observation method integrating a monitoring camera and a meteorological station provided by the present invention;

[0051] Figure 5 It is a schematic diagram of the monitoring video imaging principle of rainfall in the surface precipitation high-resolution observation method provided by the present invention by integrating the monitoring camera with the meteorological station;

[0052] Figure 6 It is a schematic diagram of the structure of a high-resolution observation device for surface precipitation that integrates a monitoring camera and a weather station provided by the present invention;

[0053] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Combine the following Figure 1 The present invention describes a high-resolution observation method for surface precipitation by integrating a monitoring camera with a weather station, comprising:

[0056] Step 101, extracting rain lines from the video captured by the surveillance camera in the area, and determining the size and number of raindrops corresponding to the rain lines;

[0057] Video rain lines refer to the images of raindrops appearing in the video. Since the exposure time of surveillance video is long, the image of falling raindrops in the video is usually linear, so it is called video rain lines. Rain lines contain information such as the falling speed, size, and number of raindrops, and are the basis for rainfall inversion. Therefore, the accurate extraction of rain lines is the primary premise of this invention. Figure 2 For illustration purposes, the rain line is marked in red.

[0058] An image segmentation algorithm, such as a convolutional neural network, can be used to extract rain lines from a video. This embodiment does not limit the method for extracting rain lines. After extracting the rain lines, the rain line extraction result can be corrected to ensure the integrity of the rain lines.

[0059] Step 102, determining the rainfall intensity value corresponding to the monitoring camera according to the size and number of the raindrops;

[0060] The size and number of raindrops contained in each rain line can be determined from the rain line extraction results. The raindrop size and number information extracted from the video can be used as input, and the rainfall intensity value can be used as output to build a rainfall intensity regression calculation model based on a convolutional neural network. First, the size and number of raindrops obtained are converted into corresponding feature vectors through dimension conversion, and then features are extracted with the help of a three-layer convolutional neural network, and the rainfall intensity value is calculated through regression.

[0061] There is a maximum pooling layer after each convolution layer in the three-layer convolutional neural network. After the calculation of the three convolutional layers, the one-dimensional feature quantity is output through regression calculation, which is the rainfall intensity value. The structure of the deep learning model for rainfall intensity regression calculation is as follows: Figure 3 shown.

[0062] Multiply the rainfall intensity value by the corresponding observation time to obtain the cumulative rainfall intensity value. The calculation formula is as follows:

[0063] (1)

[0064] in, is the cumulative rainfall (unit: mm), is the observation time (unit: hours), is the rainfall intensity value (unit: mm / h).

[0065] Step 103: The rainfall intensity value corresponding to the monitoring camera and the rainfall intensity value observed by the meteorological station in the area are integrated to obtain a spatiotemporal distribution diagram of the rainfall intensity value in the area.

[0066] This embodiment uses the existing urban monitoring camera resources in the city on the basis of the existing urban monitoring sensor network, gives full play to its observation advantages of large number, high density and fast data transmission, establishes a quantitative estimation method for precipitation using monitoring cameras, and integrates it with the existing meteorological station observations, thereby improving the spatial resolution of surface precipitation observations and effectively improving the high-resolution observation capability of urban precipitation. At the same time, no additional hardware equipment investment is required, the cost is low, and the utilization rate of urban resources is improved, which is green and environmentally friendly.

[0067] Usually the rain image can be expressed as: ( No rain background, Rain line, represents the entire rainy image). The rain line extraction or recognition task can be regarded as a process of separating the rain and background signals.

[0068] In the real world, rain does not only appear in the form of simple rain lines, but in more cases, it is the repeated superposition of multiple rain lines. When the density of rain lines is too high, it is more difficult to identify a single rain line. In view of the above problem, this embodiment improves the above formula so that it can more effectively describe the problem of identifying rain lines in cases where the rain lines overlap, the density is too high, the directions are messy, etc.

[0069] On the basis of the above embodiments, this embodiment takes into account the real-time and accuracy of video rain line extraction and proposes a new video rain line description model, the formula is as follows:

[0070] (2)

[0071] in, is the rain line image, is a video sequence; Represents the alpha matting image, which can be defined as:

[0072] (3)

[0073] in, It is the brightness value of the pixel blocked by rain, that is, the light penetration value caused by raindrops.

[0074] The expression of formula (2) about the rain picture can be converted into a pixel-based video expression:

[0075] (4)

[0076] Wherein, O is a rain-containing image composed of a rain-free background image and rain lines corresponding to the video, B is the rain-free background image, and R is an area in the background image blocked or affected by rain lines. is a rain line image, (i, j) is the coordinate of the pixel in the corresponding image, and t is the frame number in the video;

[0077] Solution This is the key to judging the rain line pixels in the video and extracting the rain line images. This embodiment is based on a large number of monitoring image samples of real rainfall scenes, extracts a large number of rain line pixels through manual annotation, and uses a deep learning regression model to fit different monitoring scenes. The changing law of can be used to describe the relationship between the rain line and the monitoring background, so as to separate the rain line from the monitoring background. The deep learning regression model can be a deep learning regression model based on a convolutional neural network.

[0078] The monitoring camera precipitation inversion model proposed in this embodiment is mainly implemented through software algorithms, has no special requirements for monitoring cameras and software algorithm implementation, is simple to deploy, plug-and-play, and has strong portability.

[0079] Based on the above embodiment, after extracting the rain line from the video captured by the surveillance camera in the area, this embodiment further includes:

[0080] Extracting visual features of rain lines from a rain line visual knowledge base, and counting the value ranges of the visual features of the rain lines;

[0081] When the visual features of the rain lines extracted from the video are not within the visual range, the rain lines are removed.

[0082] Due to factors such as lighting, background noise, and moving objects, video surveillance scenes are complex and changeable. In particular, rainy days are often accompanied by weather phenomena such as wind, which causes raindrops to collide during their descent, and changes in shape and direction visual features, which poses a huge challenge to the extraction of rain lines. This embodiment is based on a large amount of analysis, from pictures to videos, to establish image domain prior knowledge including size, shape, brightness distribution, and video prior laws such as speed, time domain, and frequency domain, which together form a visual feature knowledge base of rain lines.

[0083] Visual features of rain lines such as shape, texture, size, and brightness are selected from the visual knowledge base of rain lines as correction criteria for rain line extraction results, further improving the accuracy of rain line identification in complex monitoring scenes.

[0084] For example, by counting the value range of the visual features of rain lines, the judgment rules for rain lines are determined as follows:

[0085] Size: Rain lines with a pixel count of less than 5 are eliminated. The size of raindrops in nature is usually larger than 0.3 mm. According to the camera imaging principle, the number of pixels in ordinary commercially available surveillance cameras is greater than 20;

[0086] Shape: If a rain line is bent more than 10°, it will be rejected. The image of a free-falling raindrop is usually a straight line, and curved ones need to be rejected;

[0087] Brightness: If the brightness of the rain line is less than 50, it will be rejected. Optical research shows that raindrops have a focusing effect, and their brightness is much greater than the background. If the brightness is lower than the threshold, such as 50, it is considered a false positive;

[0088] Texture: Rain lines with large brightness differences between different pixels are eliminated. The brightness of the pixels in the same rain line is uniform and consistent. If the brightness difference between different pixels in the rain line is higher than a threshold, such as 50, it can be considered a misjudgment.

[0089] Before extracting the rain line from the video captured by the surveillance camera in the area, the method further includes:

[0090] Each rain-containing image in the video is converted into an HSV image.

[0091] Through a large number of comparisons and combined with existing research, we know that in the HSV color space, the saliency of rain lines is more prominent than that of the background. To this end, this embodiment converts the monitoring video RGB image into an HSV image and selects the visual feature vector of the brightness of the rain line as the clue and standard for rain line extraction. The structure of the semi-supervised monitoring rain line extraction deep learning model is as follows: Figure 4 shown.

[0092] Based on the above embodiment, the method of determining the size and number of raindrops corresponding to the rain line in this embodiment includes:

[0093] Determine the length and width of the rain line according to the camera imaging principle;

[0094] Calculating the ratio between the length and width of the rain line;

[0095] When the ratio is greater than a preset threshold, the size and number of raindrops corresponding to the rain line are determined.

[0096] The relationship between the length L and width D of the rain line and the camera parameters is determined by the following formula according to the camera imaging principle:

[0097] (5)

[0098] (6)

[0099] in, is the focal length, which refers to the distance between the lens and the focus where parallel light converges; Refers to the focusing distance (unit: mm), which is the distance from the mirror plane when the object image is clearest; is the object distance of the raindrop, that is, the distance from the raindrop to the lens; and Respectively represent the vertical and horizontal dimensions of the camera target surface (unit: mm); and Respectively represent the vertical and horizontal dimensions of the displayed video (unit: mm); and Respectively represent the vertical and horizontal dimensions of the rain line (unit: mm).

[0100] According to the camera imaging principle, Figure 5 As shown in the figure, only the image of raindrops in the depth of field area is less blurred and contains more accurate rainfall information. The raindrops before the near point are too close to the camera lens, the rain lines cannot be fully captured by the video and are larger in size; the raindrops after the far point have a blurry image, which affects the judgment of the raindrop size. Therefore, the main task of the rain line 3D spatial restoration model is to distinguish the rain lines within the depth of field and calculate the corresponding raindrop size. With the help of formulas (5) and (6), the aspect ratio of the rain line is calculated to judge the integrity of the rain line, and the rain lines are further screened to determine the size and number of raindrops corresponding to the complete rain line.

[0101] Based on the above embodiment, the method of determining the size and number of raindrops corresponding to the rain line in this embodiment includes:

[0102] According to the camera imaging principle, the relationship between the length L of the raindrop and the falling speed v is:

[0103] (7)

[0104] Physics provides the following formula for calculating the falling speed of raindrops near the ground:

[0105] (8)

[0106] in, is the exposure time of the monitoring camera, D is the width of the rain line, and the above formulas are solved to obtain the size of the raindrops corresponding to the rain line, and the number of the raindrops is obtained according to the size of the rain line and the size of the raindrops.

[0107] On the basis of the above embodiment, the rainfall intensity value corresponding to the monitoring camera and the rainfall intensity value observed by the meteorological station in the area are fused in this embodiment to obtain a spatiotemporal distribution diagram of the rainfall intensity value in the area, including:

[0108] Aligning the observation time of the monitoring camera and the weather station, and unifying the resolution of the observation time of the monitoring camera and the weather station to serve subsequent fusion operations;

[0109] Using the Kriging interpolation model, the observation results of the monitoring camera and the weather station are used as input to interpolate the rainfall intensity values ​​observed by the monitoring camera and the weather station to obtain the spatiotemporal distribution map of the rainfall intensity values ​​in the area. Due to the high density of cameras, the spatial resolution of the observation results is improved, solving the pain point of insufficient spatial representativeness of current surface precipitation observations.

[0110] Starting from the perspective of surveillance cameras, this embodiment makes full use of the observation advantages of urban surveillance cameras such as high density, fast transmission, and sustainability, establishes a quantitative estimation method for precipitation with surveillance cameras, and integrates the synchronous observation results of surveillance cameras and meteorological station rain gauges to form a surveillance camera-encrypted surface rainfall observation network; it is achieved on the basis of existing urban monitoring resources and surface meteorological stations without the need for additional hardware investment and addition, thereby providing new development opportunities for the refined perception and modeling of urban rainfall at low cost and high efficiency, and at the same time providing basic data support for research on waterlogging control, micrometeorological observation, water resources management, etc.

[0111] The high-resolution observation device for surface precipitation integrating a surveillance camera and a meteorological station provided by the present invention is described below. The high-resolution observation device for surface precipitation integrating a surveillance camera and a meteorological station described below and the high-resolution observation method for surface precipitation integrating a surveillance camera and a meteorological station described above can be referred to each other.

[0112] like Figure 6 As shown, the device includes an extraction module 601, a prediction module 602 and a fusion module 603, wherein:

[0113] The extraction module 601 is used to extract rain lines from the video captured by the surveillance camera in the area, and determine the size and number of raindrops corresponding to the rain lines;

[0114] The prediction module 602 is used to determine the rainfall intensity value corresponding to the monitoring camera according to the size and number of the raindrops;

[0115] The fusion module 603 is used to fuse the rainfall intensity value corresponding to the monitoring camera with the rainfall intensity value observed by the meteorological station in the area to obtain a spatiotemporal distribution diagram of the rainfall intensity value in the area.

[0116] This embodiment uses the existing urban monitoring camera resources in the city on the basis of the existing urban monitoring sensor network, gives full play to its observation advantages of large number, high density and fast data transmission, establishes a quantitative estimation method for precipitation using monitoring cameras, and integrates it with the existing meteorological station observations, thereby improving the spatial resolution of surface precipitation observations and effectively improving the high-resolution observation capability of urban precipitation. At the same time, no additional hardware equipment investment is required, the cost is low, and the utilization rate of urban resources is improved.

[0117] Figure 7An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7 As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a high-resolution observation method for surface precipitation fused with a surveillance camera and a meteorological station, the method comprising: extracting rain lines from a video captured by a surveillance camera in a region, determining the size and number of raindrops corresponding to the rain lines; determining the rainfall intensity value corresponding to the surveillance camera according to the size and number of raindrops; fusing the rainfall intensity value corresponding to the surveillance camera with the rainfall intensity value observed by the meteorological station in the region, and obtaining a spatiotemporal distribution diagram of the rainfall intensity value in the region.

[0118] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0119] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the high-resolution observation method of surface precipitation that integrates monitoring cameras and meteorological stations provided by the above-mentioned methods. The method includes: extracting rain lines from videos taken by monitoring cameras in an area, and determining the size and number of raindrops corresponding to the rain lines; determining the rainfall intensity value corresponding to the monitoring camera based on the size and number of raindrops; and fusing the rainfall intensity value corresponding to the monitoring camera with the rainfall intensity value observed by the meteorological station in the area to obtain a spatiotemporal distribution map of the rainfall intensity value in the area.

[0120] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the high-resolution surface precipitation observation method provided by the above-mentioned methods by integrating a surveillance camera with a meteorological station, the method comprising: extracting rain lines from a video captured by a surveillance camera in an area, and determining the size and number of raindrops corresponding to the rain lines; determining a rainfall intensity value corresponding to the surveillance camera based on the size and number of raindrops; and fusing the rainfall intensity value corresponding to the surveillance camera with the rainfall intensity value observed by the meteorological station in the area to obtain a spatiotemporal distribution map of the rainfall intensity value in the area.

[0121] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-resolution observation method for surface precipitation integrating a monitoring camera and a meteorological station, characterized in that: include: Extract rain lines from the video captured by the surveillance camera in the area, and determine the size and number of raindrops corresponding to the rain lines; Determining a rainfall intensity value corresponding to the monitoring camera according to the size and number of the raindrops; The rainfall intensity value corresponding to the monitoring camera and the rainfall intensity value observed by the meteorological station in the area are fused to obtain a spatiotemporal distribution map of the rainfall intensity value in the area.

2. The high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station according to claim 1 is characterized in that: The step of extracting rain lines from a video captured by a surveillance camera in an area includes: Use the rain line description model to describe each rain-containing image: ； Wherein, O is a rain-containing image composed of a rain-free background image and rain lines corresponding to the video, B is the rain-free background image, and R is an area in the background image blocked or affected by rain lines. is a rain line image, (i, j) is the coordinate of the pixel in the corresponding image, and t is the frame number in the video; Use deep learning regression models to fit different monitoring scenarios The changing law of can be used to describe the relationship between the rain line and the monitoring background, so as to achieve the separation between the rain line and the monitoring background.

3. The high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station according to claim 1 is characterized in that: After extracting the rain line from the video captured by the surveillance camera in the area, the method further includes: Extracting visual features of rain lines from a rain line visual knowledge base, and counting the value ranges of the visual features of the rain lines; When the visual features of the rain lines extracted from the video are not within the visual range, the rain lines are removed.

4. The high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station according to claim 3 is characterized in that: The visual characteristics include one or more of shape, texture, size and brightness.

5. The high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station according to claim 1 is characterized in that: Before extracting the rain line from the video captured by the surveillance camera in the area, the method further includes: Each rain-containing image in the video is converted into an HSV image.

6. The high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station according to claim 1 is characterized in that: Determining the size and number of raindrops corresponding to the rain line includes: Determine the length and width of the rain line according to the camera imaging principle; Calculating the ratio between the length and width of the rain line; When the ratio is greater than a preset threshold, the size and number of raindrops corresponding to the rain line are determined.

7. The high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station according to claim 6 is characterized in that: The length L and width D of the rain line are determined by the following formula according to the camera imaging principle: ； ； in, is the focal length, which refers to the distance between the lens and the focus where parallel light converges; Refers to the focusing distance, which is the distance from the mirror plane when the object image is clearest; is the object distance of the raindrop, that is, the distance from the raindrop to the lens; and Respectively represent the vertical and horizontal dimensions of the camera target surface; and Respectively represent the vertical and horizontal dimensions of the displayed video; and Represent the vertical and horizontal dimensions of the rain line respectively.

8. The high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station according to claim 1, characterized in that: Determining the size and number of raindrops corresponding to the rain line includes: According to the camera imaging principle, the relationship between the length L of the raindrop and the falling speed v is: ； Physics provides the following formula for calculating the falling speed of raindrops near the ground: ； in, is the exposure time of the monitoring camera, D is the width of the rain line, and the above formulas are solved to obtain the size of the raindrops corresponding to the rain line, and the number of the raindrops is obtained according to the size of the rain line and the size of the raindrops.

9. The high-resolution observation method for surface precipitation by integrating a monitoring camera with a meteorological station according to claim 1, characterized in that: The step of fusing the rainfall intensity value corresponding to the monitoring camera with the rainfall intensity value observed by the meteorological station in the area to obtain a spatiotemporal distribution diagram of the rainfall intensity value in the area includes: Aligning the observation time of the monitoring camera and the weather station, and unifying the resolution of the observation time of the monitoring camera and the weather station; The Kriging interpolation model is used to interpolate the rainfall intensity values ​​observed by the monitoring camera and the meteorological station to obtain a spatiotemporal distribution map of the rainfall intensity values ​​in the area.

10. A high-resolution observation device for surface precipitation integrating a monitoring camera and a weather station, characterized in that: include: An extraction module, used to extract rain lines from the video captured by the surveillance camera in the area, and determine the size and number of raindrops corresponding to the rain lines; A prediction module, used to determine the rainfall intensity value corresponding to the monitoring camera according to the size and number of the raindrops; The fusion module is used to fuse the rainfall intensity value corresponding to the monitoring camera with the rainfall intensity value observed by the meteorological station in the area to obtain a spatiotemporal distribution map of the rainfall intensity value in the area.

Images