Rainfall monitoring methods, devices, computer equipment and storage media

Abstract

The present invention discloses a rainfall monitoring method, device, computer equipment and storage medium, and belongs to the field of image processing. The rainfall monitoring method of the present invention can support large-scale deployment. In the initial stage, a collection unit pre-installed in each target area can be used to obtain multiple first images of each target area according to a preset frequency, so as to calculate the index average of the first image of each target area and generate a reference image corresponding to the target area; in the monitoring stage, the collection unit is used to collect the second image of each target area in real time, and the distortion data of the second image and the reference image are input into a neural network model to calculate the rainfall of the target area, thereby achieving the purpose of real-time and accurate acquisition of the rainfall of each target area.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image-based rainfall monitoring method, device, computer equipment and storage medium. Background Art

[0002] At present, the tipping bucket rain gauge is mainly used for real-time rainfall observation. The tipping bucket rain gauge is a commonly used measurement technology, which has the advantages of accurate time, automatic recording and convenient data collection. However, since the tipping bucket rain gauge adopts a mechanical working principle, when the internal tipping bucket device or sensor device is damaged, or short-circuited, it is easy to cause abnormal precipitation measurement results. When it rains, even in the same city, the rainfall in different areas is different, and the tipping bucket rain gauge cannot be deployed densely over a large area. Therefore, it is impossible to measure the rainfall in each area accurately in real time. Summary of the invention

[0003] In view of the problem that the existing tipping bucket rain gauge is not suitable for large-scale intensive deployment, a rainfall monitoring method, device, computer equipment and storage medium which are designed to be deployed over a large area are provided.

[0004] To achieve the above object, the present invention provides a rainfall monitoring method, comprising: an initial stage and a monitoring stage;

[0005] In the initial stage, a preset number of first images of each target area are obtained, and an average value of an indicator is calculated according to the preset number of first images of each target area to generate a reference image, and each target area corresponds to one reference image;

[0006] During the monitoring phase, the second image of each target area is acquired, the distortion data of each second image is extracted respectively, and the rainfall of each target area is calculated respectively based on the reference image of each target area and the distortion data of the second image.

[0007] Optionally, in the initial stage, obtaining a preset number of first images of each target area, and calculating an average value of an indicator according to the preset number of first images of each target area to generate a reference image, includes:

[0008] In the initial stage, a collection unit pre-installed in each target area collects a preset number of first images of each target area according to a preset frequency, and calculates the average value of the index of the preset number of first images of each target area;

[0009] A reference image is generated according to the average value of the index.

[0010] Optionally, the indicator averages include: brightness average, grayscale average, RGB average and contrast average;

[0011] The step of generating a reference image according to the average value of the indicator comprises:

[0012] The reference image is generated based on the first image according to the brightness average value, the grayscale average value, the RGB average value, and the contrast average value.

[0013] Optionally, in the monitoring stage, obtaining the second image of each target area, extracting the distortion data of each second image respectively, and calculating the rainfall of each target area respectively according to the reference image of each target area and the distortion data of the second image respectively, comprises:

[0014] Acquire a second image of each target area acquired by the acquisition unit during the monitoring phase;

[0015] respectively extracting the distortion data of each of the second images;

[0016] A neural network model is used to calculate the rainfall of each target area according to the reference image of each target area and the distorted data of the second image.

[0017] Optionally, the step of respectively extracting the distortion data of each second image further includes:

[0018] An anti-shake algorithm is used to correct each of the second images to obtain a corrected second image.

[0019] Optionally, the distorted data includes a line graph and a rain layer;

[0020] Extracting the distortion data of the second image includes:

[0021] identifying whether the second image includes a dynamic target;

[0022] If the second image includes a dynamic target, remove the dynamic target in the second image, and fill the area where the dynamic target is removed to obtain a third image;

[0023] sequentially performing sharpening, frequency conversion and spectrum analysis on the third image to extract the line graph;

[0024] The third image is compared with the reference image to extract the rainwater layer.

[0025] Optionally, also include:

[0026] If the second image does not include a dynamic target, sequentially performing sharpening, frequency conversion and spectrum analysis on the second image to extract the line graph;

[0027] The second image is compared with the reference image to extract the rainwater layer.

[0028] To achieve the above object, the present invention further provides a rainfall monitoring device, comprising:

[0029] A generating component, used for obtaining a preset number of first images of each target area in an initial stage, calculating an average value of an indicator according to the preset number of first images of each target area to generate a reference image, wherein each target area corresponds to one reference image;

[0030] The monitoring component is used to obtain the second image of each target area during the monitoring stage, extract the distortion data of each second image respectively, and calculate the rainfall of each target area according to the reference image of each target area and the distortion data of the second image respectively.

[0031] To achieve the above objectives, the present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0032] To achieve the above object, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0033] The rainfall monitoring method, device, computer equipment and storage medium provided by the present invention can support large-scale deployment. In the initial stage, multiple first images of each target area are obtained to calculate the index average value of the first image of each target area and generate a reference image corresponding to the target area. In the monitoring stage, the second image of each target area can be collected in real time, and the rainfall of the target area is calculated based on the distortion data of the second image and the reference image, so as to achieve the purpose of real-time and accurate acquisition of the rainfall of each target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of an embodiment of the rainfall monitoring method of the present invention;

[0035] Figure 2 A flowchart of an embodiment of extracting distortion data of a second image according to the present invention;

[0036] Figure 3 A module diagram of an embodiment of the rainfall monitoring device of the present invention;

[0037] Figure 4 A schematic diagram of the hardware architecture of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0039] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0040] The rainfall monitoring method, device, computer equipment and storage medium provided by the present invention are applicable to the field of smart cities and involve face recognition technology. The present invention can support large-scale deployment. In the initial stage, the acquisition unit pre-installed in each target area can obtain multiple first images of each target area according to a preset frequency, so as to calculate the index average of the first image of each target area and generate a reference image corresponding to the target area; in the monitoring stage, the acquisition unit is used to collect the second image of each target area in real time, and the distorted data of the second image and the reference image are input into the neural network model to calculate the rainfall of the target area, thereby achieving the purpose of real-time and accurate acquisition of the rainfall of each target area.

[0041] Embodiment 1

[0042] See also Figure 1 , a rainfall monitoring method of this embodiment includes an initial stage and a monitoring stage;

[0043] In the initial stage, a preset number of first images of each target area are obtained, and an index average value is calculated according to the preset number of first images of each target area to generate a reference image, and each target area corresponds to one reference image, including the following steps:

[0044] S1. In the initial stage, a collection unit pre-installed in each target area collects a preset number of first images of each target area according to a preset frequency, and calculates the average index value of the preset number of first images of each target area.

[0045] Wherein, the first image is an image acquired by an acquisition unit of a target area. The index average values ​​include: brightness average value, grayscale average value, RGB average value and contrast average value.

[0046] In this embodiment, the acquisition unit may be a camera.

[0047] At present, cameras have been widely and densely deployed in various areas of the city as part of the urban construction infrastructure. In practical applications, the first image of each target area can be collected by cameras distributed in various corners of the city (also known as the "eyes of the city"). For areas with sparse camera coverage, cameras can be installed on street lamps. Installing cameras on street lamps can solve the power supply problem of the camera and the light problem of shooting at night.

[0048] As an example but not limitation, the acquisition unit can be controlled according to the acquisition instruction to acquire a first image of the target area every 2 hours during the day, and 12 first images can be obtained. Based on the acquired first images, the brightness average, grayscale average, RGB average and contrast average are calculated respectively.

[0049] In this embodiment, before calculating the average value of the index, the following may be included: identifying whether each first image includes a dynamic target one by one, if a dynamic target is included, removing the dynamic target in the first image to avoid interference of the dynamic target on subsequent image processing, and filling the area where the dynamic target is removed to obtain a processed first image; sequentially sharpening, frequency conversion and spectrum analysis of the processed first image to extract a line graph, calculating the distortion of the line graph, judging whether the distortion is within the distortion threshold range, if so, it means that the current weather is not raining, calculating the average value of the index of the multiple processed first images, and executing step S2. If not, it means that the current weather is raining, and the reference image cannot be generated based on the first image, and a prompt message may be generated, and the process ends.

[0050] The above-mentioned dynamic targets refer to moving objects such as pedestrians and vehicles.

[0051] S2. Generate a reference image according to the average value of the indicator.

[0052] It should be noted that: each of the target areas corresponds to one of the reference images, and the reference image is an image when there is no rainfall.

[0053] Further, step S2 includes: generating the reference image based on the first image according to the brightness average value, the grayscale average value, the RGB average value and the contrast average value.

[0054] In this embodiment, considering that the acquisition angle of the acquisition unit does not change for the target area, the range and content of the image collected by the acquisition unit every day are basically fixed. The images collected at different times every day will have differences due to changes in light and visibility. In the case of differences, the difference between the images can be reduced by calculating the average index value of multiple images at different times, so as to generate a reference image that can reflect the optimal balance state of the target area according to the calculated average index value.

[0055] In the monitoring stage, the second image of each target area is obtained, the distortion data of each second image is extracted respectively, and the rainfall of each target area is calculated respectively according to the reference image of each target area and the distortion data of the second image, including the following steps:

[0056] S3. Acquire a second image of each target area acquired by the acquisition unit during the monitoring phase.

[0057] In this embodiment, each target area is associated with a collection unit, and the second image collected by the same collection unit can be identified, and the corresponding collection unit can be identified based on the image identification. In practical applications, the second image collected by the collection unit can be uploaded to the cloud for storage through a data transmission network.

[0058] S4. Extract the distortion data of each of the second images respectively.

[0059] Among them, the distorted data include: line graph and rainwater layer.

[0060] Considering that the weather is mainly divided into: sunny, rainy, snowy, hail, haze and sandstorm. The image collected in the weather state of haze or sandstorm will produce a blur effect, and the shape of the main lines of the image (such as: road edge lines, building edge lines) will not change significantly; the image collected in the weather state of snowfall or hail will produce occlusion noise, making the main lines discretized; the image quality and main lines of the image collected in the weather state of sunny days will not change significantly; the image collected in the weather state of rain will cause the interference of rain to refract the light, and the main lines of the image will be distorted and deformed, and the main lines will change continuously. It can be seen that there is no need to calculate the rainfall in the weather state of snowfall, hail, haze, sandstorm and sunny days. In this embodiment, it can be judged whether the current weather is raining based on whether the main lines in the image are distorted and deformed.

[0061] Further, see Figure 2 Extracting the distortion data of a second image in step S4 may include the following steps:

[0062] S41. Identify whether the second image includes a dynamic target. If so, execute step S42; if not, execute step S45.

[0063] In this embodiment, in order to avoid the interference of dynamic targets on the subsequent calculation of rainfall, image recognition technology can be used to identify whether there are dynamic targets in the second image. If so, step S42 needs to be executed to remove the dynamic targets; if not, step 45 can be executed to extract distorted data.

[0064] It should be noted that the above-mentioned dynamic targets refer to moving objects such as pedestrians and vehicles.

[0065] S42. Remove the dynamic object in the second image, and fill the area where the dynamic object is removed to obtain a third image.

[0066] In this embodiment, the Bayesian cutout algorithm is used to process the second image to extract the dynamic target therein, and blurring and feathering operations can be performed on the cutout edge. In the filling process, in order to ensure the correct evolution of the direction field, anisotropic diffusion can be used to prevent the boundary from being flattened. The image is first decomposed to obtain the structure, texture and noise of the image, and the Laplace operator (smooth function of the image) is used to fill the image.

[0067] S43. Perform sharpening, frequency conversion and spectrum analysis on the third image in sequence to extract the line graph.

[0068] The contours (edges) and lines of objects (buildings, roads, etc.) in the image are the sudden changes in grayscale in the image, and contain rich spatial high-frequency components. The contours of the third image are compensated by sharpening processing, and the edges and grayscale jump parts of the third image are increased to make the image clear. The domain of the image after sharpening processing is changed to the frequency domain, and spectrum analysis (frequency domain analysis) is performed. The lines are continuous in the time domain, and the gradient changes sharply (mutated) at the edge of the spectrum (frequency domain). The lines are extracted according to the changes in the spectrum to obtain a line graph.

[0069] S44. Compare the third image with the reference image to extract the rainwater layer, and end.

[0070] In this embodiment, since the reference image is an image in a non-raining state, the collected third image can be compared with the reference image. Noise filtering is performed on the third image to extract noise data caused by rain from the image to generate a rain layer.

[0071] In this embodiment, the rain layer can also be extracted through a deep convolutional neural network, and the rain appears in the form of dots or lines in the image.

[0072] S45. Perform sharpening, frequency conversion and spectrum analysis on the second image in sequence to extract the line graph.

[0073] In this embodiment, the contour of the second image is compensated by sharpening processing, and the edge and grayscale jump part of the second image are increased to make the image clear. The domain of the image after sharpening processing is changed to the frequency domain, and spectrum analysis is performed. In the time domain, the lines are continuous, and the gradient changes sharply at the edge of the spectrum. According to the change of the spectrum, the lines are extracted to obtain a line graph.

[0074] S46. Compare the second image with the reference image to extract the rainwater layer, and end.

[0075] In this embodiment, since the reference image is an image in a non-raining state, the collected second image can be compared with the reference image, and noise filtering is performed on the second image to extract noise data caused by rain from the image to generate a rain layer.

[0076] In this embodiment, the rain layer can also be extracted through a deep convolutional neural network, and the rain appears in the form of dots or lines in the image.

[0077] In a preferred embodiment, before executing step S4, the method may further include: using an anti-shake algorithm to correct each of the second images respectively to obtain a corrected second image.

[0078] In this embodiment, the shaking caused by wind is removed by anti-shake algorithm. The size and direction of the shaking can be detected by comparing multiple frames of images, and the image can be translated or rotated to obtain a stable image.

[0079] In this embodiment, a diffusion dithering algorithm can also be used. The algorithm uses error diffusion to achieve dithering, scans the pixels of the image from left to right and from top to bottom and standardizes them one by one (or binarizes them), and superimposes the errors generated after pixel standardization on adjacent pixels without affecting the pixels that have been processed. If some pixels are rounded down, the next pixel is more likely to be rounded up, which minimizes the average quantization error. The image processed by the diffusion dithering algorithm is more delicate, less distorted, and rich in details.

[0080] S5. Using a neural network model, respectively calculate the rainfall in each target area according to the reference image of each target area and the distorted data of the second image.

[0081] In this embodiment, the higher the density of points and lines in the rainwater layer, the greater the rainfall; the lower the density, the smaller the rainfall. Since rainwater refracts light, the higher the degree of distortion of the camera-collected image compared with the reference image, the greater the rainfall; the smaller the degree of distortion, the smaller the rainfall. The neural network model extracts the features of the line graph and the features of the rainwater layer, extracts the reference features of the reference image, and compares the extracted features with the reference features to determine the rainfall. The neural network model calculates the degree of distortion and deformation of the main lines of the camera-collected image compared with the reference image.

[0082] In a preferred embodiment, before executing step S5, the following steps may also be included:

[0083] A. Using the sample set to train the initial neural network model to obtain the neural network model used to represent the mapping relationship between the distorted data and the rainfall.

[0084] The sample set includes historical rainfall data and distorted data corresponding to each historical rainfall data.

[0085] In this embodiment, the neural network model is used to represent the correspondence between the distorted data and the rainfall. The historical distorted data in the historical image data collected by the camera is extracted, and the historical distorted data and the reference image are output to the initial neural network model for calculation in the training stage. The loss value is calculated based on the calculation result and the historical rainfall data released by the meteorological center, and the gradient is back-propagated to the parameters of the neural network. The weight value in the network is updated until the preset number of iterations is reached, and the training is terminated to obtain the neural network model. The use of the neural network model can improve the accuracy of calculating rainfall. In this embodiment, the neural network model can adopt a deep convolutional neural network model.

[0086] In this embodiment, the rainfall monitoring method can support large-scale deployment. In the initial stage, a plurality of first images of each target area can be obtained according to a preset frequency through a collection unit pre-installed in each target area, so as to calculate the index average of the first image of each target area and generate a reference image corresponding to the target area; in the monitoring stage, the second image of each target area is collected in real time by the collection unit, and the distorted data of the second image and the reference image are input into the neural network model to calculate the rainfall in the target area, thereby achieving the purpose of real-time and accurate acquisition of the rainfall in each target area, so as to provide an early warning when the rainfall suddenly increases.

[0087] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0088] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0089] In a preferred embodiment, the rainfall monitoring method may further include:

[0090] S6. Generate a travel path according to the rainfall in each of the target areas and the route map of all the target areas.

[0091] In this embodiment, the rainfall in each target area is matched with the preset threshold interval. If it matches, it means that the rainfall is within the normal range and pedestrians and vehicles can pass normally. If it does not match, it means that the rainfall is very heavy and the target area is not suitable for passage. Pedestrians and vehicles cannot pass smoothly and need to detour. Therefore, the corresponding passage path can be generated according to the rainfall situation of each target area to facilitate the normal passage of pedestrians and vehicles.

[0092] In one embodiment, the rainfall monitoring method may further include:

[0093] S7. Compare the second image with the reference image, determine difference data, and identify the actual area corresponding to the difference data.

[0094] Among them, the difference data is the image of the road waterlogging area.

[0095] In this embodiment, the road waterlogging identification model may be used to identify the road waterlogging area image to obtain the actual area and water depth of the road waterlogging area.

[0096] In this embodiment, the rainfall can be monitored in real time by the rainfall monitoring method. In the case of heavy rainfall causing a large amount of water on the road, a reasonable transfer route can be planned to protect people's safety.

[0097] Embodiment 2

[0098] See also Figure 3 , a rainfall monitoring device 1 of this embodiment includes a generating component and a monitoring component.

[0099] The generating component is used to obtain a preset number of first images of each target area in the initial stage, calculate the index average value according to the preset number of first images of each target area to generate a reference image, and each target area corresponds to one reference image.

[0100] The monitoring component is used to obtain the second image of each target area during the monitoring stage, extract the distortion data of each second image respectively, and calculate the rainfall of each target area according to the reference image of each target area and the distortion data of the second image respectively.

[0101] Furthermore, the generating component may include: a calculating unit 11 and a generating unit 12 ; the monitoring component may include a receiving unit 13 , an extracting unit 14 and a processing unit 15 .

[0102] The calculation unit 11 is used to collect a preset number of first images of each target area according to a preset frequency through a collection unit pre-installed in each target area in the initial stage, and calculate the index average value of the preset number of first images of each target area.

[0103] Wherein, the first image is an image acquired by an acquisition unit of a target area. The index average values ​​include: brightness average value, grayscale average value, RGB average value and contrast average value.

[0104] In this embodiment, the acquisition unit may be a camera.

[0105] At present, cameras have been widely and densely deployed in various areas of the city as part of the urban construction infrastructure. In practical applications, the first image of each target area can be collected by cameras distributed in various corners of the city (also known as the "eyes of the city"). For areas with sparse camera coverage, cameras can be installed on street lamps. Installing cameras on street lamps can solve the power supply problem of the camera and the light problem of shooting at night.

[0106] As an example but not limitation, the acquisition unit can be controlled according to the acquisition instruction to acquire a first image of the target area every 2 hours during the day, and 12 first images can be obtained. Based on the acquired first images, the brightness average, grayscale average, RGB average and contrast average are calculated respectively.

[0107] In this embodiment, before calculating the average index value, the following may be included: identifying whether each first image includes a dynamic target one by one, if a dynamic target is included, removing the dynamic target in the first image to avoid interference of the dynamic target on subsequent image processing, and filling the area where the dynamic target is removed to obtain a processed first image; sequentially sharpening, frequency conversion and spectrum analysis of the processed first image to extract a line graph, calculating the distortion of the line graph, judging whether the distortion is within the distortion threshold range, if so, it means that the current weather is not raining, calculating the average index value of the processed multiple first images, and generating a reference image based on the average index value. If not, it means that the current weather is raining, and the reference image cannot be generated based on the first image, and a prompt message may be generated, and the process ends.

[0108] The above-mentioned dynamic targets refer to moving objects such as pedestrians and vehicles.

[0109] The generating unit 12 is used to generate a reference image according to the indicator average value.

[0110] It should be noted that: each of the target areas corresponds to one of the reference images, and the reference image is an image when there is no rainfall.

[0111] Further, the generating unit 12 is used to generate the reference image based on the first image according to the brightness average value, the grayscale average value, the RGB average value and the contrast average value.

[0112] In this embodiment, considering that the acquisition angle of the acquisition unit does not change for the target area, the range and content of the image collected by the acquisition unit every day are basically fixed. The images collected at different times every day will have differences due to changes in light and visibility. In the case of differences, the difference between the images can be reduced by calculating the average index value of multiple images at different times, so as to generate a reference image that can reflect the optimal balance state of the target area according to the calculated average index value.

[0113] The receiving unit 13 is used to receive the second image acquired by the acquisition unit for each target area during the monitoring phase.

[0114] In this embodiment, each target area is associated with a collection unit, and the second image collected by the same collection unit can be identified, and the corresponding collection unit can be identified based on the image identification. In practical applications, the second image collected by the collection unit can be uploaded to the cloud for storage through a data transmission network.

[0115] The extraction unit 14 is used to extract the distortion data of each second image respectively.

[0116] Among them, the distorted data include: line graph and rainwater layer.

[0117] Considering that the weather is mainly divided into: sunny, rainy, snowy, hail, haze and sandstorm. The image collected in the weather state of haze or sandstorm will produce a blur effect, and the shape of the main lines of the image (such as: road edge lines, building edge lines) will not change significantly; the image collected in the weather state of snowfall or hail will produce occlusion noise, making the main lines discretized; the image quality and main lines of the image collected in the weather state of sunny days will not change significantly; the image collected in the weather state of rain will cause the interference of rain to refract the light, and the main lines of the image will be distorted and deformed, and the main lines will change continuously. It can be seen that there is no need to calculate the rainfall in the weather state of snowfall, hail, haze, sandstorm and sunny days. In this embodiment, it can be judged whether the current weather is raining based on whether the main lines in the image are distorted and deformed.

[0118] Furthermore, the extraction unit 14 may include: a recognition module, a filling module, a first extraction module and a second extraction module.

[0119] The recognition module is used to recognize whether the second image includes a dynamic target.

[0120] In this embodiment, in order to avoid the interference of dynamic targets on the subsequent calculation of rainfall, image recognition technology can be used to identify whether there are dynamic targets in the second image. If so, the dynamic targets need to be removed; if not, the distorted data can be extracted.

[0121] It should be noted that the above-mentioned dynamic targets refer to moving objects such as pedestrians and vehicles.

[0122] The filling module is used to remove the dynamic target in the second image when the second image includes the dynamic target, and fill the area where the dynamic target is removed to obtain the third image.

[0123] In this embodiment, the Bayesian cutout algorithm is used to process the second image to extract the dynamic target therein, and blurring and feathering operations can be performed on the cutout edge. In the filling process, in order to ensure the correct evolution of the direction field, anisotropic diffusion can be used to prevent the boundary from being flattened. The image is first decomposed to obtain the structure, texture and noise of the image, and the Laplace operator (smooth function of the image) is used to fill the image.

[0124] The first extraction module is used to sequentially perform sharpening, frequency conversion and spectrum analysis on the third image to extract the line graph.

[0125] The contours (edges) and lines of objects (buildings, roads, etc.) in the image are the sudden changes in grayscale in the image, and contain rich spatial high-frequency components. The contours of the third image are compensated by sharpening processing, and the edges and grayscale jump parts of the third image are increased to make the image clear. The domain of the image after sharpening processing is changed to the frequency domain, and spectrum analysis (frequency domain analysis) is performed. The lines are continuous in the time domain, and the gradient changes sharply (mutated) at the edge of the spectrum (frequency domain). The lines are extracted according to the changes in the spectrum to obtain a line graph.

[0126] The second extraction module is used to compare the third image with the reference image to extract the rainwater layer.

[0127] In this embodiment, since the reference image is an image in a non-raining state, the collected third image can be compared with the reference image. Noise filtering is performed on the third image to extract noise data caused by rain from the image to generate a rain layer.

[0128] In this embodiment, the rain layer can also be extracted through a deep convolutional neural network, and the rain appears in the form of dots or lines in the image.

[0129] When the second image does not include a dynamic target, the first extraction module is further used to sequentially perform sharpening, frequency conversion and spectrum analysis on the second image to extract the line graph.

[0130] In this embodiment, the contour of the second image is compensated by sharpening processing, and the edge and grayscale jump part of the second image are increased to make the image clear. The domain of the image after sharpening processing is changed to the frequency domain, and spectrum analysis is performed. In the time domain, the lines are continuous, and the gradient changes sharply at the edge of the spectrum. According to the change of the spectrum, the lines are extracted to obtain a line graph.

[0131] When the second image does not include a dynamic target, the second extraction module is further used to compare the second image with the reference image to extract the rainwater layer.

[0132] In this embodiment, since the reference image is an image in a non-raining state, the collected second image can be compared with the reference image, and noise filtering is performed on the second image to extract noise data caused by rain from the image to generate a rain layer.

[0133] In this embodiment, the rain layer can also be extracted through a deep convolutional neural network, and the rain appears in the form of dots or lines in the image.

[0134] In a preferred embodiment, before extracting the distorted data of each second image by using the extracting unit 14, the method may further include: correcting each second image by using an anti-shake algorithm to obtain a corrected second image.

[0135] In this embodiment, the shaking caused by wind is removed by anti-shake algorithm. The size and direction of the shaking can be detected by comparing multiple frames of images, and the image can be translated or rotated to obtain a stable image.

[0136] In this embodiment, a diffusion dithering algorithm can also be used. The algorithm uses error diffusion to achieve dithering, scans the pixels of the image from left to right and from top to bottom and standardizes them one by one (or binarizes them), and superimposes the errors generated after pixel standardization on adjacent pixels without affecting the pixels that have been processed. If some pixels are rounded down, the next pixel is more likely to be rounded up, which minimizes the average quantization error. The image processed by the diffusion dithering algorithm is more delicate, less distorted, and rich in details.

[0137] The processing unit 15 is used to respectively calculate the rainfall of each target area according to the reference image of each target area and the distorted data of the second image using a neural network model.

[0138] In this embodiment, the higher the density of points and lines in the rainwater layer, the greater the rainfall; the lower the density, the smaller the rainfall. Since rainwater refracts light, the higher the degree of distortion of the camera-collected image compared with the reference image, the greater the rainfall; the smaller the degree of distortion, the smaller the rainfall. The neural network model extracts the features of the line graph and the features of the rainwater layer, extracts the reference features of the reference image, and compares the extracted features with the reference features to determine the rainfall. The neural network model calculates the degree of distortion and deformation of the main lines of the camera-collected image compared with the reference image.

[0139] In a preferred embodiment, the rainfall monitoring device 1 may further include:

[0140] The training unit is used to train the initial neural network model using the sample set to obtain the neural network model used to represent the mapping relationship between the distorted data and the rainfall.

[0141] The sample set includes historical rainfall data and distorted data corresponding to each historical rainfall data.

[0142] In this embodiment, the neural network model is used to represent the correspondence between the distorted data and the rainfall. The historical distorted data in the historical image data collected by the camera is extracted, and the historical distorted data and the reference image are output to the initial neural network model for calculation in the training stage. The loss value is calculated based on the calculation result and the historical rainfall data released by the meteorological center, and the gradient is back-propagated to the parameters of the neural network. The weight value in the network is updated until the preset number of iterations is reached, and the training is terminated to obtain the neural network model. The use of the neural network model can improve the accuracy of calculating rainfall. In this embodiment, the neural network model can adopt a deep convolutional neural network model.

[0143] In this embodiment, the rainfall monitoring device 1 can support large-scale deployment. In the initial stage, the acquisition unit pre-installed in each target area can obtain multiple first images of each target area according to a preset frequency, so as to calculate the index average of the first image of each target area and generate a reference image corresponding to the target area; in the monitoring stage, the acquisition unit is used to collect the second image of each target area in real time, and the distorted data of the second image and the reference image are input into the neural network model to calculate the rainfall in the target area, thereby achieving the purpose of real-time and accurate acquisition of the rainfall in each target area, so as to give an early warning when the rainfall suddenly increases.

[0144] In a preferred embodiment, the rainfall monitoring device 1 may further include:

[0145] A path generation unit is used to generate a travel path according to the rainfall in each of the target areas and the route map of all the target areas.

[0146] In this embodiment, the rainfall in each target area is matched with the preset threshold interval. If it matches, it means that the rainfall is within the normal range and pedestrians and vehicles can pass normally. If it does not match, it means that the rainfall is very heavy and the target area is not suitable for passage. Pedestrians and vehicles cannot pass smoothly and need to detour. Therefore, the corresponding passage path can be generated according to the rainfall situation of each target area to facilitate the normal passage of pedestrians and vehicles.

[0147] In one embodiment, the rainfall monitoring device 1 may further include:

[0148] The recognition unit is used to compare the second image with the reference image, determine difference data, and recognize the actual area corresponding to the difference data.

[0149] Among them, the difference data is the image of the road waterlogging area.

[0150] In this embodiment, the road waterlogging identification model may be used to identify the road waterlogging area image to obtain the actual area and water depth of the road waterlogging area.

[0151] In this embodiment, the rainfall can be monitored in real time by the rainfall monitoring device 1. When a large amount of rainfall causes a large amount of water on the road, a reasonable transfer route can be planned to protect people's safety.

[0152] Embodiment 3

[0153] To achieve the above purpose, the present invention also provides a computer device 2, which includes multiple computer devices 2. The components of the rainfall monitoring device 1 of the second embodiment can be dispersed in different computer devices 2. The computer device 2 can be a smart phone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server or cabinet server (including independent servers, or server clusters composed of multiple servers) that execute programs. The computer device 2 of this embodiment includes at least but is not limited to: a memory 21, a processor 23, a network interface 22 and a rainfall monitoring device 1 (refer to Figure 4 ). It should be pointed out that Figure 4 Only a computer device 2 having components is shown, but it should be understood that implementing all of the components shown is not a requirement and more or fewer components may alternatively be implemented.

[0154] In this embodiment, the memory 21 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 21 can be an internal storage unit of the computer device 2, such as a hard disk or memory of the computer device 2. In other embodiments, the memory 21 can also be an external storage device of the computer device 2, such as a plug-in hard disk equipped on the computer device 2, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 21 can also include both the internal storage unit of the computer device 2 and its external storage device. In this embodiment, the memory 21 is generally used to store the operating system and various application software installed on the computer device 2, such as the program code of the rainfall monitoring method of embodiment 1. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or are to be output.

[0155] The processor 23 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 23 is generally used to control the overall operation of the computer device 2, such as executing control and processing related to data interaction or communication with the computer device 2. In this embodiment, the processor 23 is used to run the program code stored in the memory 21 or process data, such as running the rainfall monitoring device 1.

[0156] The network interface 22 may include a wireless network interface or a wired network interface, and the network interface 22 is generally used to establish a communication connection between the computer device 2 and other computer devices 2. For example, the network interface 22 is used to connect the computer device 2 to an external terminal through a network, and to establish a data transmission channel and a communication connection between the computer device 2 and the external terminal. The network may be a wireless or wired network such as an Intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, or Wi-Fi.

[0157] It should be pointed out that Figure 4 Only computer device 2 is shown with components 21 - 23 , but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0158] In this embodiment, the rainfall monitoring device 1 stored in the memory 21 can also be divided into one or more program modules, and the one or more program modules are stored in the memory 21 and executed by one or more processors (processor 23 in this embodiment) to complete the present invention.

[0159] Embodiment 4

[0160] To achieve the above purpose, the present invention also provides a computer-readable storage medium, which includes multiple storage media, such as flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, CD, server, App application mall, etc., on which a computer program is stored, and the program realizes the corresponding function when executed by the processor 23. The computer-readable storage medium of this embodiment is used to store the rainfall monitoring device 1, and when executed by the processor 23, the rainfall monitoring method of embodiment 1 is realized.

[0161] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0163] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A rainfall monitoring method, characterized in that: include: Initial and monitoring phases; In the initial stage, a preset number of first images of each target area are acquired, and an index average value is calculated based on the preset number of first images of each target area to generate a benchmark image, and each target area corresponds to one benchmark image; wherein, it includes: in the initial stage, a collection unit pre-installed in each target area is used to collect a preset number of first images of each target area according to a preset frequency, and an index average value of the preset number of first images of each target area is calculated; a benchmark image is generated based on the index average value; the index average value includes: brightness average value, grayscale average value, RGB average value and contrast average value; generating a benchmark image based on the index average value includes: generating the benchmark image based on the brightness average value, the grayscale average value, the RGB average value and the contrast average value based on the first image; During the monitoring stage, a second image of each of the target areas is acquired, and the distortion data of each of the second images is extracted respectively. The rainfall of each of the target areas is calculated respectively based on the baseline image of each target area and the distortion data of the second image; wherein, the neural network model is used to extract the features of the line graph and the features of the rainwater layer, as well as the baseline features of the baseline image, and the extracted features are compared with the baseline features to determine the rainfall, and the distortion data of the second image and the baseline image are input into the neural network model to calculate the rainfall of the target area.

2. The rainfall monitoring method according to claim 1, characterized in that: In the monitoring stage, the second image of each target area is obtained, the distortion data of each second image is extracted respectively, and the rainfall of each target area is calculated respectively according to the reference image of each target area and the distortion data of the second image, including: Acquire a second image of each target area acquired by the acquisition unit during the monitoring phase; respectively extracting the distortion data of each of the second images; A neural network model is used to calculate the rainfall of each target area according to the reference image of each target area and the distorted data of the second image.

3. The rainfall monitoring method according to claim 2, characterized in that: The step of respectively extracting the distortion data of each of the second images further comprises: An anti-shake algorithm is used to correct each of the second images to obtain a corrected second image.

4. The rainfall monitoring method according to claim 2, characterized in that: The distorted data includes a line graph and a rain layer; Extracting the distortion data of the second image includes: identifying whether the second image includes a dynamic target; If the second image includes a dynamic target, remove the dynamic target in the second image, and fill the area where the dynamic target is removed to obtain a third image; sequentially performing sharpening, frequency conversion and spectrum analysis on the third image to extract the line graph; The third image is compared with the reference image to extract the rainwater layer.

5. The rainfall monitoring method according to claim 4, characterized in that: Also includes: If the second image does not include a dynamic target, sequentially performing sharpening, frequency conversion and spectrum analysis on the second image to extract the line graph; The second image is compared with the reference image to extract the rainwater layer.

6. A rainfall monitoring device, characterized in that: include: A generating component, used for acquiring a preset number of first images of each target area in an initial stage, calculating an index average value based on the preset number of first images of each target area to generate a reference image, and each target area corresponds to one reference image; wherein, it includes: acquiring a preset number of first images of each target area according to a preset frequency by means of a collecting unit pre-installed in each target area in the initial stage, and calculating an index average value of the preset number of first images of each target area; generating a reference image according to the index average value; the index average value includes: brightness average value, grayscale average value, RGB average value and contrast average value; generating a reference image according to the index average value includes: generating the reference image according to the brightness average value, the grayscale average value, the RGB average value and the contrast average value based on the first image; A monitoring component is used to obtain the second image of each target area during the monitoring stage, extract the distortion data of each second image respectively, and calculate the rainfall of each target area according to the baseline image of each target area and the distortion data of the second image respectively; wherein, the neural network model is used to extract the characteristics of the line graph and the characteristics of the rainwater layer, and the baseline characteristics of the baseline image, and the extracted characteristics are compared with the baseline characteristics to determine the rainfall, and the distortion data of the second image and the baseline image are input into the neural network model to calculate the rainfall of the target area.

7. A computer device, characterized in that: The computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images