Rainfall monitoring method and system, storage medium and electronic equipment

Abstract

The invention is suitable for the technical field of rainfall monitoring, and provides a rainfall monitoring method and system, a storage medium and electronic device.The rainfall monitoring method comprises the steps that a rainfall event flow acquired by a dynamic vision sensor is obtained in response, and a rainfall image is obtained through reconstruction based on the rainfall event flow; identifying rainfall intensity corresponding to rainfall occurring in the rainfall image; and under the condition that the rainfall intensity meets the alarm condition, alarm prompt information is generated according to the rainfall intensity and the rainfall occurrence time period, so that the technical problem that an existing rainfall monitoring method cannot accurately monitor rainfall data in real time to meet rainfall monitoring and early warning requirements is solved.

Description

Technical Field

[0001] The present application belongs to the technical field of rainfall monitoring, and more specifically, to a rainfall monitoring method, system, storage medium and electronic device. Background Art

[0002] Among the existing rainfall monitoring technologies, ground rain gauges, radar and satellite remote sensing rainfall, and rainfall measurement technology based on traditional cameras are generally used.

[0003] However, all of the above existing methods have some problems, which make it impossible to accurately and real-time monitor rainfall data to meet the needs of rainfall monitoring and early warning. For example, the observation points of ground rain gauges are unevenly distributed and limited in number. Most of them are offline, with high construction and maintenance costs, and it is difficult to obtain real-time collaborative, high-spatial-resolution rainfall data. Radar and satellite remote sensing rainfall hardware facilities and inversion rainfall intensity are insufficient, and the accuracy of rainfall measurement needs to be improved. They are not sensitive to heavy rainfall and cannot provide real-time and accurate data. The imaging effect of rainfall measurement technology based on traditional cameras decreases when the background changes clutteredly. It is difficult to capture raindrops in a heavy rainfall and white fog environment. The large amount of data makes it difficult to process online in real time, resulting in delayed transmission of results. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a rainfall monitoring method, system, storage medium and electronic device, aiming to solve the technical problem that the existing rainfall monitoring methods cannot accurately and real-time monitor rainfall data to meet the needs of rainfall monitoring and early warning.

[0005] To achieve the above object, according to a first aspect of the present application, a rainfall monitoring method is provided, the method comprising:

[0006] In response to acquiring a rainfall event stream collected by a dynamic visual sensor, reconstructing a rainfall image based on the rainfall event stream;

[0007] Identifying the rainfall intensity corresponding to the rainfall occurring in the rainfall image;

[0008] When the rainfall intensity meets the alarm condition, an alarm prompt message is generated according to the rainfall intensity and the rainfall occurrence period.

[0009] Optionally, in a possible implementation manner of the first aspect, the reconstructing a rainfall image based on the rainfall event stream includes:

[0010] Acquire event parameters of each rainfall event in the rainfall event stream, wherein the event parameters include: pixel coordinates, illumination change polarity and timestamp;

[0011] Determine the change process of each rainfall event over time according to the mapping relationship between the timestamp and the pixel coordinates and the polarity of the light change;

[0012] The rainfall image is reconstructed based on the change process of all the rainfall events over time.

[0013] Optionally, in a possible implementation manner of the first aspect, the identifying a rainfall intensity corresponding to the rainfall occurring in the rainfall image includes:

[0014] Acquire feature information contained in the rainfall image, wherein the feature information at least includes at least one of the following: background feature, noise feature and raindrop layer feature;

[0015] The feature information corresponding to the rainfall image is input into a pre-trained rainfall recognition model to obtain the rainfall intensity output by the rainfall recognition model, wherein the rainfall recognition model is obtained by training an initial neural network model using multiple groups of training data, and each group of the training data includes: sample feature information of a sample rainfall image, and a sample rainfall intensity corresponding to the sample feature information.

[0016] Optionally, in a possible implementation manner of the first aspect, the rainfall recognition model processes the feature information of the rainfall image as follows to obtain the rainfall intensity:

[0017] extracting a raindrop portion in the rainfall image based on feature information of the rainfall image;

[0018] Calculating the raindrop density, average raindrop particle size and final moving speed of the raindrop portion in the rainfall image;

[0019] The rainfall intensity is calculated based on the raindrop density, the average raindrop particle size and the raindrop end moving speed.

[0020] Optionally, in a possible implementation manner of the first aspect, extracting a raindrop part in the rainfall image based on feature information of the rainfall image includes:

[0021] Using an edge detection algorithm, extracting a raindrop approximation portion in the rainfall image based on feature information of the rainfall image;

[0022] Using an image gradient calculation algorithm to calculate image gradient values ​​of the approximate raindrop portion in the horizontal direction and the vertical direction;

[0023] Performing non-maximum suppression processing on the image gradient value to determine edge pixels of the approximate raindrop portion;

[0024] The raindrop portion in the rainfall image is determined according to the edge pixels of the approximate raindrop portion.

[0025] Optionally, in a possible implementation manner of the first aspect, determining the raindrop portion in the rainfall image according to the edge pixels approximating the raindrop portion includes:

[0026] In a case where the edge pixel is greater than a first edge threshold, determining to retain the edge pixel;

[0027] In the case where the edge pixel is less than a second edge threshold, determining to remove the edge pixel, wherein the first edge threshold is greater than the second edge threshold;

[0028] When the edge pixel is smaller than the first edge threshold and larger than the second edge threshold, determining whether to remove the edge pixel according to connection information between the edge pixel and the retained edge pixels;

[0029] The raindrop portion in the rainfall image is determined according to the retained edge pixels.

[0030] Optionally, in a possible implementation manner of the first aspect, calculating the final moving speed of the raindrop portion in the rainfall image includes:

[0031] Determine a pixel point at the raindrop end position of the raindrop part in the rainfall image;

[0032] Determine the motion trajectory, motion time and pixel displacement of the raindrop portion according to the pixel point at the end position of the raindrop;

[0033] The final moving speed of the raindrop portion in the rainfall image is calculated according to the movement trajectory, movement time and pixel displacement of the raindrop portion.

[0034] Optionally, in a possible implementation manner of the first aspect, the method further includes:

[0035] Acquiring measurement information of rain gauges distributed in a plurality of the areas;

[0036] The change process of the rainfall intensity is determined according to the change process of the measurement information.

[0037] Optionally, in a possible implementation manner of the first aspect, before identifying the rainfall intensity of the rainfall in the rainfall image, the method further includes:

[0038] The rainfall image is subjected to image preprocessing, wherein the image preprocessing includes at least one of the following: denoising processing and enhancement processing.

[0039] According to a second aspect of the present application, a rainfall monitoring system is provided, comprising:

[0040] Dynamic visual sensors are set at observation points to collect rainfall events and obtain rainfall event streams;

[0041] A processor is connected to the plurality of dynamic visual sensors and is used to reconstruct a rainfall image based on the rainfall event stream and identify the rainfall intensity corresponding to the rainfall occurring in the rainfall image. When the rainfall intensity meets the alarm condition, an alarm prompt message is generated according to the rainfall intensity and the rainfall occurrence period.

[0042] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

[0043] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the methods described above when executing the computer program.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the methods described above.

[0045] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes any one of the methods described in the first aspect.

[0046] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0047] The rainfall monitoring method provided by the present application reconstructs a rainfall image based on the rainfall event stream in response to the rainfall event stream acquired by the dynamic visual sensor; identifies the rainfall intensity corresponding to the rainfall occurring in the rainfall image; and generates an alarm prompt message based on the rainfall intensity and the rainfall occurrence period when the rainfall intensity meets the alarm condition. This enables timely provision of accurate rainfall information to relevant departments and personnel, making up for the defects of untimely data transmission of offline ground rain gauges and the difficulty of real-time online processing of traditional camera data, which leads to delayed transmission of results; and solves the technical problem that the existing rainfall monitoring method cannot accurately and real-time monitor rainfall data to meet the needs of rainfall monitoring and early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 is a flow chart of a rainfall monitoring method provided in an embodiment of the present application;

[0050] Figure 2 is a flow chart of an optional rainfall monitoring method provided in an embodiment of the present application;

[0051] Figure 3 is a flow chart of an optional rainfall monitoring method provided in an embodiment of the present application;

[0052] Figure 4 is a flow chart of an optional rainfall monitoring method provided in an embodiment of the present application;

[0053] Figure 5 is a flow chart of an optional rainfall monitoring method provided in an embodiment of the present application;

[0054] Figure 6 is a structural schematic diagram of a rainfall monitoring system provided in an embodiment of the present application;

[0055] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0057] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0058] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0059] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0060] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0061] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0062] First, some terms in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0063] Prewitt operator: It is an operator used for image edge detection. It detects edges by calculating the gradient of pixels in the image. It uses a specific template to perform convolution operations on the image to determine the horizontal and vertical gradient components, and then determines the edge position.

[0064] Laplace operator: an edge detection operator that detects edges by calculating the second-order derivative of an image. It is sensitive to grayscale changes in the image and can detect mutation areas and edges in the image.

[0065] Lucas-Kanade (LK) optical flow method: It is a method for calculating the motion of pixels in an image. It assumes that the motion of pixels is consistent within a small local area and determines the motion vector of the pixel by solving the optical flow equation.

[0066] Horn-Schunck optical flow algorithm: It is an optical flow calculation method that solves the optical flow field by minimizing the smoothness constraint of the optical flow and the constant brightness constraint of the image, and can obtain a more accurate optical flow estimate.

[0067] PCA-F l ow algorithm: It is a principal component analysis optical flow algorithm that uses principal component analysis to model and estimate optical flow, which can improve the accuracy and efficiency of optical flow calculation to a certain extent.

[0068] LeNet-5 convolutional neural network model: It is an early convolutional neural network model, mainly used for tasks such as handwritten digit recognition, and has structures such as convolutional layers, pooling layers, and fully connected layers.

[0069] Visual Geometry Group (VGG) series models: a group of deep convolutional neural network models known for their simple structure and good performance, usually with deeper network layers and small convolution kernels.

[0070] The ResNet (ResNet) series of models solves the gradient vanishing problem in deep neural network training by introducing residual connections. They can train very deep networks and achieve excellent performance.

[0071] The above is a brief introduction to the nouns involved in the embodiments of the present application, which will not be repeated below.

[0072] Among the existing rainfall monitoring technologies, ground rain gauges, radar and satellite remote sensing rainfall, and rainfall measurement technology based on traditional cameras are generally used.

[0073] First, a ground rain gauge is a device used to measure precipitation, usually placed on the ground to monitor and record the amount of rain. Rain gauges consist of various types of rain gauges, such as floating leak, tipping bucket or pressure rain gauges. They receive rain falling in the water collector and record it, and then transmit the data to a data center or monitoring center for analysis and use. However, the observation points of ground rain gauges are unevenly distributed and limited in number, and most of them are operated offline. The cost of building and maintaining a high-density rain gauge is high, so it is difficult to obtain real-time coordinated and high-spatial-resolution rainfall data.

[0074] Second, radar rain measurement is a method of measuring precipitation using radar technology. The radar system emits radio waves. When encountering raindrops, part of the energy will be reflected back to the radar receiver to form an echo. The precipitation is estimated based on the intensity and reflectivity of the echo. Satellite remote sensing rain measurement uses remote sensing sensors on satellites to estimate precipitation. Satellite sensors receive microwave or infrared radiation signals reflected from the ground and the atmosphere, and process and analyze the received signals to infer cloud characteristics and precipitation possibilities. However, radar and satellite remote sensing rain measurement have deficiencies in hardware facilities and inversion of rainfall intensity. The accuracy of rain measurement needs to be further improved, and the observation of heavy rainfall is not sensitive enough, making it difficult to provide real-time and accurate rainfall data.

[0075] Third, with the development of optical technology, rainfall measurement methods based on optical sensors have emerged and gradually developed, mainly using optical detection technologies such as traditional frame cameras to observe precipitation. However, when the background is cluttered and changing, the imaging effect of raindrops based on traditional frame cameras will decrease, resulting in reduced algorithm effect. It is difficult to capture raindrops in the white fog environment caused by heavy rainfall. In addition, the amount of data from traditional frame cameras is large, and it is difficult to process the data online in real time, resulting in delayed transmission of results.

[0076] It can be seen that the above existing methods all have some problems, which make it impossible to accurately and real-time monitor rainfall data to meet rainfall monitoring and early warning needs.

[0077] This application example provides an example of a rainfall monitoring method, please refer to Figure 1 As shown, Figure 1 A schematic flow chart of a rainfall monitoring method provided in the present application is shown. As an example but not a limitation, the method can be applied to a rainfall monitoring system.

[0078] S101, in response to obtaining a rainfall event stream collected by a dynamic vision sensor, reconstructing a rainfall image based on the rainfall event stream.

[0079] S102, identifying the rainfall intensity corresponding to the rainfall occurring in the rainfall image.

[0080] S103: When the rainfall intensity meets the alarm condition, generate alarm prompt information according to the rainfall intensity and the rainfall occurrence period.

[0081] In the example of this application, a rainfall monitoring system is distributed and set up at multiple different observation points in a geographical area. For example, the rainfall monitoring system can be installed at locations prone to heavy rainfall or where flood control is weak, to conduct online monitoring of urban rainfall conditions. In specific usage scenarios, the rainfall events at the observation points are collected by the dynamic visual sensor in the rainfall monitoring system, which solves the problem of uneven distribution and limited number of observation points at ground rain gauges, and can more comprehensively cover the monitoring area and obtain richer rainfall data, thereby improving spatial resolution.

[0082] Optionally, the dynamic vision sensor can also be understood as an event camera, which can be a traditional camera or a hybrid camera that is a fusion of a traditional camera and a frame camera.

[0083] Secondly, for example, a dynamic visual analysis algorithm can be used to reconstruct a rainfall image (rainfall process image) based on the rainfall event flow and identify the rainfall intensity of the rainfall in the rainfall image. Compared with the rainfall measurement methods of radar, satellite remote sensing and traditional cameras, the accuracy of rainfall intensity is improved, avoiding the deficiencies of radar and satellite in hardware facilities and inversion of rainfall intensity, as well as the problem of reduced imaging effect of traditional cameras in environments with cluttered backgrounds and heavy rainfall and white fog.

[0084] Finally, when the rainfall intensity meets the alarm conditions, an alarm prompt message is generated and sent to the client, so as to provide accurate rainfall information to relevant departments and personnel in a timely manner and realize real-time early warning, which makes up for the defects of untimely data transmission of offline ground rain gauges and the difficulty of real-time online processing of traditional camera data, resulting in delayed result transmission.

[0085] For a possible implementation, please refer to Figure 2 As shown, Figure 2 A schematic flow chart of a rainfall monitoring method provided by the present application is shown, which reconstructs a rainfall image based on a rainfall event stream, including:

[0086] S201, obtaining event parameters of each rainfall event in a rainfall event stream, wherein the event parameters include: pixel coordinates, illumination change polarity, and timestamp.

[0087] S202, determining the change process of each rainfall event over time according to the mapping relationship between the timestamp and the pixel coordinates and the polarity of the illumination change.

[0088] S203: Reconstruct a rainfall image based on the change process of all rainfall events over time.

[0089] In an optional implementation, the location and time information of each rainfall event is accurately obtained by acquiring event parameters such as pixel coordinates, illumination change polarity, and timestamp of each rainfall event in the rainfall event stream.

[0090] Specifically, the pixel coordinates specify the location information of each rainfall event in the image. Obtaining the pixel coordinates can accurately track the distribution of raindrops at different observation points. The polarity of illumination change reflects the impact of raindrops on light. Since the presence of raindrops changes the propagation path of light, thereby causing changes in illumination, by recording the polarity of illumination change, the different effects of raindrops and other objects on light can be further distinguished. The timestamp records the time point when each rainfall event occurs, which specifically provides information in the time dimension for analyzing the dynamic changes of rainfall.

[0091] According to the mapping relationship between timestamp and pixel coordinates, and between timestamp and polarity of illumination change, the change process of each rainfall event over time can be determined. It should be understood that according to the mapping relationship between timestamp and pixel coordinates, and between timestamp and polarity of illumination change, different event parameters can be linked to construct the change process or change trajectory of each raindrop over time. Specifically, the higher the rainfall intensity, the greater the amplitude of event change, and the number of rainfall events has a certain positive correlation with the amplitude of time change and rainfall intensity.

[0092] For example, through the correspondence between timestamps and pixel coordinates, the position changes of raindrops at different time points can be determined; combined with the polarity of light changes, it is possible to understand how the impact of raindrops on light changes over time during movement.

[0093] Since the change process can show the entire process of each raindrop from appearance to disappearance in detail, it provides rich dynamic information for reconstructing rainfall images. Furthermore, by dynamically tracking the development trajectory of rainfall events, the dynamic change process of rainfall can be more accurately understood, overcoming the problem that the imaging effect of traditional cameras decreases when the background changes clutteredly, and it is difficult to capture raindrops in a white fog environment caused by heavy rainfall.

[0094] Finally, the rainfall image is reconstructed based on the change process of all rainfall events. Specifically, the change process of each raindrop is integrated to form a complete rainfall scene image. By comprehensively considering the position, illumination change and time change of all raindrops, a more realistic and accurate rainfall image is restored, so that the rainfall information collected by the dynamic visual sensor can be integrated to realize comprehensive rainfall monitoring of the observation point or monitoring point, making up for the uneven distribution and limited number of ground rain gauges. At the same time, it also solves the problem that the traditional frame camera has a large amount of data and is difficult to process online in real time, resulting in delayed result transmission, thereby improving the accuracy and real-time performance of rainfall monitoring.

[0095] In a possible implementation, before identifying the rainfall intensity of the rainfall in the rainfall image, the method further includes: performing image preprocessing on the rainfall image, wherein the image preprocessing includes at least one of the following: denoising processing and enhancement processing.

[0096] In the actual acquisition process, due to the influence of various factors, the rainfall image may contain some noise, such as sensor noise, environmental noise, etc. These noises will affect the subsequent accurate identification of rainfall intensity. Through denoising, the noise interference in the rainfall image can be removed, making the image clearer, highlighting the raindrop part, and improving the accuracy of rainfall feature extraction.

[0097] In some cases, rainfall images may appear dim or blurry due to poor lighting conditions or other reasons. Enhancement processing can enhance the contrast and brightness of rainfall images, making raindrops more visible and easier to analyze and identify. At the same time, enhancement processing can also improve the quality of the image, making it more detailed, and help to more accurately calculate parameters such as raindrop density, average particle size, and final moving speed.

[0098] Compared with the existing technology, denoising and enhancement processing can improve the image quality and make raindrops easier to identify in complex backgrounds, as the imaging effect of traditional cameras deteriorates when the background changes clutter. For the problem that radar and satellite remote sensing rainfall detection is not accurate, preprocessed rainfall images can provide more accurate input for rainfall recognition models, thereby improving the recognition accuracy of rainfall intensity. For the problem that it is difficult for ground rain gauges to obtain high spatial resolution rainfall data, by preprocessing rainfall images collected by dynamic visual sensors, information from different observation points can be better integrated, and spatial resolution and comprehensive monitoring can be improved.

[0099] In a possible implementation, identifying the rainfall intensity corresponding to the rainfall occurring in the rainfall image includes:

[0100] Acquire feature information contained in the rainfall image, wherein the feature information includes at least one of the following: background feature, noise feature and raindrop layer feature;

[0101] The feature information corresponding to the rainfall image is input into the pre-trained rainfall recognition model to obtain the rainfall intensity output by the rainfall recognition model.

[0102] The rainfall recognition model is obtained by training the initial neural network model with multiple sets of training data, and each set of training data includes: sample feature information of the sample rainfall image, and sample rainfall intensity corresponding to the sample feature information.

[0103] In the above optional implementation, first of all, it is of great significance to obtain various feature information such as background features, noise features and raindrop layer features contained in the rainfall image. For example, background features can help distinguish different environmental scenes, such as cities, mountains or plains, and different backgrounds may affect the performance and monitoring of rainfall. The extraction of noise features helps to eliminate other interference factors in the environment and improve the accuracy of rainfall monitoring. The raindrop layer features directly reflect the state of rainfall, including the size, density and distribution of raindrops.

[0104] Then, these feature information are input into the pre-trained rainfall recognition model, which is pre-trained with a large number of sets of training data. During the pre-training process, each set of training data contains: sample feature information of sample rainfall images, and sample rainfall intensity corresponding to the sample feature information. By continuously adjusting the parameters of the model, the model can learn the complex mapping relationship between different feature combinations and specific rainfall intensities.

[0105] Analysis shows that compared with traditional radar rainfall measurement, this method is not limited by hardware facilities and can adapt to different monitoring environments more flexibly. Radar rainfall measurement may be interfered by various factors when inverting rainfall intensity, while the method based on the neural network model can comprehensively consider multiple features, reduce the impact of interference factors, and improve accuracy.

[0106] Compared with satellite remote sensing rainfall monitoring, this method can provide higher spatial resolution. Although satellite remote sensing can cover a large area, it may not be precise enough for rainfall monitoring in local areas. Dynamic visual sensors can obtain more detailed local rainfall information, and combined with neural network models for analysis, it can accurately identify rainfall intensity in different areas.

[0107] Compared with ground-based rain gauges, this method overcomes the problem of uneven distribution and limited number of observation points. Ground-based rain gauges are expensive to build and maintain, and it is difficult to achieve real-time collaborative monitoring. However, the use of dynamic visual sensors and neural network models can quickly process large amounts of data, achieve real-time monitoring and accurate early warning, and provide stronger support for flood prevention and disaster reduction.

[0108] For a possible implementation, please refer to Figure 3 As shown, Figure 3 A schematic flow chart of a rainfall monitoring method provided by the present application is shown. The rainfall recognition model processes the characteristic information of the rainfall image as follows to obtain the rainfall intensity:

[0109] S301, extracting raindrop parts in the rainfall image based on feature information of the rainfall image;

[0110] S302, calculating the raindrop density, average raindrop particle size and final moving speed of the raindrop part in the rainfall image;

[0111] S303, calculating the rainfall intensity according to the raindrop density, the average particle size of the raindrops and the final moving speed of the raindrops.

[0112] Optionally, in the example of the present application, the rainfall recognition model can be a LeNet-5 convolutional neural network model, or a VGG series model, a ResNet series model, etc.

[0113] In this implementation, the rainfall recognition model first extracts the raindrops based on the feature information of the rainfall image. This step can overcome the problem that traditional cameras have difficulty accurately capturing raindrops in a cluttered background and in a white foggy environment with heavy rainfall. By utilizing a variety of feature information, raindrops can be separated more accurately, improving the ability to recognize rainfall.

[0114] Next, the rainfall recognition model calculates the raindrop density, average particle size and end-of-raindrop moving speed of the raindrop part in the rainfall image. It should be understood that the above three parameters can reflect the characteristics of rainfall from different angles. For example, the raindrop density can reflect the intensity and range of rainfall. The greater the density, the stronger the rainfall. The average particle size of raindrops can reflect the size distribution of raindrops, which is important for judging the type and intensity of rainfall. The end-of-raindrop moving speed can reflect the dynamic changes of rainfall and the influence of factors such as wind.

[0115] Finally, the rainfall identification model calculates the rainfall intensity based on the calculated raindrop density, average raindrop size, and raindrop end moving speed. This calculation method that integrates multiple parameters is more accurate and reliable than the traditional single measurement method. Compared with ground rain gauges, this method does not require a large number of observation points and high-cost construction and maintenance. It can obtain a wider range of rainfall data through dynamic visual sensors and obtain accurate rainfall intensity through model calculation. Compared with radar and satellite remote sensing rainfall, this method has significantly improved in accuracy, avoiding the shortcomings of hardware facilities and inverted rainfall intensity. At the same time, it is more sensitive to the observation of heavy rainfall and can provide real-time and accurate rainfall data.

[0116] For a possible implementation, please refer to Figure 4 As shown, Figure 4 A schematic flow chart of a rainfall monitoring method provided by the present application is shown, which extracts raindrop parts in a rainfall image based on feature information of the rainfall image, including:

[0117] S401, using an edge detection algorithm to extract an approximate raindrop portion in the rainfall image based on feature information of the rainfall image.

[0118] S402, using an image gradient calculation algorithm to calculate image gradient values ​​of the approximate raindrop portion in the horizontal direction and the vertical direction.

[0119] S403, performing non-maximum suppression processing on the image gradient value to determine edge pixels approximating the raindrop portion.

[0120] S404, determining the raindrop portion in the rainfall image according to the edge pixels approximating the raindrop portion.

[0121] Optionally, the edge detection algorithm in the example of the present application may be an image Canny edge detection algorithm or other edge detection methods such as a Prewitt operator and a Laplacian operator; the image gradient calculation algorithm in the example of the present application may be a Sobel operator.

[0122] In addition, the Lucas-Kanade (LK) optical flow method can also be used in the example of this application to track the pixel point at the end of the raindrop in the picture, track the movement trajectory, movement time and pixel displacement of the raindrop, and calculate the end movement speed of the raindrop. The Lucas-Kanade (LK) optical flow method used in the example of this application can also be replaced by the Horn-Schunck optical flow algorithm, PCA-Flow algorithm, etc.

[0123] In the above optional implementation, the rainfall recognition model can effectively extract the raindrop part in the rainfall image through the following steps. Specifically, first, an edge detection algorithm is used to extract the approximate raindrop part based on the feature information of the rainfall image. The edge detection algorithm can quickly locate the edge contour of the object based on the image features, such as changes in color, brightness, etc. In the rainfall image, the possible raindrop area can be found more accurately by using information such as background features, noise features, and raindrop layer features, overcoming the problem that traditional cameras are difficult to capture raindrops in complex environments.

[0124] Next, the image gradient calculation algorithm is used to calculate the image gradient values ​​of the approximate raindrop part in the horizontal and vertical directions. Since the image gradient reflects the degree of change of the pixel value in the image, it is very important for determining the edge of the object. By calculating the gradient value, the edge information of the raindrop can be further refined and the accuracy of raindrop extraction can be improved.

[0125] Then, non-maximum suppression is performed on the image gradient value to determine the edge pixels of the approximate raindrop part. Non-maximum suppression can remove those pixels that are not local maxima, thereby retaining the most obvious edge pixels, and can remove some blurred edges and noise, making the edge of the raindrop clearer and more definite.

[0126] Finally, the raindrop part in the rainfall image is determined based on the edge pixels of the approximate raindrop part. Through the above series of processing, raindrops can be accurately extracted from the rainfall image, providing a reliable basis for the subsequent calculation of raindrop density, average particle size, final moving speed and other parameters.

[0127] Compared with traditional rainfall monitoring technology, ground rain gauges are limited by the distribution and number of observation points, making it difficult to monitor rainfall comprehensively and accurately; radar and satellite remote sensing rainfall detection are insufficient in accuracy and sensitivity to heavy rainfall; and this method based on image feature information and a series of algorithm processing can extract raindrop information in real time and accurately, providing strong support for providing real-time and accurate rainfall data and alarm prompt information.

[0128] For a possible implementation, please refer to Figure 5 As shown, Figure 5 A schematic flow chart of a rainfall monitoring method provided by the present application is shown, which determines the raindrop portion in the rainfall image according to the edge pixels approximating the raindrop portion, including:

[0129] S501: When the edge pixel is greater than a first edge threshold, determine to retain the edge pixel.

[0130] S502: When the edge pixel is smaller than the second edge threshold, determine to remove the edge pixel, wherein the first edge threshold is greater than the second edge threshold.

[0131] S503: When the edge pixel is smaller than the first edge threshold and larger than the second edge threshold, determine whether to remove the edge pixel according to connection information between the edge pixel and the retained edge pixels.

[0132] S504: Determine the raindrop portion in the rainfall image according to the retained edge pixels.

[0133] In this implementation, first, the preset first edge threshold and second edge threshold are determined, and the edge pixels of the raindrop-like part are classified and processed. When the edge pixel is greater than the first edge threshold, it is determined to retain the edge pixel to quickly screen out the more obvious raindrop edges, ensuring accurate recognition of larger and clearer raindrops.

[0134] Then, when the edge pixel is less than the second edge threshold, it is determined to remove the edge pixel. Since the second edge threshold is relatively low, edge pixels less than this threshold are likely to be noise or unclear parts. Removing these pixels can reduce interference and improve the accuracy of raindrop extraction.

[0135] In addition, for edge pixels that are less than the first edge threshold and greater than the second edge threshold, whether to remove them is determined based on the connection information between the edge pixels and the retained edge pixels. This processing method fully considers the uncertainty of edge pixels. By performing connection analysis with the determined reliable edge pixels, it can be further determined whether these edge pixels in the middle range are part of the raindrop. For example, if the edge pixel has a good connection relationship with the retained edge pixel, it indicates that the edge pixel is likely to be the edge of the raindrop, so the edge pixel is retained; if the edge pixel has no connection relationship with the retained edge pixel, the edge pixel is removed.

[0136] Finally, the raindrop part in the rainfall image is determined based on the retained edge pixels. After the above processing, the real raindrop part can be accurately extracted from the approximate raindrop part to provide a data basis for the subsequent calculation of parameters such as rainfall intensity.

[0137] Compared with traditional rainfall monitoring technology, this method based on edge pixel classification and connection analysis is more detailed and accurate in processing raindrop edges. It can accurately extract raindrops in complex rainfall images, improve the accuracy and reliability of rainfall intensity calculation, and provide strong support for real-time and accurate rainfall monitoring and early warning.

[0138] In a possible implementation, calculating the end moving speed of a raindrop portion in a rainfall image includes:

[0139] Determine the pixel point at the end of the raindrop in the rainfall image.

[0140] According to the pixel point at the end of the raindrop, the motion trajectory, motion time and pixel displacement of the raindrop part are determined.

[0141] According to the motion trajectory, motion time and pixel displacement of the raindrop part, the end moving speed of the raindrop part in the rainfall image is calculated.

[0142] In this implementation, the final moving speed of the raindrop part in the rainfall image is accurately calculated through the following steps, which effectively solves the problems in the prior art and achieves good technical effects.

[0143] First, determine the pixel point at the end of the raindrop in the rainfall image. This step can be done by analyzing the edge pixels of the raindrop to find the end position of the raindrop in the image. Since raindrops form a certain trajectory during the falling process, determining the pixel point at the end position provides key reference information for the subsequent calculation of the motion trajectory, motion time and pixel displacement of the raindrop.

[0144] Next, the motion trajectory, motion time, and pixel displacement of the raindrop are determined based on the pixel point at the end of the raindrop. By analyzing multiple consecutive frames of rainfall images, the motion path of the raindrop from the starting position to the end position can be tracked to determine its motion trajectory. The pixel displacement is obtained by calculating the position change of the raindrop between different frames, which reflects the distance the raindrop moves in the image.

[0145] Finally, according to the motion trajectory, motion time and pixel displacement of the raindrop part, the final moving speed of the raindrop part in the rainfall image is calculated. The relationship between time information and pixel displacement can be used to convert the pixel displacement into actual physical displacement, and the final moving speed of the raindrop can be calculated in combination with the time interval.

[0146] Compared with traditional rainfall monitoring technology, this method can calculate the final moving speed of raindrops in real time by analyzing the raindrop part in the rainfall image, providing important parameters for a more comprehensive understanding of rainfall characteristics. This helps to improve the accuracy of rainfall intensity calculation and provide more powerful support for timely and accurate rainfall monitoring and early warning.

[0147] In a possible implementation, measurement information of rain gauges distributed in multiple areas may also be obtained; and the change process of rainfall intensity may be determined based on the change process of the measurement information.

[0148] In this implementation, the rain gauge, as a traditional rainfall measurement device, can directly measure the precipitation at a specific location. By setting up rain gauges in multiple areas, actual rainfall data at different locations can be obtained, making up for the limitations of a single monitoring method.

[0149] Since the measurement information of the rain gauge usually includes data such as the accumulated precipitation over a period of time, the changing trend of rainfall intensity can be inferred by analyzing the changes of these data over time. For example, if the precipitation measured by the rain gauge increases rapidly in a short period of time, it means that the rainfall intensity is increasing; conversely, if the precipitation gradually decreases, it means that the rainfall intensity is decreasing.

[0150] Compared with the existing technology, in order to solve the problem of uneven distribution and limited number of ground rain gauges, the density of monitoring points can be increased and the spatial resolution can be improved by setting up rain gauges in multiple areas. At the same time, the measurement information of the rain gauge can be verified with other monitoring means such as dynamic visual sensors to improve the accuracy of rainfall intensity calculation. For the problem that radar and satellite remote sensing rainfall detection is not accurate and is not sensitive to heavy rainfall, the field measurement data of the rain gauge can be used as a supplement to more accurately reflect the changes in rainfall intensity in local areas. For the problem that traditional cameras have large amounts of data and are difficult to process in real time, the measurement information of the rain gauge is relatively simple and easy to analyze in real time, which helps to timely determine the changes in rainfall intensity and provide more timely support for early warning and decision-making.

[0151] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0152] Corresponding to the rainfall monitoring method in the above embodiment, Figure 6 is a schematic diagram of a rainfall monitoring system provided in an embodiment of the present application, referring to Figure 6 , the rainfall monitoring system includes:

[0153] Dynamic vision sensor 601 (the number of the dynamic vision sensor can be one or more, Figure 6 ), which is schematically shown in FIG. 1 , is set at an observation point to collect rainfall events and obtain a rainfall event stream.

[0154] Processor 602 is connected to the dynamic visual sensor and is used to reconstruct a rainfall image based on the rainfall event stream and identify the rainfall intensity corresponding to the rainfall occurring in the rainfall image. When the rainfall intensity meets the alarm condition, an alarm prompt information is generated according to the rainfall intensity and the rainfall occurrence period.

[0155] Optionally, the rainfall monitoring system can be distributed at multiple different observation points in a geographical area. For example, the rainfall monitoring system can be installed at locations prone to heavy rainfall or weak in flood control to conduct online monitoring of urban rainfall status. In specific usage scenarios, the rainfall events at the observation points are collected by the dynamic visual sensor in the rainfall monitoring system, which solves the problem of uneven distribution and limited number of observation points of ground rain gauges, and can more comprehensively cover the monitoring area and obtain richer rainfall data, thereby improving spatial resolution.

[0156] Secondly, rainfall images are reconstructed and rainfall intensity is identified based on the rainfall event stream. Compared with rainfall measurement by radar and satellite remote sensing and rainfall measurement by traditional cameras, the accuracy of rainfall intensity is improved. This avoids the deficiencies of radar and satellite in hardware facilities and inversion of rainfall intensity, as well as the problem of reduced imaging effect of traditional cameras in environments with cluttered backgrounds and white fog caused by heavy rainfall.

[0157] Finally, when the rainfall intensity meets the alarm conditions, an alarm prompt message is generated and sent to the client, so as to provide accurate rainfall information to relevant departments and personnel in a timely manner and realize real-time early warning, thus making up for the defects of untimely data transmission of offline ground rain gauges and the difficulty of real-time online processing of traditional camera data, which leads to delayed result transmission.

[0158] In summary, the rainfall monitoring system provided in the example of this application is based on dynamic vision. Through the collaborative work of dynamic vision sensors and processors, it effectively solves the common problems of existing rainfall monitoring technologies and can achieve accurate, real-time, and high-resolution rainfall monitoring.

[0159] In addition, in this application example, Figure 6 As shown, the rainfall monitoring system also includes: a storage module 603, a power module 604 (such as a solar power module), a communication module 605 (such as a 4G communication module, a Beidou positioning and timing module), and a tipping bucket rain gauge 606. The rainfall monitoring system integrated with each module is installed in locations prone to heavy rainfall or weak flood control to conduct online monitoring of urban rainfall status.

[0160] By encapsulating the dynamic vision sensor 601, processor 602 (computing module), storage module 603, power module 604 (such as solar power module) and communication module 605 (such as 4G communication module, Beidou positioning and timing module), a stable and reliable modular packaging model is built to realize the functions of monitoring data collection, calculation, analysis and transmission. The dynamic vision sensor is responsible for sensing the change of light intensity, the computing module is responsible for analyzing the event stream data, judging whether an event is rainfall and the rainfall level, the storage module is responsible for storing rainfall process event data, and the data is transmitted through the 4G communication module under normal monitoring conditions. When natural disasters such as heavy rainfall occur, the Beidou positioning and timing module is used to transmit key data information. In addition, Figure 6 The tipping bucket rain gauge 606 is used to provide the actual rainfall intensity and provide an error judgment basis for the dynamic visual sensor monitoring results.

[0161] The algorithm is written using event stream information for image reconstruction. First, an optimization algorithm is established based on the image characteristics of the background, noise, and raindrop layer to preliminarily extract raindrop information, and the input event information is classified and processed. Through optical flow estimation, feature extraction and fusion, the classification of rainfall and non-rainfall is achieved. Secondly, the event data is denoised and enhanced to make the image restoration more realistic and clear. Contrast enhancement technology and noise reduction methods are used to process low-contrast and high-noise rainfall images, and local contrast is enhanced to solve the problem of uneven lighting.

[0162] Then, the restored rainfall image is used to extract raindrop related parameters, the Canny edge detection algorithm is used to extract the raindrop part in the picture, the Sobel operator is used to calculate the gradient of the image, the non-maximum suppression of the gradient image is performed to refine the edge, the double threshold strategy is used to adjust the threshold to detect and connect the edge, and the raindrop density and the average particle size of the raindrops are calculated based on the extracted raindrops; the Lucas-Kanade (LK) optical flow method is used to track the pixel points at the end position of the raindrops in the picture, the motion trajectory, motion time and pixel displacement of the raindrops are tracked, and the end moving speed of the raindrops is calculated.

[0163] Finally, the LeNet-5 convolutional neural network model is used to establish the mapping relationship between rainfall image information R(x, y, z) (x is the raindrop density, y is the average particle size of the raindrop, and z is the final velocity of the raindrop) and rainfall intensity I, so that the rainfall intensity can be estimated using the trained neural model. For rainfall videos, a recurrent neural network is used to extract the temporal features of the video, so that rainfall recognition can be achieved based on video information.

[0164] In the specific application process, the dynamic visual sensor receives the input of light intensity change information, perceives the dynamic light change and outputs the event stream. Each event carries information such as pixel coordinates, light change polarity and timestamp, indicating that the light intensity received by the pixel sensor point at the corresponding coordinate position at a certain moment has changed. The higher the rainfall intensity, the more pixels in the picture have light intensity changes, the faster the light intensity changes, and the more events are generated. The number of events has a certain correlation with the rainfall intensity. Therefore, it is intended to explore the correlation between the change in the number of events and the rainfall intensity. The event stream timestamp is used as an independent variable, and the event number information in the rainfall process is superimposed with the microsecond to millisecond scale integration time to restore the change process of the number of events over time. In addition, according to the rainfall intensity change process obtained in the second step and the measurement results of the tipping bucket rain gauge, the correlation between the change process of the number of events and the change process of the rainfall intensity is observed and analyzed, so as to realize the change of rainfall intensity through the change of the number of events, which is helpful to directly use the event data to realize the rapid analysis of the rainfall process and make a preliminary judgment on whether it is raining.

[0165] Multiple event cameras are used to carry out real-time online monitoring of the rainfall status at multiple points in the city. The Beidou system is used to realize data fusion and communication of multiple event cameras. The Beidou control machine can realize two-way communication between the Beidou system and the rainfall monitoring device, thereby integrating the data of multiple event cameras and the algorithm processing results (whether it is raining, rainfall intensity and rainfall occurrence time). When natural disasters such as heavy rain or short-term heavy rainfall occur, early warning prompts are issued through the short message system.

[0166] In addition, the example of this application can also be combined with the ground rain gauge currently used by meteorological stations as a city rainfall monitoring system for real-time monitoring, analysis and management. The monitoring data can be sent to the central station through a variety of communication methods. After the central station collects these data, the rainfall intensity is analyzed, and multiple event cameras arranged in different locations work together to restore the regional distribution map of rainfall status.

[0167] Corresponding to the rainfall monitoring method of the above embodiment, an example of a rainfall monitoring device is provided. The device can be implemented as part or all of a computer device by software, hardware or a combination of both. The computer device can be Figure 7 The electronic device shown. The rainfall monitoring device comprises:

[0168] A monitoring unit, configured to reconstruct a rainfall image based on the rainfall event stream in response to the rainfall event stream acquired by the dynamic visual sensor;

[0169] an identification unit, used for identifying the rainfall intensity corresponding to the rainfall occurring in the rainfall image;

[0170] The alarm unit is used to generate alarm prompt information according to the rainfall intensity and rainfall occurrence period when the rainfall intensity meets the alarm condition.

[0171] It should be noted that the rainfall monitoring device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0172] The functional units and modules in the above embodiments may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit, and the above integrated units may be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0173] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0174] An embodiment of the present application further provides an electronic device, the electronic device comprising one or more processors and a memory;

[0175] The memory is coupled to one or more processors, and the memory is used to store computer program codes, the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the rainfall monitoring method shown above.

[0176] The electronic device may be a mobile phone, a smart screen, a tablet computer, a wearable electronic device, an in-vehicle electronic device, an augmented reality (AR) device, a virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a projector, or a communication device such as a server, a storage device, a base station, or a smart car, etc. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0177] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored; when the computer-readable storage medium is executed on an electronic device, the electronic device executes the rainfall monitoring method shown above.

[0178] Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer-readable storage media can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media integrated therein. Available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media, or semiconductor media (e.g., solid-state drives (SSDs)), etc.

[0179] The embodiment of the present application also provides a computer program product including computer instructions, which, when executed on an electronic device, enables the electronic device to execute the rainfall monitoring method shown above.

[0180] The computer storage medium and computer program product provided in the above-mentioned embodiments of the present application are used to execute the method provided above. Therefore, the beneficial effects that can be achieved can refer to the corresponding beneficial effects of the method provided above, and will not be repeated here.

[0181] In the above embodiments, it can also be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (such as: coaxial cable, optical fiber, data subscriber line (Digital Subscriber Line, DSL)) or wireless (such as: infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital versatile disc (DVD)), or a semiconductor medium (such as a solid state drive (SSD)), etc.

[0182] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, the electronic device 700 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0183] The memory 701 can be used to store computer software programs 702 and modules. The processor 703 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 701. The memory 701 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 701 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0184] Among them, the processor 703 may include one or more processors such as a central processing unit, an application processor (AP), a baseband processor, etc. The processor may be the nerve center and command center of the wireless router. The processor 703 may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching instructions and executing instructions. The memory 701 may be used to store computer executable program codes, and the executable program codes include instructions. The processor 703 executes various functional applications and data processing of the network device by running the instructions stored in the memory. The memory 701 may include a program storage area and a data storage area, such as storing data of a sound signal to be played. For example, the memory may be a double rate synchronous dynamic random access memory DDR or a flash memory Flash, etc.

[0185] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0186] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments applied for herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0187] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0188] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0189] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A rainfall monitoring method, characterized in that: include: In response to acquiring a rainfall event stream collected by a dynamic visual sensor, reconstructing a rainfall image based on the rainfall event stream; Identifying the rainfall intensity corresponding to the rainfall occurring in the rainfall image; When the rainfall intensity meets the alarm condition, an alarm prompt message is generated according to the rainfall intensity and the rainfall occurrence period.

2. The method according to claim 1, characterized in that The step of reconstructing a rainfall image based on the rainfall event stream comprises: Acquire event parameters of each rainfall event in the rainfall event stream, wherein the event parameters include: pixel coordinates, illumination change polarity and timestamp; Determine the change process of each rainfall event over time according to the mapping relationship between the timestamp and the pixel coordinates and the polarity of the light change; The rainfall image is reconstructed based on the change process of all the rainfall events over time.

3. The method according to claim 1, characterized in that The identifying the rainfall intensity corresponding to the rainfall occurring in the rainfall image includes: Acquire feature information contained in the rainfall image, wherein the feature information at least includes at least one of the following: background feature, noise feature and raindrop layer feature; The feature information corresponding to the rainfall image is input into a pre-trained rainfall recognition model to obtain the rainfall intensity output by the rainfall recognition model, wherein the rainfall recognition model is obtained by training an initial neural network model using multiple groups of training data, and each group of the training data includes: sample feature information of a sample rainfall image, and a sample rainfall intensity corresponding to the sample feature information.

4. The method according to claim 3, characterized in that The rainfall recognition model processes the characteristic information of the rainfall image as follows to obtain the rainfall intensity: extracting a raindrop portion in the rainfall image based on feature information of the rainfall image; Calculating the raindrop density, average raindrop particle size and final moving speed of the raindrop portion in the rainfall image; The rainfall intensity is calculated based on the raindrop density, the average raindrop particle size and the raindrop end moving speed.

5. The method according to claim 4, characterized in that Extracting a raindrop portion in the rainfall image based on feature information of the rainfall image includes: Using an edge detection algorithm, extracting a raindrop-like portion in the rainfall image based on feature information of the rainfall image; Using an image gradient calculation algorithm to calculate image gradient values ​​of the approximate raindrop portion in the horizontal direction and the vertical direction; Performing non-maximum suppression processing on the image gradient value to determine edge pixels of the approximate raindrop portion; The raindrop portion in the rainfall image is determined according to the edge pixels of the approximate raindrop portion.

6. The method according to claim 5, characterized in that Determining the raindrop portion in the rainfall image according to the edge pixels of the approximate raindrop portion includes: In a case where the edge pixel is greater than a first edge threshold, determining to retain the edge pixel; In the case where the edge pixel is less than a second edge threshold, determining to remove the edge pixel, wherein the first edge threshold is greater than the second edge threshold; When the edge pixel is smaller than the first edge threshold and larger than the second edge threshold, determining whether to remove the edge pixel according to connection information between the edge pixel and the retained edge pixels; The raindrop portion in the rainfall image is determined according to the retained edge pixels.

7. The method according to claim 4, characterized in that Calculating the final moving speed of the raindrop portion in the rainfall image comprises: Determine a pixel point at the raindrop end position of the raindrop part in the rainfall image; Determine the motion trajectory, motion time and pixel displacement of the raindrop portion according to the pixel point at the end position of the raindrop; The final moving speed of the raindrop portion in the rainfall image is calculated according to the movement trajectory, movement time and pixel displacement of the raindrop portion.

8. A rainfall monitoring system, characterized in that: include: Dynamic visual sensors are set at observation points to collect rainfall events and obtain rainfall event streams; A processor is connected to the dynamic visual sensor and is used to reconstruct a rainfall image based on the rainfall event stream and identify the rainfall intensity corresponding to the rainfall occurring in the rainfall image. When the rainfall intensity meets the alarm condition, an alarm prompt information is generated according to the rainfall intensity and the rainfall occurrence period.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images