Crowdsourcing video rainfall quantitative calculation method based on knowledge graph

Abstract

The invention belongs to the technical field of rainfall observation, and discloses a crowdsourcing video rainfall quantitative calculation method based on a knowledge graph, and the method comprises the following steps: firstly constructing a rainfall video template library, extracting video attributes of crowdsourcing videos in the rainfall video template library, and storing the video attributes into the knowledge graph; secondly, preprocessing the target crowdsourcing video, extracting the video attribute of the target crowdsourcing video, matching the video attribute of the target crowdsourcing video with the video attribute of the crowdsourcing video in the rainfall video template library in sequence, screening out a video set closest to the target crowdsourcing video in the rainfall video template library, and based on the video set, extracting the video attribute of the target crowdsourcing video; similarity calculation is completed based on a rainfall video similarity calculation method; and finally, comparing a preset similarity threshold with the calculated similarity to realize rainfall quantitative calculation of the target crowdsourcing video. According to the method, the crowdsourcing video can be effectively utilized for rainfall estimation, and the accuracy of rainfall estimation is enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of rainfall observation, and more specifically relates to a crowdsourcing video rainfall quantitative calculation method based on a knowledge graph. Background Art

[0002] Rainfall monitoring is of great significance in weather forecasting and disaster prevention. Traditional rainfall monitoring methods, such as weather radar, satellite remote sensing, and ground-based weather stations, can provide relatively accurate rainfall data, but their construction and maintenance costs are high, and their deployment density is limited, making them unable to cover all areas, especially in cities and remote areas. With the popularization of smartphones and video surveillance equipment, crowdsourced videos have become a new data source. In cities and remote areas, video data provides an additional data source that traditional meteorological monitoring cannot cover, helping to make up for the shortcomings of traditional meteorological monitoring. However, the application of crowdsourced videos in rainfall monitoring faces the following challenges: (1) Lack of accurate shooting parameters: Crowdsourced videos usually lack important information such as shooting angle, time, and camera position; (2) Complex scene content: Crowdsourced videos capture a wide variety of scenes, and the dynamic and static elements contained in the videos (such as pedestrians, vehicles, trees, roads, etc.) make the rainfall characteristics vary greatly in different scenes, making it difficult to extract effective information from them; (3) Limitations of video analysis: Most existing video analysis methods assume that the shooting parameters are known and perform processing based on these parameters, but cannot adapt to the situation where shooting parameters are missing in crowdsourced videos.

[0003] How to effectively use crowdsourced videos for rainfall estimation, especially in the absence of clear shooting parameters, has become an urgent problem in the field of meteorological monitoring. Summary of the Invention

[0004] In order to solve the above technical problems, this application provides a method for quantitatively calculating rainfall from crowdsourced videos based on knowledge graphs. This method effectively uses crowdsourced videos for rainfall estimation, especially when shooting parameters are missing, to provide a new rainfall monitoring solution.

[0005] In order to achieve the above objectives, this application is implemented through the following technical solutions:

[0006] This application is a crowdsourced video rainfall quantitative calculation method based on knowledge graph, which specifically includes the following steps:

[0007] Step 1: Build a rainfall video template library, extract the video attributes of the crowdsourced videos in the rainfall video template library, and convert the video attributes of the crowdsourced videos in the rainfall video template library into structured data and store them in the knowledge graph;

[0008] Step 2: Perform data cleaning and preprocessing on the target crowdsourced video for rainfall quantitative calculation, extract the video attributes of the target crowdsourced video, and align the extracted video attributes of the target crowdsourced video with the structured data stored in the knowledge graph of the rainfall video template library;

[0009] Step 3: According to the video attribute levels defined in the knowledge graph, the video attributes of the target crowdsourced video are matched with the video attributes of the crowdsourced videos in the rainfall video template library in sequence, and the video set in the rainfall video template library that is closest to the target crowdsourced video is selected;

[0010] Step 4: Based on the target crowdsourced video matched and screened in step 3 and the video collection of crowdsourced videos in the rainfall video template library, a rainfall video similarity calculation method based on frequency domain analysis is used to complete the similarity calculation;

[0011] Step 5: Compare the similarity calculated in step 4 with the preset similarity threshold to determine whether the rainfall signals between the crowdsourced video in the rainfall video template library and the target video are similar. Finally, the rainfall information of the most matching video in the rainfall video template library is assigned to the target crowdsourced video to achieve quantitative calculation of the rainfall in the target crowdsourced video.

[0012] The present application is further improved in that the video attributes of the crowdsourced video in the rainfall video template library and the video attributes of the target crowdsourced video both include three levels of attributes, specifically:

[0013] First-level attributes: Video features are extracted through first-level attributes. Rainfall video features include: shooting time period, shooting angle, image quality, and scene dynamics.

[0014] Secondary attributes: Extract the scene components of the video through secondary attributes;

[0015] Level 3 attributes: Rainfall feature information in the video is extracted through level 3 attributes, and the rainfall feature information includes rainfall intensity and rainfall level.

[0016] The present application is further improved in that: in step 1, the construction of the rainfall video template library specifically includes the following steps:

[0017] Step 1.1: Extract the primary video attributes of the crowdsourced videos in the rainfall video template library, including extracting the shooting time period, calculating the scene dynamics, extracting scene elements and analyzing their proportions, identifying shooting angles, and observing rainfall information.

[0018] Step 1.2: Convert the video attributes extracted from the crowdsourced video into structured data and store them in the nodes of the knowledge graph. In the knowledge graph, establish nodes and relationships. The nodes are crowdsourced video clips or crowdsourced video attributes in the rainfall video template library, and the relationships are used to connect the associations between the crowdsourced video clips and the corresponding crowdsourced video attributes.

[0019] The present application is further improved in that: Step 2 of data cleaning and preprocessing of the target crowdsourced video specifically includes the following steps:

[0020] Step 2.1.1, format unification: convert the target crowdsourcing videos into MP4 format;

[0021] Step 2.1.2, duration unification: If the duration of the target crowdsourced video is longer than the length of the crowdsourced videos in the rainfall video template library, the target crowdsourced video is cut into multiple segments;

[0022] Step 2.1.3, frame rate standardization: adjust the frame rate of the target crowdsourced video to be consistent with any frame rate in the crowdsourced videos in the rainfall video template library, and unify the playback speed of the target crowdsourced video and the crowdsourced videos in the rainfall video template library;

[0023] Step 2.1.4: Quickly filter target crowdsourcing videos without rain: By analyzing the texture changes of the target crowdsourcing video frames, the Laplace operator is used to detect the amplitude of local brightness changes in the target crowdsourcing video image to determine whether there is rain in the target crowdsourcing video. If the texture change is large, it means that there is rain in the target crowdsourcing video. If the texture change is small, it means that there is no rain in the target crowdsourcing video. For target crowdsourcing videos without rain, the rainfall intensity is determined to be 0. The calculation method is as follows:

[0024]

[0025] Where I(x,y) is the pixel intensity of the target crowdsourcing video frame rate, Represents the Laplace operator or second-order derivative of the target crowdsourced video image at position (x, y), It is the second-order partial derivative of I(x,y) with respect to x, that is, first find the first-order partial derivative of I(x,y) with respect to x, and then find the first-order partial derivative of the result with respect to x. The second-order partial derivative with respect to y indicates the degree of curvature of I(x,y) in the y direction.

[0026] The present application is further improved in that: Step 1 of extracting the video attributes of the crowdsourced video in the rainfall video template library and Step 2 of extracting the video attributes of the target crowdsourced video both include:

[0027] Extracting the shooting time period: This includes extracting time metadata, that is, extracting the shooting time information from the video metadata. If the metadata is missing, the time period is determined by calculating the overall brightness of the video. The determination method is:

[0028] a. Extract frames from the video and convert them into grayscale images;

[0029] b. Calculate the mean-brightness of the grayscale image:

[0030]

[0031] Where gray(i) is the grayscale value of the i-th grayscale image, N is the total number of pixels, and a threshold T is set. If the value exceeds this threshold, it is judged to be daytime, and if the value is lower than the threshold, it is judged to be nighttime. The judgment conditions are:

[0032]

[0033] Calculate scene dynamics: Use optical flow to obtain motion vector data, quantify the degree of dynamics in the scene, and determine dynamics, including the average size of the optical flow vector, the distribution of the moving object's motion direction, the proportion of high-motion areas, and the uniformity of motion.

[0034] The scene element extraction and proportion analysis specifically involves using a deep learning semantic segmentation model to segment scene elements at the pixel level, obtaining the proportion of different elements in the picture. The video is then fed frame by frame into the trained deep learning semantic segmentation model, which processes each frame and outputs a semantic segmentation result map of the same size as the input image. Each pixel is assigned a classification label. The pixel values ​​of each segmentation map are counted, and the pixel proportion of each type of element is calculated. This information is then converted into secondary attributes of the crowdsourced video.

[0035] Shooting angle recognition: Specifically, first, the horizon line is identified based on the video elements (i.e., sky or road) identified by the deep learning semantic segmentation model. Then, the position of the horizon in the video image is calculated. Assuming that the detected horizon is at position yh in the image and the image center is yc, the relative index of the pitch angle is calculated based on the position difference:

[0036] relative angle=yh-yc.

[0037] The shooting angle is determined by the positive and negative values ​​of the relative angle: relative angle = 0 means horizontal shooting; relative angle > 0 means upward shooting; relative angle < 0 means downward shooting;

[0038] Observe rainfall information: By synchronizing observation data, each video is equipped with a corresponding precipitation intensity label. Based on the rainfall intensity, rainfall level labels are established: light rain, moderate rain, heavy rain, and rainfall characteristic information is constructed: rainfall intensity and rainfall level.

[0039] The present application is further improved in that the dynamic degree of the scene is calculated as follows:

[0040] Average size of optical flow vectors: Use the optical flow method to detect the vector data of moving objects in the video, such as vehicles and pedestrians, and calculate the average size of the optical flow vectors of all video clip pixels:

[0041]

[0042] Among them, u i and v i are the optical flow components of pixel i in the video clip in the x and y directions, respectively, and N is the total number of pixels;

[0043] Distribution of moving object's motion direction: Obtain the motion trend of objects in the scene by statistically analyzing the direction distribution of motion vectors;

[0044] High motion area ratio: Set the threshold of the optical flow vector size to identify the high dynamic area in the scene, reflecting the proportion of the high dynamic area in the scene:

[0045]

[0046] Among them, Numberofhigh-motionpixels is the high-dynamic area, i.e., the area of ​​fast-moving vehicles or pedestrians;

[0047] Motion uniformity: The uniformity of motion is judged by the standard deviation of the optical flow vector.

[0048] The present application is further improved in that: Step 3 sequentially matches the video attributes of the target crowdsourced video with the video attributes of the crowdsourced videos in the rainfall video template library, specifically including the following steps:

[0049] Step 3.1: First-level attribute matching, including:

[0050] Time period matching: Select the time period field of the crowdsourced videos in the rainfall video template library and retrieve the crowdsourced videos with the same time period as the target crowdsourced video;

[0051] Shooting angle matching: Select the shooting angle field of the crowdsourced video from the rainfall video template library and retrieve the crowdsourced video with the same shooting angle as the target crowdsourced video;

[0052] Image quality matching: Select the image quality field of the crowdsourced videos in the rainfall video template library and retrieve the crowdsourced videos with the same image quality as the target crowdsourced video;

[0053] Scene dynamics matching: Select the scene dynamics field of the crowdsourced videos from the rainfall video template library and retrieve the crowdsourced videos with the same dynamics as the target crowdsourced video;

[0054] The crowdsourced videos of the rainfall video template library selected through the first-level attribute matching enter the subsequent second-level attribute matching;

[0055] Step 3.2: Secondary attribute matching, including:

[0056] Scene element matching: Select scene element fields from the crowdsourced videos in the rainfall video template library and retrieve crowdsourced videos with the same scene element categories as the target crowdsourced video;

[0057] Calculation of scene element similarity: Calculate the similarity between scene elements based on the difference in the proportion of scene elements;

[0058] Step 3.3: Calculate the comprehensive similarity of the scene. The similarities of the primary attributes, secondary attributes, and tertiary attributes of the crowdsourced videos in the rainfall video template library are combined into a comprehensive similarity using the weighted average method:

[0059] S total (A,B)=w1·S1(A,B)+w2·S2(A,B)+w3·S3(A,B)

[0060] Among them: S1(A,B), S2(A,B), S3(A,B) are the similarities of the first-level attributes, second-level attributes, and third-level attributes respectively; w1, w2, w3 are the weights of the first-level attributes, second-level attributes, and third-level attributes respectively.

[0061] The present application is further improved in that: in step 4, the method for calculating the similarity of rainfall videos based on frequency domain analysis specifically includes the following steps:

[0062] Step 4.1: Perform frequency domain conversion on each frame of the target crowdsourcing video to extract the spectrum information of the target crowdsourcing video. Specifically, by performing frequency domain processing on each frame of the target crowdsourcing video signal, the distribution of the target crowdsourcing video frame on different frequency components is obtained. For each frame of the target crowdsourcing video signal s(t), the spectrum information S(f) of the target crowdsourcing video is extracted by Fourier transform:

[0063] S(f)=F{s(t)}

[0064] Where f represents frequency and F represents Fourier transform operation;

[0065] Step 4.2: Calculate the spectral kurtosis and spectral entropy of the target crowdsourcing video signal, and use the spectral kurtosis and spectral entropy to measure the rainfall characteristics of the target crowdsourcing video, where:

[0066] The calculation formula of spectral kurtosis is:

[0067]

[0068] in, is the mean of the target crowdsourced video spectrum, is the variance of the target crowdsourced video spectrum;

[0069] The calculation formula of spectral entropy is:

[0070]

[0071] Among them, P(f i ) represents the target crowdsourcing video frequency f i The energy density at , N is the total number of frequencies;

[0072] Step 4.3, Adaptive Frequency Selection: By calculating spectral kurtosis and spectral entropy, the effective frequency range of the rainfall signal in the target crowdsourced video is determined, and an adaptive method is used to select the most representative frequency band. The adaptive method includes:

[0073] Spectral kurtosis peak extraction: The frequency range corresponding to the first 30% peaks in the target crowdsourced video spectrum is selected as the preliminary high-frequency band;

[0074] Spectral entropy minimization: according to the spectral entropy value, the spectral entropy value within the frequency range is minimized;

[0075] Dynamic frequency band adjustment: Dynamically adjust the frequency band width according to the time-varying characteristics of rainfall signals;

[0076] Determine the effective frequency range f valid =[f min ,f max ], and perform subsequent similarity calculation based on this frequency band, f min is the minimum value of spectral entropy, f max is the peak value of spectral kurtosis;

[0077] Step 4.4, frequency domain similarity calculation: According to the adaptively selected effective frequency range f valid , calculate the similarity of rainfall signals between the crowdsourced videos in the rainfall video template library and the target crowdsourced video, and use the weighted frequency domain similarity measure to calculate the similarity S between the crowdsourced videos in the rainfall video template library and the target crowdsourced video signals freq :

[0078]

[0079] Among them, S freq is the frequency domain similarity, f valid is the effective frequency range, w(f) is the adaptive weight of the frequency component, X A (f) is the spectral component of the crowdsourced video in the rainfall video template library at frequency f, obtained by performing fast Fourier transform on each frame of the crowdsourced video in the rainfall video template library. A (f)| is the amplitude of the spectrum of video A, i.e., the crowdsourced video in the rainfall video template library, at frequency f, which represents the energy intensity of the frequency component. XB (f) is the spectral component of video B, i.e., the target crowdsourcing video, at frequency f, obtained by performing fast Fourier transform on each frame of video B, i.e., the target crowdsourcing video. B (f)| is the amplitude of the spectrum of video B, i.e., the target crowdsourcing video, at frequency f, which represents the energy intensity of video B at this frequency component, Σ fvalid |X A (f)| 2 is the total energy of the crowdsourced video in the effective frequency range in video A, i.e., the rainfall video template library, ∑ fvalid |X B (f)| 2 is the total energy of video B, the target crowdsourcing video, within the effective frequency range, and is the total energy of the normalized video A, i.e., the crowdsourced video in the rainfall video template library, and the normalized video B, i.e., the target crowdsourced video.

[0080] The beneficial effects of this application are:

[0081] This application does not require precise shooting parameters. By constructing a rainfall-related knowledge graph and combining it with scene features and precipitation information in crowdsourced videos for reasoning, it avoids the strong dependence of traditional methods on camera parameters and can effectively process crowdsourced videos that lack shooting parameters.

[0082] This application is based on knowledge graph reasoning. By constructing a multi-level knowledge graph, it associates and infers information such as the shooting environment, scene dynamics, and rainfall characteristics in crowdsourced videos. Using the rules in the knowledge graph, it automatically infers the rainfall intensity and type in crowdsourced videos, thus supporting rainfall estimation.

[0083] In summary, this application can effectively utilize crowdsourced videos for rainfall estimation, especially when shooting parameters are missing, providing a new rainfall monitoring solution. Through reasoning based on knowledge graphs, the limitations of traditional methods are overcome, not only enhancing the accuracy of rainfall estimation but also providing a low-cost, efficient technical path for future meteorological monitoring. While improving the accuracy of rainfall estimation, this method also provides a powerful supplement to traditional meteorological monitoring methods, providing a more flexible solution for rainfall monitoring in different scenarios and environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 This is a flowchart of the application.

[0085] Figure 2 This is a schematic diagram of the video attribute classification of this application.

[0086] Figure 3This is a schematic diagram of the presence and absence of rainfall obtained by the application through calculation of a 3-second video obtained by the application.

[0087] Figure 4 This is a schematic diagram of the presence and absence of rainfall obtained by calculating 7 3-second videos obtained by this application. DETAILED DESCRIPTION

[0088] The following drawings illustrate embodiments of the present invention. For clarity, many practical details are included in the following description. However, it should be understood that these practical details are not intended to limit the present invention. In other words, in some embodiments of the present invention, these practical details are not essential. Furthermore, to simplify the drawings, some commonly used structures and components are depicted in simplified schematic form.

[0089] like Figure 1 As shown, this application is a crowdsourcing video rainfall quantitative calculation method based on knowledge graph, and the rainfall quantitative calculation method specifically includes the following steps:

[0090] Step 1: Build a rainfall video template library, extract the video attributes of the crowdsourced videos in the library, and convert these attributes into structured data, storing them in a knowledge graph. Use a graph database such as Neo4j to store the knowledge graph to support efficient querying of complex relationships. Associate different attribute nodes with crowdsourced video clips to support fast retrieval and comparison. Specifically, building the rainfall video template library includes the following steps:

[0091] Step 1.1: Extract the primary video attributes of the crowdsourced videos in the rainfall video template library, including extracting the shooting time period, calculating the scene dynamics, extracting scene elements and analyzing their proportions, identifying shooting angles, and observing rainfall information.

[0092] Step 1.2: Convert the video attributes extracted from the crowdsourced video into structured data and store them in the nodes of the knowledge graph. In the knowledge graph, establish nodes and relationships. Nodes are crowdsourced video clips or crowdsourced video attributes in the rainfall video template library, and relationships are used to connect the crowdsourced video clips with the corresponding crowdsourced video attributes. The associations mainly include:

[0093] Attribute association: The association relationship between the first-level video attribute node, the second-level video attribute node, the third-level video attribute node and the video clip node.

[0094] Similarity association: Generate association relationships between similar videos based on the characteristics of video attributes to facilitate subsequent similar video recommendations and retrieval.

[0095] Temporal and spatial correlation: Build relationships between rainfall videos at the same location at different times to analyze rainfall changes over time.

[0096] Step 2: Perform data cleaning and preprocessing on the target crowdsourcing video that requires quantitative rainfall calculation, extract the video attributes of the target crowdsourcing video, and align the extracted video attributes of the target crowdsourcing video with the structured data stored in the knowledge graph of the rainfall video template library.

[0097] Data cleaning and preprocessing of target crowdsourced videos specifically include the following steps:

[0098] Step 2.1.1. Format unification: Convert the target crowdsourcing videos into MP4 format to ensure compatibility with subsequent processing.

[0099] Step 2.1.2, duration unification: If the duration of the target crowdsourcing video is longer than the length of the crowdsourcing video in the rainfall video template library, the target crowdsourcing video will be cut into multiple segments and subsequently processed in sequence.

[0100] Step 2.1.3, frame rate standardization: adjust the target crowdsourced video frame rate to be consistent with any frame rate in the rainfall video template library crowdsourced video, unify the playback speed of the target crowdsourced video and the rainfall video template library crowdsourced video, to facilitate subsequent feature extraction and matching;

[0101] Step 2.1.4, quickly filter out rain-free target crowdsourcing videos: By analyzing the texture changes of the target crowdsourcing video frames, use the Laplace operator to detect the amplitude of local brightness changes in the target crowdsourcing video image to determine whether there is rain in the target crowdsourcing video. If the texture changes are large, it means that there may be rain in the target crowdsourcing video. If the texture changes are small, it means that there is no rain in the target crowdsourcing video. For target crowdsourcing videos without rain, the rainfall intensity is determined to be 0, and subsequent calculations are no longer continued, that is, the subsequent rainfall intensity calculation steps are skipped. The Laplace operator can highlight the edges and rapidly changing areas in the target crowdsourcing video image by calculating the second-order derivative of the target crowdsourcing video image. Raindrops usually cause local texture disturbances, which appear as rapid changes or blurring effects in the image. The calculation method is as follows:

[0102]

[0103] Where I(x,y) is the pixel intensity of the target crowdsourcing video frame rate, Represents the Laplace operator or second-order derivative of the target crowdsourced video image at position (x, y), It is the second-order partial derivative of I(x,y) with respect to x, that is, first find the first-order partial derivative of I(x,y) with respect to x, and then find the first-order partial derivative of the result with respect to x, which describes the degree of bending of I(x,y) in the x direction. The second-order partial derivative with respect to y indicates the curvature of I(x,y) in the y direction. It is used to detect areas of brightness changes in an image.

[0104] like Figure 2 As shown, the video attributes of the crowdsourced videos in the rainfall video template library and the video attributes of the target crowdsourced video both include three levels of attributes, specifically:

[0105] First-level attributes: Video features are extracted through first-level attributes. Rainfall video features include: shooting time period, shooting angle, image quality, and scene dynamics.

[0106] Secondary attributes: The scene components and their proportions of the video are extracted through secondary attributes to reflect the differences in rainfall video content or video elements. The scene components include vehicles, pedestrians, trees, roads, and the sky.

[0107] Level 3 attributes: Rainfall feature information in the video is extracted through level 3 attributes, and the rainfall feature information includes rainfall intensity and rainfall level.

[0108] Extracting video attributes of crowdsourced videos in the rainfall video template library and extracting video attributes of target crowdsourced videos both include:

[0109] Extracting the shooting time period: This includes extracting time metadata, that is, extracting shooting time information from the video metadata. Shooting time information is usually included in the file attributes of crowdsourced videos. If metadata is missing, the time period is determined by calculating the overall brightness of the video. The average brightness of nighttime videos is lower, while that of daytime videos is higher. The determination method is:

[0110] a. Extract frames from the video and convert them into grayscale images so that only the brightness information is focused;

[0111] b. Calculate the mean pixel value mean_brightness of the grayscale image:

[0112]

[0113] Where gray(i) is the grayscale value of the i-th grayscale image, N is the total number of pixels, and a threshold T is set. If the value exceeds this threshold, it is judged to be daytime, and if the value is lower than the threshold, it is judged to be nighttime. The judgment conditions are:

[0114] if mean_brightness>T,then daytime

[0115] else,nighttime

[0116] By adjusting the threshold T, it can adapt to different lighting conditions;

[0117] Calculate the scene dynamics: Use the optical flow method to obtain motion vector data, quantify the degree of dynamics in the scene, and judge the dynamics, including the average size of the optical flow vector, the distribution of the moving object's movement direction, the proportion of high-motion areas, and the uniformity of movement. The specific calculation of the scene dynamics is as follows:

[0118] Average size of optical flow vectors: Use the optical flow method to detect the vector data of moving objects in the video, such as vehicles and pedestrians, and calculate the average size of the optical flow vectors of all video clip pixels:

[0119]

[0120] Among them, u i and v i are the optical flow components of pixel i in the video clip in the x and y directions, respectively, and N is the total number of pixels;

[0121] Distribution of moving object movement directions: The movement trend of objects in the scene is obtained by statistically analyzing the directional distribution of motion vectors. For example, if the flow of vehicles has a clear direction, such as from left to right, it means that the scene is highly dynamic.

[0122] High motion area ratio: Set the threshold of the optical flow vector size, identify the high dynamic area in the scene through the threshold of the optical flow vector size, and calculate the percentage of the threshold of the optical flow vector size in the total scene, reflecting the proportion of the high dynamic area in the scene:

[0123]

[0124] Among them, the number of high-motion pixels is the high-dynamic area, that is, the area of ​​fast-moving vehicles or pedestrians;

[0125] Motion uniformity: The standard deviation of the optical flow vector is used to determine the uniformity of motion. A higher standard deviation indicates that there are multiple moving objects at different speeds in the scene (for example, vehicles and pedestrians at the same time), indicating a strong dynamics.

[0126] The scene element extraction and proportion analysis specifically involves using the deep learning semantic segmentation model DeepLab V3+ to perform pixel-level segmentation on scene elements, determining the proportion of different elements in the image. The video is then fed frame by frame into the trained deep learning semantic segmentation model, which processes each frame and outputs a semantic segmentation result map of the same size as the input image. Each pixel corresponds to a classification label such as "car," "person," or "tree." The pixel values ​​of each segmentation map are statistically analyzed, and the pixel proportion of each type of element, such as vehicles, pedestrians, and trees, is calculated. This information is then converted into secondary attributes of the crowdsourced video.

[0127] Shooting angle recognition: This method uses horizon detection to identify video shooting angles. Specifically, the horizon is first identified based on the sky or road, a video feature identified by the DeepLab V3+ deep learning semantic segmentation model. The horizon's position in the video image is then calculated. Assuming the detected horizon is at position yh in the image and the image center is yc, the relative index of the pitch angle is calculated using the position difference:

[0128] relative angle=yh-yc.

[0129] The shooting angle is determined based on the positive and negative values ​​of the relative angle: relative angle = 0 means horizontal shooting; relative angle > 0 means upward shooting; relative angle < 0 means downward shooting. The above method can effectively determine the shooting angle type of the camera video, such as horizontal, downward, or upward, providing angle information for subsequent crowdsourced video analysis.

[0130] Observe rainfall information: Based on synchronized observation data from rain gauges, radar, etc., each video is equipped with a corresponding precipitation intensity label. Based on the rainfall intensity, rainfall level labels are established: light rain, moderate rain, heavy rain, and rainfall characteristic information is constructed: rainfall intensity and rainfall level.

[0131] Step 3: Based on the video attribute levels defined in the knowledge graph, the video attributes of the target crowdsourced video are matched with the video attributes of the crowdsourced videos in the rainfall video template library in sequence, and the video set in the rainfall video template library that is closest to the target crowdsourced video is selected. This specifically includes the following steps:

[0132] Step 3.1: First-level attribute matching, including:

[0133] The first-level attributes are the most basic attributes and are matched first. At this stage, the application can simply perform equal value matching, such as whether the shooting time period and shooting angle are consistent, or define a matching score for each attribute.

[0134] Time period matching: Select the time period field of the crowdsourced videos in the rainfall video template library and retrieve the crowdsourced videos with the same time period as the target crowdsourced video;

[0135] Shooting angle matching: Select the shooting angle field of the crowdsourced video from the rainfall video template library and retrieve the crowdsourced video with the same shooting angle as the target crowdsourced video;

[0136] Image quality matching: Select the image quality field of the crowdsourced videos in the rainfall video template library and retrieve the crowdsourced videos with the same image quality as the target crowdsourced video;

[0137] Scene dynamics matching: The scene dynamics field of the crowdsourced videos in the rainfall video template library is selected to retrieve crowdsourced videos with the same dynamics as the target crowdsourced video. In this embodiment, the dynamics threshold is set to ±0.3.

[0138] The crowdsourced videos of the rainfall video template library selected through the first-level attribute matching enter the subsequent second-level attribute matching;

[0139] Step 3.2: Secondary attribute matching, including:

[0140] Scene element matching: Select scene element fields from the crowdsourced videos in the rainfall video template library and retrieve crowdsourced videos with the same scene element categories as the target crowdsourced video;

[0141] Calculation of scene element similarity: Based on the difference in the proportion of scene elements, the similarity between scene elements is calculated using methods such as Euclidean distance and cosine similarity. For example, for the difference in the proportion of vehicles, the absolute difference in the proportion of vehicles in the crowdsourced video and the template library video can be calculated:

[0142]

[0143] Among them, x1 and x2 are the proportions of vehicles in videos A and B, and the same applies to other factors.

[0144] Step 3.3: Calculate the comprehensive scene similarity and select the video set in the rainfall video template library that is closest to the target crowdsourced video: The similarities of the primary attributes, secondary attributes, and tertiary attributes of the crowdsourced videos in the rainfall video template library are combined into a comprehensive similarity using the weighted average method:

[0145] S total (A,B)=w1·S1(A,B)+w2·S2(A,B)+w3·S3(A,B)

[0146] Among them: S1(A,B), S2(A,B), S3(A,B) are the similarities of the first-level attribute, second-level attribute, and third-level attribute respectively; w1, w2, w3 are the weights of the first-level attribute, second-level attribute, and third-level attribute, indicating the influence of the attribute in the total similarity.

[0147] Step 4: Based on the target crowdsourced video matched and screened in step 3 and the video collection of crowdsourced videos in the rainfall video template library, a rainfall video similarity calculation method based on frequency domain analysis is used to complete the similarity calculation.

[0148] The rainfall video similarity calculation method based on frequency domain analysis specifically includes the following steps:

[0149] Step 4.1: Perform frequency domain conversion on each frame of the target crowdsourcing video to extract the spectrum information of the target crowdsourcing video. Specifically, by performing frequency domain processing such as fast Fourier transform (FFT) or wavelet transform on each frame of the target crowdsourcing video signal, the distribution of the target crowdsourcing video frame on different frequency components is obtained. For each frame of the target crowdsourcing video signal s(t), the spectrum information S(f) of the target crowdsourcing video is extracted by Fourier transform:

[0150] S(f)=F{s(t)}

[0151] Where f represents frequency and F represents Fourier transform operation;

[0152] The frequency characteristics of the rainfall signal are calculated based on the spectral information, focusing on high-frequency components that may appear in the video, such as the vibration signals of falling raindrops, and low-frequency components, such as the periodic patterns of rainfall intensity changes.

[0153] Step 4.2: To effectively identify the frequency domain characteristics of rainfall signals, calculate the spectral kurtosis and spectral entropy of the target crowdsourcing video signal, and use the spectral kurtosis and spectral entropy to measure the rainfall characteristics of the target crowdsourcing video.

[0154] Spectral kurtosis calculation: Spectral kurtosis is used to measure the sharpness of the spectrum. Higher spectral kurtosis is usually associated with short-term and strongly changing signals such as rainfall.

[0155] The calculation formula of spectral kurtosis is:

[0156]

[0157] in, is the mean of the target crowdsourced video spectrum, is the variance of the target crowdsourced video spectrum;

[0158] Spectral entropy calculation: Valley entropy is used to describe the uniformity of spectrum distribution. A higher spectral entropy value indicates a more complex signal. Rainfall signals usually have higher spectral entropy. The formula for calculating spectral entropy is:

[0159]

[0160] Among them, P(f i ) represents the target crowdsourcing video frequency f i The energy density at , N is the total number of frequencies;

[0161] Step 4.3, Adaptive Frequency Selection: After extracting spectral features, the effective frequency range of the rainfall signal in the target crowdsourced video is determined by calculating spectral kurtosis and spectral entropy. An adaptive method is then used to select the most representative frequency band. The adaptive method includes:

[0162] Spectral kurtosis peak extraction: Select frequency bands with high spectral kurtosis in the target crowdsourced video spectrum. These bands usually contain high-frequency components of rainfall signals. The frequency range corresponding to the first 30% peaks in the target crowdsourced video spectrum is selected as the preliminary high-frequency band.

[0163] Spectral entropy minimization: Based on the spectral entropy value, a frequency range is selected to minimize the spectral entropy within that frequency range, ensuring that the signal within the selected frequency band is not affected by excessive background noise. Spectral entropy reflects the uncertainty of the signal spectrum. A lower spectral entropy value indicates that the signal's frequency components are concentrated in a smaller frequency band, indicating a more ordered and deterministic system. Conversely, a higher spectral entropy value indicates that the signal's energy is widely distributed across multiple frequency bands, making the system more unstable. By minimizing spectral entropy, the frequency band that best represents the rainfall signal can be found.

[0164] Specifically, a sliding window method is used to gradually divide the target crowdsourced video spectrum into different frequency bands and calculate the spectral entropy of each band. The spectral entropy of each band reflects the uniformity of the signal energy distribution within that band, with smaller values ​​indicating more concentrated energy. Among all the frequency bands, the band with the smallest spectral entropy is selected as the target frequency range, representing the core characteristics of the rainfall signal. Based on the time-varying nature of the signal, the bandwidth is dynamically adjusted according to different environmental conditions to capture the changes in the rainfall signal under different circumstances, ensuring that the frequency range always adapts to the changes in the signal.

[0165] Dynamic Bandwidth Adjustment: Dynamically adjusts the bandwidth based on the time-varying characteristics of rainfall signals. In heavy rainfall or high wind speeds, the frequency domain characteristics of rainfall signals may expand to lower frequency bands. The bandwidth is automatically adjusted to capture these signal changes.

[0166] The present application provides a dynamic frequency band adjustment method based on the time-varying characteristics of rainfall signals, which automatically adjusts the frequency bandwidth to adapt to changes in the signal frequency domain, ensuring that changes in rainfall signals are effectively captured. When the rainfall signal intensity is large or the wind speed is high, the frequency domain characteristics of the rainfall signal may extend to lower frequency bands, and the traditional frequency band setting may not be able to fully cover these changes, so it is necessary to dynamically adjust the frequency bandwidth. First, the power spectral density (PSD) of the rainfall signal is calculated to obtain the frequency distribution of the signal. Then, based on the spectrum data, the Gaussian model is used to fit the spectrum curve, and the least squares method is used to optimize the fitting parameters to determine the main frequency and bandwidth of the spectrum. Finally, according to the changes in the spectrum characteristics of the signal, the bandwidth is automatically adjusted. When the spectrum of the rainfall signal expands, the bandwidth is increased to capture the low-frequency components; when the spectrum is concentrated, the bandwidth is contracted to improve the accurate capture of high-frequency signals.

[0167] Finally, determine the effective frequency range f valid =[f min ,f max ], and perform subsequent similarity calculation based on this frequency band, f min is the minimum value of spectral entropy, f max is the peak value of spectral kurtosis;

[0168] Step 4.4, frequency domain similarity calculation: According to the adaptively selected effective frequency range f valid , calculate the similarity of rainfall signals between the crowdsourced videos in the rainfall video template library and the target crowdsourced video, and use the weighted frequency domain similarity measure to calculate the similarity S between the crowdsourced videos in the rainfall video template library and the target crowdsourced video signals freq :

[0169]

[0170] Among them, S freq is the frequency domain similarity, which represents the similarity of the two videos in frequency domain features. The larger the value, the more similar the rainfall signals in the videos are. valid is the effective frequency range, which is obtained through adaptive frequency selection and contains the frequency components of the rainfall signal. w(f) is the adaptive weight of the frequency component. It is used to weight the importance of the frequency component and can adjust the weight according to the contribution of different frequency components to the rainfall signal. A (f) is the spectral component of the crowdsourced video in the rainfall video template library at frequency f, obtained by performing fast Fourier transform on each frame of video A. A (f)| is the amplitude of the spectrum of video A at frequency f, which represents the energy intensity of the frequency component, X B(f) is the spectral component of video B, the target crowdsourcing video, at frequency f, obtained by performing fast Fourier transform on each frame of video B. B (f)| is the amplitude of the spectrum of video B, i.e., the target crowdsourcing video, at frequency f, which represents the energy intensity of video B at this frequency component. fvalid |X A (f)| 2 is the total energy of video A within the effective frequency range, ∑ fvalid |X B (f)| 2 is the total energy of video B in the effective frequency range, and To normalize video A and video B, ensure that the calculation of frequency domain similarity is not affected by the difference in video signal strength.

[0171] Step 5: Compare the similarity calculated in step 4 with the preset similarity threshold to determine whether the rainfall signals between the crowdsourced video in the rainfall video template library and the target video are similar. Finally, the rainfall information of the most matching video in the rainfall video template library is assigned to the target crowdsourced video to achieve quantitative calculation of the rainfall in the target crowdsourced video.

[0172] This application obtains two videos of 3 seconds each, labeled A and B. Figure 3 As shown, Figure A shows the street scene on a rainy day as seen from the car window. The ground is wet and the environment has obvious characteristics of rainfall; Figure B shows the scene of a city overpass and road at night, and there is no sign of rainfall on the road surface.

[0173] The two original videos were processed and their average Laplacian variance was calculated. The average Laplacian variance measures the degree of texture variation within a video frame; a larger variance indicates a more significant texture change. The average Laplacian variance for video A is 390.95, while the average Laplacian variance for video B is 72.58.

[0174] Rainfall detection is performed based on the mean Laplace operator variance. When the variance exceeds the threshold value of 100 set in this embodiment, it is determined that there is rainfall in the video. The final rainfall detection conclusion is: there is rainfall in video A, but not in video B.

[0175] like Figure 4 As shown, this application obtained 7 original videos, each 3 seconds long, marked as A, B, C, D, E, F, and G. Among them, picture A shows raindrops falling on the car window; picture B shows the scenery beside the car window; picture C shows the road in the rain; picture D is the road scene under the lights at night; picture E is a picture of the roof and some trees; picture F is a flat road surface; picture G is a picture of the eaves and some plant branches and leaves.

[0176] After Fourier transforming these seven original videos, the corresponding spectrograms were obtained. Spectrograms reflect the characteristic distribution of the video in the frequency domain. The spectral kurtosis and spectral entropy values ​​for each video were also calculated. Spectral kurtosis is a measure of the impact of a signal; a higher value indicates a more significant impact component. The spectral kurtosis values ​​for each video are: A: 656716.1, B: 433064.5, C: 851677.1, D: 471731.1, E: 293082.3, F: 1917396.0, and G: 1251912.7. Spectral entropy describes the uncertainty or randomness of a signal; a higher value indicates greater signal uncertainty. The spectral entropy values ​​for each video are: A: 3.1, B: 3.2, C: 0.9, D: 3.3, E: 4.0, F: 0.7, and G: 2.2.

[0177] Through comprehensive analysis of these data, we can gain a preliminary understanding of the differences in frequency domain characteristics between videos of different scenarios. For example, the spectral entropies of A, B, and E are relatively close, indicating that they depict vehicle-related scenes. The spectral kurtosis of B and E is similar, indicating that both videos contain street elements based on the vehicle. The frequency spectra and spectral entropies of C and E are similar, indicating that these two videos depict scenes of roads on rainy days.

[0178] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A crowdsourcing video rainfall quantitative calculation method based on knowledge graph, characterized by: The rainfall quantitative calculation method specifically comprises the following steps: Step 1: Build a rainfall video template library, extract the video attributes of the crowdsourced videos in the rainfall video template library, and convert the video attributes of the crowdsourced videos in the rainfall video template library into structured data and store them in the knowledge graph; Step 2: Perform data cleaning and preprocessing on the target crowdsourced video for rainfall quantitative calculation, extract the video attributes of the target crowdsourced video, and align the extracted video attributes of the target crowdsourced video with the structured data stored in the knowledge graph of the rainfall video template library; Step 3: According to the video attribute levels defined in the knowledge graph, the video attributes of the target crowdsourced video are matched with the video attributes of the crowdsourced videos in the rainfall video template library in sequence, and the video set in the rainfall video template library that is closest to the target crowdsourced video is selected; Step 4: The target crowdsourced video matched and screened in step 3 is compared with the video collection of crowdsourced videos in the rainfall video template library, and the rainfall video similarity calculation method based on frequency domain analysis is used to complete the similarity calculation; Step 5: Compare the similarity calculated in step 4 with the preset similarity threshold to determine whether the rainfall signals between the crowdsourced video in the rainfall video template library and the target video are similar. Finally, the rainfall information of the most matching video in the rainfall video template library is assigned to the target crowdsourced video to achieve quantitative calculation of the rainfall in the target crowdsourced video.

2. The method for quantitatively calculating rainfall from crowdsourced videos based on knowledge graph according to claim 1 is characterized by: The video attributes of the crowdsourced videos in the rainfall video template library and the video attributes of the target crowdsourced video both include three levels of attributes, specifically: First-level attributes: Video features are extracted through first-level attributes. Rainfall video features include: shooting time period, shooting angle, image quality, and scene dynamics. Secondary attributes: Extract the scene components of the video through secondary attributes; Level 3 attributes: Rainfall feature information in the video is extracted through level 3 attributes, and the rainfall feature information includes rainfall intensity and rainfall level.

3. The method for quantitatively calculating rainfall from crowdsourced videos based on knowledge graph according to claim 2 is characterized by: In step 1, the construction of the rainfall video template library specifically includes the following steps: Step 1.1: Extract the primary video attributes of the crowdsourced videos in the rainfall video template library, including extracting the shooting time period, calculating the scene dynamics, extracting scene elements and analyzing their proportions, identifying shooting angles, and observing rainfall information. Step 1.2: Convert the video attributes extracted from the crowdsourced video into structured data and store them in the nodes of the knowledge graph. In the knowledge graph, establish nodes and relationships. The nodes are crowdsourced video clips or crowdsourced video attributes in the rainfall video template library, and the relationships are used to connect the associations between the crowdsourced video clips and the corresponding crowdsourced video attributes.

4. The method for quantitatively calculating rainfall from crowdsourced videos based on knowledge graph according to claim 3 is characterized by: Step 2 of performing data cleaning and preprocessing on the target crowdsourced video specifically includes the following steps: Step 2.1.1, format unification: convert the target crowdsourcing videos into MP4 format; Step 2.1.2, duration unification: If the duration of the target crowdsourced video is longer than the length of the crowdsourced videos in the rainfall video template library, the target crowdsourced video is cut into multiple segments; Step 2.1.3, frame rate standardization: adjust the frame rate of the target crowdsourced video to be consistent with any frame rate in the crowdsourced videos in the rainfall video template library, and unify the playback speed of the target crowdsourced video and the crowdsourced videos in the rainfall video template library; Step 2.1.4: Quickly filter target crowdsourcing videos without rain: By analyzing the texture changes of the target crowdsourcing video frames, the Laplace operator is used to detect the amplitude of local brightness changes in the target crowdsourcing video image to determine whether there is rain in the target crowdsourcing video. If the texture change is large, it means that there is rain in the target crowdsourcing video. If the texture change is small, it means that there is no rain in the target crowdsourcing video. For target crowdsourcing videos without rain, the rainfall intensity is determined to be 0. The calculation method is as follows: Where I(x,y) is the pixel intensity of the target crowdsourcing video frame rate, ▽ 2 I(x,y) represents the Laplace operator or second-order derivative of the target crowdsourced video image at position (x,y). It is the second-order partial derivative of I(x,y) with respect to x, that is, first find the first-order partial derivative of I(x,y) with respect to x, and then find the first-order partial derivative of the result with respect to x. The second-order partial derivative with respect to y indicates the degree of curvature of I(x,y) in the y direction.

5. The method for quantitatively calculating rainfall from crowdsourced videos based on knowledge graph according to claim 4 is characterized in that: Step 1 extracting the video attributes of the crowdsourced videos in the rainfall video template library and step 2 extracting the video attributes of the target crowdsourced video both include: Extracting the shooting time period: This includes extracting time metadata, that is, extracting the shooting time information from the video metadata. If the metadata is missing, the time period is determined by calculating the overall brightness of the video. The determination method is: a. Extract frames from the video and convert them into grayscale images; b. Calculate the mean pixel value mean_brightness of the grayscale image: Where gray(i) is the grayscale value of the i-th grayscale image, N is the total number of pixels, and a threshold T is set. If the value exceeds this threshold, it is judged to be daytime, and if the value is lower than the threshold, it is judged to be nighttime. The judgment conditions are: Calculate scene dynamics: Use optical flow to obtain motion vector data, quantify the degree of dynamics in the scene, and determine dynamics, including the average size of the optical flow vector, the distribution of the moving object's motion direction, the proportion of high-motion areas, and the uniformity of motion. The scene element extraction and proportion analysis specifically involves using a deep learning semantic segmentation model to segment scene elements at the pixel level, obtaining the proportion of different elements in the picture. The video is then fed frame by frame into the trained deep learning semantic segmentation model, which processes each frame and outputs a semantic segmentation result map of the same size as the input image. Each pixel is assigned a classification label. The pixel values ​​of each segmentation map are counted, and the pixel proportion of each type of element is calculated. This information is then converted into secondary attributes of the crowdsourced video. Shooting angle recognition: Specifically, first, the horizon line is identified based on the video elements (i.e., sky or road) identified by the deep learning semantic segmentation model. Then, the position of the horizon in the video image is calculated. Assuming that the detected horizon is at position yh in the image and the image center is yc, the relative index of the pitch angle is calculated based on the position difference: relative angle=yh-yc. The shooting angle is determined by the positive and negative values ​​of the relative angle: relative angle = 0 means horizontal shooting; relative angle > 0 means upward shooting; relative angle < 0 means downward shooting; Observe rainfall information: By synchronizing observation data, each video is equipped with a corresponding precipitation intensity label. Based on the rainfall intensity, rainfall level labels are established: light rain, moderate rain, heavy rain, and rainfall characteristic information is constructed: rainfall intensity and rainfall level.

6. The method for quantitatively calculating rainfall from crowdsourced videos based on knowledge graph according to claim 5 is characterized by: The calculation of scene dynamics is as follows: Average size of optical flow vectors: Use the optical flow method to detect the vector data of moving objects in the video, such as vehicles and pedestrians, and calculate the average size of the optical flow vectors of all video clip pixels: Among them, u i and v i are the optical flow components of pixel i in the video clip in the x and y directions, respectively, and N is the total number of pixels; Distribution of moving object's motion direction: Obtain the motion trend of objects in the scene by statistically analyzing the direction distribution of motion vectors; High motion area ratio: Set the threshold of the optical flow vector size to identify the high dynamic area in the scene, reflecting the proportion of the high dynamic area in the scene: Among them, Number of high-motion pixels is the high-dynamic area, that is, the area of ​​fast-moving vehicles or pedestrians; Motion uniformity: The uniformity of motion is judged by the standard deviation of the optical flow vector.

7. The method for quantitatively calculating rainfall from crowdsourced videos based on knowledge graph according to claim 6 is characterized by: Step 3 sequentially matches the video attributes of the target crowdsourced video with the video attributes of the crowdsourced videos in the rainfall video template library, specifically including the following steps: Step 3.1: First-level attribute matching, including: Time period matching: Select the time period field of the crowdsourced videos in the rainfall video template library and retrieve the crowdsourced videos with the same time period as the target crowdsourced video; Shooting angle matching: Select the shooting angle field of the crowdsourced video from the rainfall video template library and retrieve the crowdsourced video with the same shooting angle as the target crowdsourced video; Image quality matching: Select the image quality field of the crowdsourced videos in the rainfall video template library and retrieve the crowdsourced videos with the same image quality as the target crowdsourced video; Scene dynamics matching: Select the scene dynamics field of the crowdsourced videos from the rainfall video template library and retrieve the crowdsourced videos with the same dynamics as the target crowdsourced video; The crowdsourced videos of the rainfall video template library selected through the first-level attribute matching enter the subsequent second-level attribute matching; Step 3.2: Secondary attribute matching, including: Scene element matching: Select scene element fields from the crowdsourced videos in the rainfall video template library and retrieve crowdsourced videos with the same scene element categories as the target crowdsourced video; Calculation of scene element similarity: Calculate the similarity between scene elements based on the difference in the proportion of scene elements; Step 3.3: Calculate the comprehensive similarity of the scene. The similarities of the primary attributes, secondary attributes, and tertiary attributes of the crowdsourced videos in the rainfall video template library are combined into a comprehensive similarity using the weighted average method: S total (A,B)=w1·S1(A,B)+w2·S2(A,B)+w3·S3(A,B) Among them: S1(A,B), S2(A,B), S3(A,B) are the similarities of the first-level attributes, second-level attributes, and third-level attributes respectively; w1, w2, w3 are the weights of the first-level attributes, second-level attributes, and third-level attributes respectively.

8. The method for quantitatively calculating rainfall from crowdsourced videos based on knowledge graph according to claim 1 is characterized by: In step 4, the rainfall video similarity calculation method based on frequency domain analysis specifically includes the following steps: Step 4.1: Perform frequency domain conversion on each frame of the target crowdsourcing video to extract the spectrum information of the target crowdsourcing video. Specifically, by performing frequency domain processing on each frame of the target crowdsourcing video signal, the distribution of the target crowdsourcing video frame on different frequency components is obtained. For each frame of the target crowdsourcing video signal s(t), the spectrum information S(f) of the target crowdsourcing video is extracted by Fourier transform: S(f)=F{s(t)} Where f represents frequency and F represents Fourier transform operation; Step 4.2: Calculate the spectral kurtosis and spectral entropy of the target crowdsourcing video signal, and use the spectral kurtosis and spectral entropy to measure the rainfall characteristics of the target crowdsourcing video, where: The calculation formula of spectral kurtosis is: in, is the mean of the target crowdsourced video spectrum, is the variance of the target crowdsourced video spectrum; The calculation formula of spectral entropy is: Among them, P(f i ) represents the target crowdsourcing video frequency f i The energy density at , N is the total number of frequencies; Step 4.3, Adaptive Frequency Selection: By calculating spectral kurtosis and spectral entropy, the effective frequency range of the rainfall signal in the target crowdsourced video is determined, and an adaptive method is used to select the most representative frequency band. The adaptive method includes: Spectral kurtosis peak extraction: The frequency range corresponding to the first 30% peaks in the target crowdsourced video spectrum is selected as the preliminary high-frequency band; Spectral entropy minimization: according to the spectral entropy value, the spectral entropy value within the frequency range is minimized; Dynamic frequency band adjustment: Dynamically adjust the frequency band width according to the time-varying characteristics of rainfall signals; Determine the effective frequency range f valid =[f min ,f max ], and perform subsequent similarity calculation based on this frequency band, f min is the minimum value of spectral entropy, f max is the peak value of spectral kurtosis; Step 4.4, frequency domain similarity calculation: According to the adaptively selected effective frequency range f valid , calculate the similarity of rainfall signals between the crowdsourced videos in the rainfall video template library and the target crowdsourced video, and use the weighted frequency domain similarity measure to calculate the similarity S between the crowdsourced videos in the rainfall video template library and the target crowdsourced video signals freq : Among them, S freq is the frequency domain similarity, f valid is the effective frequency range, w(f) is the adaptive weight of the frequency component, X A (f) is the spectral component of the crowdsourced video in the rainfall video template library at frequency f, obtained by performing fast Fourier transform on each frame of the crowdsourced video in the rainfall video template library. A (f)| is the amplitude of the spectrum of video A, i.e., the crowdsourced video in the rainfall video template library, at frequency f, which represents the energy intensity of the frequency component. X B (f) is the spectral component of video B, i.e., the target crowdsourcing video, at frequency f, obtained by performing fast Fourier transform on each frame of video B, i.e., the target crowdsourcing video. B (f)| is the amplitude of the spectrum of video B, i.e., the target crowdsourcing video, at frequency f, which represents the energy intensity of video B at this frequency component. fvalid |X A (f)| 2 is the total energy of the crowdsourced video in the effective frequency range in video A, i.e., the rainfall video template library, ∑ fvalid |X B (f)| 2 is the total energy of video B, the target crowdsourcing video, within the effective frequency range, and is the total energy of the normalized video A, i.e., the crowdsourced video in the rainfall video template library, and the normalized video B, i.e., the target crowdsourced video.

Images