A smart rural monitoring system based on multi-dimensional data analysis

Abstract

The present invention relates to the field of smart rural monitoring, and in particular to a smart rural monitoring system for multidimensional data analysis. The present invention is provided with a flight controller, which is used to control a number of unmanned aerial vehicles equipped with image acquisition units to fly along a predetermined path to collect night images of a number of rural areas; a feature analyzer, which is used to receive night images of different rural areas and perform texture analysis based on the night images; the image analyzer includes a clustering unit and an enhancement unit: the clustering unit is used to determine the texture interference characterization value of the night image acquired in the area based on the texture feature for the texture masking interference value and similarity of the night recognition target, so as to divide the texture interference category; the enhancement unit responds to the division result of the clustering unit, and is used to adaptively process the night images of each rural area. The present invention can more accurately identify smoke characteristics and timely alarm on the premise of ensuring data reliability, avoid the occurrence of fire, and save computing resources at the same time.

Description

Technical Field

[0001] The present invention relates to the field of smart rural monitoring, and in particular to a smart rural monitoring system for multi-dimensional data analysis. Background Art

[0002] At present, the vegetation coverage rate in rural areas of my country is relatively high. Fires occur due to environmental factors or other human factors. For example, wildfires are caused by the combined effects of geographical location and weather changes. Rural residents do not pay attention to fire regulations in densely vegetated areas and cause fires. Therefore, it is crucial to predict and warn of fires. Artificial intelligence technologies such as deep learning are used to intelligently identify and analyze the multi-dimensional data in the collected images to achieve high-precision and high-sensitivity fire and smoke detection.

[0003] In actual situations, due to the influence of rural vegetation cover, the fire source is obscured and difficult to find, especially in the early stages of a fire. The fire source is weak or cannot be directly observed, and only smoke characteristics can be observed under the influence of vegetation cover. Especially when monitoring large rural areas at night, identifying smoke characteristics requires image enhancement, which requires processing massive amounts of data and has a high data processing load. Summary of the invention

[0004] To this end, the present invention provides a smart rural monitoring system for multi-dimensional data analysis to overcome the problem in the prior art that the fire source is obscured and difficult to detect due to the influence of rural vegetation cover, especially in the early stage of a fire, the fire source is weak or cannot be directly observed, and only smoke characteristics can be observed under the influence of vegetation cover. Especially when monitoring large rural areas at night, image enhancement is required to identify smoke characteristics, massive data needs to be processed, and the data processing load is high.

[0005] To achieve the above objectives, the present invention provides a smart rural monitoring system for multi-dimensional data analysis, which includes:

[0006] A flight controller, which is used to control a number of drones equipped with image acquisition units to fly along a predetermined path to collect nighttime images of a number of rural areas;

[0007] A feature analyzer connected to each flight controller is used to receive night images of different rural areas and perform texture analysis based on the night images, including obtaining a plurality of frames of night images of a single area to annotate texture features, determining a texture masking interference value of the texture features for a night recognition target, and performing image enhancement on the night images to determine the similarity of the texture features before and after enhancement;

[0008] An image analyzer, which is connected to the flight controller and the feature analyzer respectively, comprises a clustering unit and an enhancement unit,

[0009] The clustering unit is used to determine the texture interference representation value of the night image acquired in the area based on the texture masking interference value of the night recognition target and the similarity of the texture feature to divide the texture interference category;

[0010] The enhancement unit is responsive to the division result of the clustering unit to process the nighttime images of each rural area, including:

[0011] Extracting several frames of nighttime images for a rural area and comparing them with a basic reference image of the rural area to determine the similarity of texture features, and determining whether to enhance the nighttime images in the rural area based on the similarity and changes in the several frames of nighttime images;

[0012] or, enhancing a plurality of frames of nighttime images in the rural area;

[0013] An early warning device is connected to the image analyzer to identify abnormal smoke features in the enhanced nighttime image and determine whether to issue an early warning prompt.

[0014] Furthermore, the feature analyzer is used to obtain the annotated texture features of several frames of nighttime images for a single area, and determine the texture masking interference value of the texture features for the nighttime recognition target, including:

[0015] It is used to grayscale each frame of nighttime image and identify several clusters in the grayscale image;

[0016] It is used to mark the effective clusters based on the brightness values ​​corresponding to the clusters, and use the effective clusters as texture features;

[0017] It is used to obtain the similarity variance of several texture features in each frame of nighttime images, and obtain the texture masking interference value of the texture feature for the nighttime recognition target.

[0018] Furthermore, the clustering unit is used to identify the texture masking interference value of the target at night based on the texture feature and the texture interference characterization value of the night image acquired in the similarity determination area, including:

[0019] The ratio of the texture masking interference value of the texture feature for nighttime recognition target to the texture masking interference threshold is calculated and the corresponding weight coefficient is assigned as the first interference feature;

[0020] To calculate the ratio of the preset similarity threshold to the similarity of the texture feature and assign a corresponding weight coefficient as the second interference feature;

[0021] The sum of the first interference feature and the second interference feature is determined as a texture interference characterization value.

[0022] Furthermore, the clustering unit is used to classify the texture interference categories, including:

[0023] If the texture interference representation value is greater than or equal to the texture interference representation threshold, the night image is classified as a strong interference category;

[0024] If the texture interference representation value is less than the texture interference representation threshold, the night image is a weak interference category.

[0025] Furthermore, the enhancement unit is used to process the nighttime images of each rural area, including:

[0026] If the night image is of the weak interference category, extract several frames of night images for the rural area and compare them with the basic reference image of the rural area to determine the similarity of texture features, and determine whether to enhance the night image in the rural area based on the similarity and the changes of several consecutive frames of night images;

[0027] If the night image is of a strong interference category, several frames of night images in the rural area are enhanced.

[0028] Furthermore, the feature parser is also used to determine a basic reference image, including:

[0029] To analyze a number of frames of night images of each rural area and compare the similarity of each night image with the remaining night images;

[0030] The night image corresponding to the maximum similarity is determined as the basic reference image of the rural area.

[0031] Furthermore, the enhancement unit is used to determine the change of several frames of nighttime images, including:

[0032] Used to compare the similarity of texture features of adjacent nighttime images;

[0033] It is used to obtain the variance of each similarity as a change representation value.

[0034] Furthermore, the enhancing unit is used to determine whether to enhance the nighttime image in the rural area, including:

[0035] If the similarity is greater than a preset reference similarity threshold and the change representation value is less than a preset change representation value threshold, it is determined that the night image in the rural area needs to be enhanced.

[0036] Furthermore, the warning device is used to determine whether to issue a warning prompt, including:

[0037] If abnormal smoke features are found in the enhanced nighttime image, an early warning will be issued.

[0038] Furthermore, the image analyzer also includes a storage unit for storing the enhanced nighttime image.

[0039] Compared with the prior art, the present invention is provided with a flight controller, which is used to control a number of unmanned aerial vehicles equipped with image acquisition units to fly along a predetermined path to collect night images of a number of rural areas; a feature analyzer, which is used to receive night images for different rural areas and perform texture analysis based on the night images; an image analyzer, including a clustering unit and an enhancement unit: the clustering unit is used to determine the texture interference characterization value of the night images acquired in the area based on the texture features and similarity of the night images to divide the texture interference category; the enhancement unit responds to the division result of the clustering unit to adaptively process the night images of each rural area. The present invention can more accurately identify smoke characteristics and give an alarm in time to avoid the occurrence of fire while ensuring data reliability, and can save computing resources at the same time.

[0040] In particular, the present invention analyzes night images of different rural areas through a feature analyzer for classification. In actual situations, night images are relatively fuzzy and fewer texture features can be extracted. For example, in the night images corresponding to some areas of farmland with high planting density, only the texture features of the farmland boundaries may be extracted. For some farmland with sparse planting, the influence of plant boundaries will affect the image clustering effect, resulting in more discrete texture features. When there is smoke at night, due to the influence of image blur, the smoke may be presented as relatively dense texture features in the night image, especially under the influence of surrounding discrete texture features, it is not easy to be identified. Therefore, it is considered to determine the texture masking interference value of the texture feature for the night recognition target in combination with the discrete type of the texture feature, and further determine and calculate the texture interference characterization value to characterize the interference of the texture features in the night images in the corresponding rural area to the texture features generated when smoke occurs, so as to adaptively select the image enhancement method for the subsequent classification of the texture interference category of the night images in the rural area, provide data support, and thus improve the accuracy of identifying potential fires at night.

[0041] In particular, the image analyzer of the present invention determines the texture interference category of the night images acquired in the rural area, and adaptively selects the image enhancement method. Under the weak texture interference category, considering that the texture features of the smoke are less disturbed, the acquired night images are directly compared with the basic reference images. If there are smoke texture features, the similarity of the texture features determined by the comparison will be high, and the image enhancement can quickly identify the smoke features while ensuring reliability. Moreover, since the smoke occurs continuously, in order to avoid misjudgment, the changes of several texture features in consecutive frames are considered, and comprehensive verification is performed. Then, only the night images with potential anomalies are enhanced and accurately determined. Then, under the premise of ensuring reliability, the amount of data analysis is reduced when conducting night supervision in large rural areas to ensure the recognition of fire anomalies.

[0042] In particular, in the strong texture interference category, the smoke texture features in the night image are interfered by the surrounding discrete texture features and have poor accuracy. Therefore, in order to ensure recognition accuracy, all night images corresponding to the rural area are enhanced to ensure the reliability of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A functional module diagram of a smart rural monitoring system for multi-dimensional data analysis according to an embodiment of the invention;

[0044] Figure 2 A logical decision diagram for classifying texture interference categories for an embodiment of the invention;

[0045] Figure 3 A logical decision diagram for processing nighttime images of various rural areas for an embodiment of the invention;

[0046] Figure 4 This is a logical decision diagram for determining whether to issue a warning prompt according to an embodiment of the invention. DETAILED DESCRIPTION

[0047] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0049] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the term "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] See also Figures 1 to 4 As shown, Figure 1 This is a functional module diagram of a smart rural monitoring system for multi-dimensional data analysis according to an embodiment of the present invention. Figure 2 A logical decision diagram for classifying texture interference categories according to an embodiment of the present invention, Figure 3 This is a logic decision diagram for processing nighttime images of various rural areas in an embodiment of the present invention. Figure 4 This is a logical decision diagram for determining whether to issue an early warning prompt in an embodiment of the present invention. The smart rural monitoring system for multidimensional data analysis in an embodiment of the present invention includes:

[0051] A flight controller, which is used to control a number of drones equipped with image acquisition units to fly along a predetermined path to collect nighttime images of a number of rural areas;

[0052] A feature analyzer connected to each flight controller is used to receive night images of different rural areas and determine the degree of masking of texture features in the night images for night recognition targets, including obtaining a plurality of frames of night images for a single area and annotating texture features, determining a texture masking interference value of the texture features for night recognition targets, and performing image enhancement on the night images to determine the similarity of texture features before and after enhancement;

[0053] An image analyzer, which is connected to the flight controller and the feature analyzer respectively, comprises a clustering unit and an enhancement unit,

[0054] The clustering unit is used to determine the texture interference representation value of the night image acquired in the area based on the texture masking interference value and the similarity, so as to classify the texture interference category;

[0055] The enhancement unit is responsive to the division result of the clustering unit to process the nighttime images of each rural area, including:

[0056] Extracting several frames of nighttime images for a rural area and comparing them with a basic reference image of the rural area to determine the similarity of texture features, and determining whether to enhance the nighttime images in the rural area based on the similarity and changes in the several frames of nighttime images;

[0057] or, enhancing a plurality of frames of nighttime images in the rural area;

[0058] An early warning device is connected to the image analyzer to identify abnormal smoke features in the enhanced nighttime image and determine whether to issue an early warning prompt.

[0059] Specifically, there is no limitation on the specific structure of the image acquisition unit, which only needs to have the function of acquiring nighttime images of rural areas, such as a high-definition camera, which will not be elaborated here.

[0060] Specifically, there is no limitation on the specific structures of the flight controller, feature analyzer, image analyzer and early warning device, and they themselves or each unit therein can be composed of logic components or a combination of logic components, and the logic components include a field programmable processor, a computer or a microprocessor in a computer.

[0061] Specifically, there is no specific limitation on the predetermined path of the drone, and technical personnel in this field can set it according to the specific situation, as long as it can cover the rural area that needs to be monitored. It can be understood that images can be taken at several fixed positions in the form of shooting points on the predetermined path with the same shooting angle to facilitate comparison of images acquired at different times. This will not be elaborated here.

[0062] Specifically, the feature analyzer is used to obtain the annotated texture features of several frames of nighttime images for a single area, and determine the texture masking interference value of the texture features for the nighttime recognition target, including:

[0063] It is used to grayscale each frame of nighttime image and identify several clusters in the grayscale image;

[0064] It is used to mark the effective clusters based on the brightness values ​​corresponding to the clusters, and use the effective clusters as texture features;

[0065] It is used to obtain the similarity variance of several texture features in each frame of nighttime images, and obtain the texture masking interference value of the texture feature for the nighttime recognition target.

[0066] It is understandable that the clusters in the night image represent the texture. For clusters with lower brightness, the boundaries are unclear, which affects subsequent judgment. Therefore, clusters with lower brightness values ​​are eliminated. The elimination criteria can be set by technicians in this field, which will not be elaborated here.

[0067] Specifically, there is no limitation on the image processing method for the similarity of texture features between nighttime images. For example, non-texture features in the image can be removed for comparison, and the perceptual hashing algorithm (pHash) can be used to obtain the low-frequency information of the image through DCT (discrete cosine transform), and the hash value of the image is calculated. The similarity of two images is determined by comparing their hash values. The calculation process of the pHash algorithm is relatively simple and can quickly process large amounts of image data, which will not be elaborated here.

[0068] Specifically, the clustering unit is used to identify the texture masking interference value of the target at night based on the texture feature and the texture interference representation value of the night image acquired in the similarity determination area, including:

[0069] The ratio of the texture masking interference value of the texture feature for nighttime recognition target to the texture masking interference threshold is calculated and the corresponding weight coefficient is assigned as the first interference feature. The weight coefficient is assigned by multiplying the weight coefficient with the corresponding value.

[0070] To calculate the ratio of the preset similarity threshold to the similarity of the texture feature and assign a corresponding weight coefficient as the second interference feature;

[0071] The sum of the first interference feature and the second interference feature is determined as a texture interference characterization value.

[0072] In this embodiment, when performing weighted summation, the weight corresponding to the first interference feature is set to 0.55, and the weight corresponding to the second interference feature is set to 0.45;

[0073] The texture masking interference value threshold F0 of the texture feature for nighttime target recognition and the similarity threshold X0 of the texture feature are pre-determined, and a number of image data for rural areas are obtained. The texture masking interference value and similarity data of the texture feature for nighttime target recognition are called, and the mean of the texture masking interference value and the mean of the similarity are solved. The texture masking interference threshold is set to between 1.02 and 1.11 times the mean of the texture masking interference value, and the similarity threshold is set to between 1.25 and 1.35 times the mean of the similarity.

[0074] The present invention analyzes night images of different rural areas through a feature analyzer for classification. In actual situations, night images are relatively fuzzy and fewer texture features can be extracted. For example, in the night images corresponding to some areas with high planting density of farmland, only the texture features of the farmland boundary may be extracted. For some farmlands with sparse planting, the influence of plant boundaries will affect the image clustering effect, resulting in more discrete texture features. When there is smoke at night, due to the influence of image blur, the smoke may be presented as a relatively dense number of texture features in the night image, especially under the influence of surrounding discrete texture features, it is not easy to be identified. Therefore, it is considered to combine the discrete type of texture features to determine the texture masking interference value of the texture feature for the night recognition target, and further determine the texture interference characterization value to characterize the interference of the texture features in the night images in the corresponding rural area to the texture features generated when smoke occurs, so as to adaptively select the image enhancement method for the subsequent classification of the texture interference category of the night images in the rural area, provide data support, and thus improve the accuracy of identifying potential fires at night.

[0075] Specifically, the clustering unit is used to divide the texture interference categories into:

[0076] If the texture interference representation value is greater than or equal to the texture interference representation threshold, the night image is classified as a strong interference category;

[0077] If the texture interference representation value is less than the texture interference representation threshold, the night image is a weak interference category.

[0078] The texture interference characterization threshold G0 is selected in the interval [1.11,1.24].

[0079] Specifically, the enhancement unit is used to process the nighttime images of various rural areas, including:

[0080] If the night image is of the weak interference category, extract several frames of night images for the rural area and compare them with the basic reference image of the rural area to determine the similarity of texture features, and determine whether to enhance the night image in the rural area based on the similarity and the changes of several consecutive frames of night images;

[0081] The image analyzer of the present invention determines the texture interference category of the night images obtained in the rural area, and adaptively selects the image enhancement method. Under the weak texture interference category, considering that the texture features of the smoke are less disturbed, the collected night images are directly compared with the basic reference images. If there are smoke texture features, the similarity of the texture features determined by the comparison will be high, and the image enhancement can quickly identify the smoke features while ensuring reliability. Moreover, since the smoke occurs continuously, in order to avoid misjudgment, the changes of several texture features in continuous frames are considered, and comprehensive verification is performed. Then, only the night images with potential anomalies are enhanced and accurately determined. Then, under the premise of ensuring reliability, the amount of data analysis is reduced when conducting night supervision in large rural areas, and the recognition of fire anomalies is guaranteed.

[0082] If the night image is of a strong interference category, several frames of night images in the rural area are enhanced.

[0083] In the strong texture interference category, the smoke texture features in the night image are interfered by the surrounding discrete texture features and have poor accuracy. Therefore, in order to ensure recognition accuracy, all night images in the corresponding rural area are enhanced to ensure the reliability of monitoring.

[0084] Specifically, the feature parser is also used to determine a basic reference image, including:

[0085] To analyze a number of frames of night images of each rural area and compare the similarity of each night image with the remaining night images;

[0086] The night image corresponding to the maximum similarity is determined as the basic reference image of the rural area.

[0087] Specifically, the enhancement unit is used to determine the change of several frames of nighttime images, including:

[0088] Used to compare the similarity of texture features of adjacent nighttime images;

[0089] It is used to obtain the variance of each similarity as a change representation value.

[0090] Specifically, if the similarity is greater than a preset reference similarity threshold and the change representation value is less than a preset change representation value threshold, it is determined that the night image in the rural area needs to be enhanced.

[0091] Specifically, the change characterization value threshold is preset, and several enhanced night images of rural areas with smoke characteristics are solved to solve the mean of the change characterization value, and 0.85 times the mean of the change characterization value is set as the change characterization value threshold.

[0092] The benchmark similarity threshold is 0.95 times the similarity threshold.

[0093] Specifically, the warning device is used to determine whether to issue a warning prompt, including:

[0094] If abnormal smoke features are found in the enhanced nighttime image, an early warning will be issued.

[0095] The image analyzer further comprises a storage unit for storing the enhanced nighttime image.

[0096] In this embodiment, a pre-trained image recognition model capable of identifying abnormal characteristics of smoke can be implanted in the early warning device. There is no limitation on the specific form of the image recognition model, which can be an image recognition model based on a neural network architecture or other forms, which will not be elaborated here.

[0097] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A smart rural monitoring system for multi-dimensional data analysis, characterized in that: include: A flight controller, which is used to control a number of drones equipped with image acquisition units to fly along a predetermined path to collect nighttime images of a number of rural areas; A feature analyzer connected to each flight controller is used to receive night images of different rural areas and determine the degree of masking of texture features in the night images for night recognition targets, including obtaining a plurality of frames of night images for a single area and annotating texture features, determining a texture masking interference value of the texture features for night recognition targets, and performing image enhancement on the night images to determine the similarity of texture features before and after enhancement; An image analyzer, which is connected to the flight controller and the feature analyzer respectively, comprises a clustering unit and an enhancement unit, The clustering unit is used to determine the texture interference representation value of the night image acquired in the area based on the texture masking interference value and the similarity, so as to classify the texture interference category; The enhancement unit is responsive to the division result of the clustering unit to process the nighttime images of each rural area; An early warning device, which is connected to the image analyzer and is used to identify abnormal smoke features in the enhanced nighttime image and determine whether to issue an early warning prompt; The feature analyzer is used to obtain the annotated texture features of several frames of nighttime images for a single area, and determine the texture masking interference value of the texture features for the nighttime recognition target, including: It is used to grayscale each frame of nighttime image and identify several clusters in the grayscale image; It is used to mark the effective clusters based on the brightness values ​​corresponding to the clusters, and use the effective clusters as texture features; To obtain the similarity variance of several texture features in each frame of nighttime image, and obtain the texture masking interference value of the texture feature for the nighttime recognition target; The clustering unit is used to identify the texture masking interference value of the target at night based on the texture feature and the texture interference representation value of the night image acquired in the similarity determination area, including: The ratio of the texture masking interference value of the texture feature for nighttime recognition target to the texture masking interference threshold is calculated and the corresponding weight coefficient is assigned as the first interference feature; To calculate the ratio of the preset similarity threshold to the similarity of the texture feature and assign a corresponding weight coefficient as the second interference feature; for determining the sum of the first interference feature and the second interference feature as a texture interference characterization value; The clustering unit is used to divide the texture interference categories into: If the texture interference representation value is greater than or equal to the texture interference representation threshold, the night image is classified as a strong interference category; If the texture interference representation value is less than the texture interference representation threshold, the night image is a weak interference category; The enhancement unit is used to process the nighttime images of various rural areas, including: If the night image is of the weak interference category, extract several frames of night images for the rural area and compare them with the basic reference image of the rural area to determine the similarity of texture features, and determine whether to enhance the night image in the rural area based on the similarity and the changes of several consecutive frames of night images; If the night image is of a strong interference category, several frames of night images in the rural area are enhanced.

2. The smart rural monitoring system for multidimensional data analysis according to claim 1 is characterized in that: The feature parser is also used to determine a basic reference image, including: To analyze a number of frames of night images of each rural area and compare the similarity of each night image with the remaining night images; The night image corresponding to the maximum similarity is determined as the basic reference image of the rural area.

3. The smart rural monitoring system for multidimensional data analysis according to claim 1 is characterized in that: The enhancement unit is used to determine the change of several frames of nighttime images, including: Used to compare the similarity of texture features of adjacent nighttime images; It is used to obtain the variance of each similarity as a change representation value.

4. The smart rural monitoring system for multidimensional data analysis according to claim 3 is characterized in that: The enhancement unit is used to determine whether to enhance the nighttime image in the rural area, including: If the similarity is greater than a preset reference similarity threshold and the change representation value is less than a preset change representation value threshold, it is determined that the night image in the rural area needs to be enhanced.

5. The smart rural monitoring system for multidimensional data analysis according to claim 1 is characterized in that: The early warning device is used to determine whether to issue an early warning prompt, including: If abnormal smoke features are found in the enhanced nighttime image, an early warning will be issued.

6. The smart rural monitoring system for multidimensional data analysis according to claim 1 is characterized in that: The image analyzer further comprises a storage unit for storing the enhanced nighttime image.

Images