A night vision image-based patrol supervision method and device

Abstract

The application discloses a kind of based on night vision image's patrol supervision method and device, it is related to the technical field of image processing.The night vision image of target area is obtained, the quality of night vision image is judged to obtain judging result;If night vision image is low-quality image, the initial image is obtained by preprocessing night vision image;Global feature extraction is carried out to initial image to obtain global feature, local feature extraction is carried out to initial image to obtain local feature;Fusion feature is obtained by fusing global feature and local feature, and patrol supervision is carried out according to fusion feature.The automatic process carries out quality judgment, preprocessing, global and local feature extraction and fusion to night vision image, effectively distinguishes image quality and optimizes low-quality image, comprehensively captures image information, improves decision accuracy, realizes the efficient, accurate patrol supervision of industrial park.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a patrol and monitoring method and device based on night vision images. Background Technology

[0002] With the continuous expansion of industrial parks and the increasing demands for safety management, traditional patrol and supervision methods are no longer sufficient to meet the needs for efficiency and accuracy. In recent years, patrol and supervision technology based on night vision images has emerged, which, by introducing advanced image processing and analysis methods, enables real-time monitoring and intelligent analysis of nighttime scenes.

[0003] Publication No. CN118675229A discloses a smart community visualization real-time monitoring system and method based on big data. The system senses human features in the monitored target area; imports a blurring command and several real-time monitoring images of the target area, generating and displaying a partially blurred image; maps and identifies several real-time monitoring images of the target area, generating a removal command; acquires the partially blurred image and the removal command, and sends early warning feedback information; based on the early warning feedback information, displays and identifies the real-time monitoring video stream of the target area, generating a suspicious judgment command; based on the suspicious judgment command, sends an intervention patrol command. This system enables real-time security monitoring of the monitored area. Before a preliminary judgment is made on suspicious individuals, the system partially blurs their images, ensuring their privacy while allowing observers to monitor and identify their behavior. Upon identifying suspicious behavior, an intervention patrol command is promptly generated.

[0004] Traditional patrol and supervision still has some problems. It is impossible to quantitatively judge the quality of night vision images, resulting in low-quality images entering the subsequent processing process, which affects the accuracy and efficiency of supervision. There is a lack of optimization processing methods for low-quality images, which makes it difficult to lose or blur image details, making it difficult to conduct accurate analysis and capture image information accurately. This leads to insufficient decision-making basis, poor supervision effect, and affects the efficiency of patrol and supervision in industrial parks. Summary of the Invention

[0005] The purpose of this invention is to address the problems mentioned in the background art, such as the inability to quantify and judge the quality of night vision images, which leads to low-quality images entering the subsequent processing flow, affecting the accuracy and efficiency of supervision. The lack of optimization processing methods for low-quality images results in the loss or blurring of image details, making it difficult to conduct accurate analysis and accurately capture image information, leading to insufficient decision-making basis, poor supervision effect, and affecting the efficiency of industrial park inspection and supervision. Therefore, this invention proposes an inspection and supervision method and device based on night vision images.

[0006] A first aspect of this invention provides a patrol and monitoring method based on night vision images, the method comprising: Acquire night view images of the target area, and perform quality assessment on the night view images to obtain assessment results; the assessment results include: low-quality images and high-quality images; If the night view image is a low-quality image, then the night view image is preprocessed to obtain an initial image; Global features are obtained by performing global feature extraction on the initial image, and local features are obtained by performing local feature extraction on the initial image. The global features and the local features are fused to obtain fused features, and patrol and supervision are carried out based on the fused features.

[0007] Optionally, the quality of the night view image is assessed to obtain a judgment result, including: The night vision image is mapped by an encoder to obtain a feature vector set; Obtain a preset codebook, and replace each feature vector in the feature vector set according to the preset codebook to obtain the target feature vector set; The target feature vector set is reconstructed by a decoder to obtain a reconstructed image, and the similarity between the reconstructed image and the night view image is calculated to obtain a similarity value; If the similarity value is less than the similarity threshold, the night view image is determined to be a low-quality image.

[0008] Optionally, global feature extraction is performed on the initial image to obtain global features, including: The initial image is converted to obtain an HSV color space map and a Lab color space map. Channels are extracted from the HSV color space map to obtain the V channel, and channels are extracted from the Lab color space map to obtain the L channel and ab channel. Calculate the histograms of the V channel, the L channel, and the ab channel to obtain the V histogram, L histogram, and ab histogram; The global features are obtained by normalizing the V histogram, L histogram, and ab histogram and then splicing them together.

[0009] Optionally, local features are extracted from the initial image to obtain local features, including: The initial image is sequentially input into six convolutional layers to obtain the first feature map, and the first feature map is sequentially input into four residual modules to obtain the second feature map; The second feature map is sequentially input into six transposed convolutional layers to obtain the third feature map, and the dimensions of the third feature map are adjusted to obtain local features.

[0010] Optionally, inspection and supervision can be carried out based on the fusion features, including: The cosine similarity value between the fused feature and the features stored in the preset database is calculated, and the data feature corresponding to the maximum cosine similarity value is obtained. The inspection and supervision plan is determined based on the data feature. The preset database contains multiple data features, and each data feature corresponds to an inspection and supervision plan.

[0011] A second aspect of this invention provides a patrol and monitoring device based on night vision images, the device comprising: The quality assessment module is used to acquire night view images of the target area, and to assess the quality of the night view images to obtain assessment results; the assessment results include: low-quality images and high-quality images; A preprocessing module is used to preprocess the night view image to obtain an initial image if the night view image is a low-quality image; The feature extraction module is used to perform global feature extraction on the initial image to obtain global features, and to perform local feature extraction on the initial image to obtain local features; The feature fusion module is used to fuse the global features and the local features to obtain fused features, and to perform inspection and supervision based on the fused features.

[0012] Optionally, the quality judgment module includes: The feature quantization module is used to map the night view image through the encoder to obtain a feature vector set; The feature replacement module is used to obtain a preset codebook and replace each feature vector in the feature vector set according to the preset codebook to obtain a target feature vector set; The image reconstruction module is used to reconstruct the target feature vector set through the decoder to obtain a reconstructed image, and to calculate the similarity between the reconstructed image and the night view image to obtain a similarity value; The quality determination module is used to determine that the night view image is a low-quality image if the similarity value is less than the similarity threshold.

[0013] Optionally, the feature extraction module includes: The channel extraction module is used to convert the initial image to obtain an HSV color space map and a Lab color space map, extract the V channel from the HSV color space map, and extract the L channel and ab channel from the Lab color space map. The channel calculation module is used to calculate the histograms of the V channel, the L channel, and the ab channel to obtain the V histogram, the L histogram, and the ab histogram; The global stitching module is used to normalize the V histogram, the L histogram, and the ab histogram, and then stitch and fuse them to obtain global features.

[0014] Optionally, the feature extraction module further includes: The feature extraction module is used to sequentially input the initial image into six convolutional layers to obtain a first feature map, and sequentially input the first feature map into four residual modules to obtain a second feature map; The local extraction module is used to sequentially input the second feature map into six transposed convolutional layers to obtain the third feature map, and to adjust the dimensions of the third feature map to obtain local features.

[0015] Optionally, the feature fusion module is further configured to calculate the cosine similarity value between the fused feature and the features stored in the preset database, obtain the data feature corresponding to the maximum cosine similarity value, and determine the inspection and supervision plan based on the data feature; the preset database contains multiple data features, and each data feature corresponds to an inspection and supervision plan.

[0016] The beneficial effects of this invention are: This invention proposes a patrol and supervision method based on night vision images. The method involves acquiring night vision images of a target area, assessing their quality, and preprocessing them to obtain an initial image if the image is of low quality. Global and local features are then extracted from the initial image to obtain global features and local features, respectively. These global and local features are then fused to obtain a fused feature, which is used for patrol and supervision. This automated process, involving quality assessment, preprocessing, global and local feature extraction, and fusion, effectively distinguishes image quality, optimizes low-quality images, comprehensively captures image information, improves decision-making accuracy, and enables efficient and precise patrol and supervision of industrial parks. Attached Figure Description

[0017] Figure 1 A flowchart of a patrol and monitoring method based on night vision images is provided for an embodiment of the present invention; Figure 2 A model structure diagram of a patrol and monitoring method based on night vision images is provided for an embodiment of the present invention; Figure 3 This invention provides a schematic diagram of the structure of a patrol and monitoring device based on night vision images. Detailed Implementation

[0018] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0019] This invention provides a patrol and monitoring method based on night vision images. See also... Figure 1 , Figure 1A flowchart illustrating a patrol and monitoring method based on night vision images, provided as an embodiment of the present invention. The method includes the following steps: S101, acquire night view images of the target area, and perform quality assessment on the night view images to obtain the assessment result; S102, If the night view image is a low-quality image, then preprocess the night view image to obtain the initial image; S103, global features are extracted from the initial image to obtain global features, and local features are extracted from the initial image to obtain local features; S104, the global features and local features are fused to obtain the fused features, and the patrol and supervision are carried out based on the fused features.

[0020] The judgment results include: low-quality images and high-quality images; The present invention provides a patrol and supervision method based on night vision images. Through an automated process, night vision images are subjected to quality judgment, preprocessing, global and local feature extraction and fusion. This effectively distinguishes image quality and optimizes low-quality images, comprehensively captures image information, improves decision-making accuracy, and achieves efficient and precise patrol and supervision of industrial parks.

[0021] In one implementation, the night view image of the target area is judged in terms of quality, which can distinguish between low-quality images and high-quality images. Low-quality images may lose details or become blurry due to insufficient lighting, noise interference, or equipment failure. This ensures that only images with qualified quality enter the subsequent processing flow, thereby improving the reliability and stability of the system.

[0022] In one implementation, for low-quality images, preprocessing operations such as noise reduction and contrast enhancement can improve the image quality to a level suitable for further analysis. This not only optimizes the visual effect of the image but also provides a more accurate data foundation for subsequent feature extraction and fusion, ensuring that the entire system's image processing is based on high-quality input, thereby improving the effectiveness and accuracy of the patrol and supervision system.

[0023] In one implementation, global and local features are extracted from the initial image to capture key information from different angles. Global features provide overall information about the image, such as the layout and general structure of the scene, which helps to understand the macro scene of the image. Local features, on the other hand, focus on the details in the image, such as the edges and textures of objects, which can more precisely identify specific objects and activities in the image.

[0024] In one implementation, a feature extraction method that combines global and local features enables the system to simultaneously grasp both macroscopic and microscopic information of an image, enhancing the comprehensiveness and richness of feature representation. For example, in the inspection and supervision of industrial parks, global features can help the system quickly locate the approximate area where an event occurs, while local features can identify specific intrusion behaviors or abnormal objects, thereby providing a more comprehensive and accurate basis for subsequent decision-making and improving the system's adaptability to complex scenarios and recognition accuracy.

[0025] In one implementation, global and local features are fused to obtain fused features. This fusion process combines the advantages of both global and local features to form a more powerful feature representation. The fused features not only contain the overall information of the image but also retain rich details, enabling the system to make decisions based on more comprehensive and accurate features during patrol and monitoring. For example, when judging whether there is abnormal behavior such as someone climbing over a wall, fused features can more accurately identify the intruder's outline, movements, and other details. Simultaneously, by combining this with the overall layout of the scene, the rationality of the behavior can be judged, thereby improving the accuracy and reliability of the decision. Patrol and monitoring methods based on fused features can effectively reduce misjudgments and omissions, optimize the effect of patrol and monitoring, ensure the safety and normal operation of industrial parks, and improve the overall performance and practicality of the system.

[0026] In one implementation, image acquisition, quality assessment, preprocessing, feature extraction, and fusion are all automated operations, which improves the system's operating efficiency and reduces errors and delays that may be caused by manual operation. In the inspection and supervision of industrial parks, the automated system can monitor and analyze the monitored area in real time and continuously, promptly detect and handle abnormal situations, and greatly improve the efficiency and timeliness of inspection and supervision.

[0027] In one embodiment, the quality assessment of night view images to obtain a assessment result includes: The night vision image is mapped by an encoder to obtain a set of feature vectors; Obtain a preset codebook, and replace each feature vector in the feature vector set according to the preset codebook to obtain the target feature vector set; The target feature vector set is reconstructed by the decoder to obtain the reconstructed image, and the similarity between the reconstructed image and the night view image is calculated to obtain the similarity value; If the similarity value is less than the similarity threshold, the night view image is determined to be a low-quality image.

[0028] In one implementation, a target feature vector set is obtained by replacing each feature vector in the feature vector set according to a preset codebook: the preset codebook contains multiple preset vectors (the preset vectors in the preset codebook serve as standards, and the regions corresponding to the preset vectors are the same regions as the shooting areas of the night view images, used to quantify image quality), the feature vector set contains multiple feature vectors, the similarity between the target feature vector and each preset vector in the preset codebook is calculated, and the preset vector with the highest similarity is replaced with the target feature vector. The target feature vector is any one of the feature vectors in the feature vector set, so all are replaced to obtain the target feature vector set; the similarity calculation method includes: cosine similarity.

[0029] In one implementation, a feature vector set is obtained by mapping the night view image through an encoder, and then the feature vectors are replaced using a preset codebook. The image is then reconstructed by a decoder, and a similarity value is calculated to determine the image quality. This approach can extract the intrinsic features of the night view image, elevating the judgment of image quality from an intuitive visual level to a quantitative level based on feature vectors. Compared with traditional methods that rely solely on manual observation or simple pixel-level comparison, this approach can more accurately identify quality defects in the image, such as blurring and noise, thus providing a more accurate basis for the quality assessment of night view images.

[0030] In one implementation, after judging the quality of night vision images, if the image is determined to be a low-quality image, targeted image processing operations can be performed. For low-quality night vision images, denoising and enhancement algorithms are used for preprocessing, while high-quality images can skip these steps and be directly applied to subsequent applications. This reduces the waste of resources caused by uniformly processing all images and improves the overall efficiency of night vision image processing.

[0031] In one embodiment, global feature extraction is performed on the initial image to obtain global features, including: The initial image is converted to obtain an HSV color space map and a Lab color space map. Channels are extracted from the HSV color space map to obtain the V channel, and channels are extracted from the Lab color space map to obtain the L channel and ab channel. Calculate the histograms of the V channel, L channel, and ab channel to obtain the V histogram, L histogram, and ab histogram; The global features are obtained by normalizing the V histogram, L histogram, and ab histogram and then splicing them together.

[0032] In one implementation, the initial image is converted to obtain an HSV color space map and a Lab color space map. In the RGB space, brightness and color are highly coupled, while HSV / Lab can directly separate key channels. In one implementation, the V channel is obtained by extracting channels from the HSV color space map, and the L channel and ab channel are obtained by extracting channels from the Lab color space map. The target channel is extracted from the converted color space. The V channel (containing only luminance information) is extracted from the HSV space, and the L channel (containing luminance information) and ab channel (containing only color information) are extracted from the Lab space. Irrelevant channels (such as the H / S channels of HSV and meaningless redundant information of Lab) are discarded.

[0033] In one implementation, the histograms of the V channel, L channel, and ab channel are calculated to obtain the V histogram, L histogram, and ab histogram: The histograms of the V channel, L channel, and ab channel are calculated respectively to obtain the V channel histogram, L channel histogram, and ab channel histogram. The global statistical characteristics of each channel, such as brightness distribution and color distribution patterns, are captured through the histograms.

[0034] In one implementation, the V histogram, L histogram, and ab histogram are normalized and then concatenated to obtain the global feature: the V histogram, L histogram, and ab histogram are normalized for dimension unification to facilitate subsequent concatenation; for example: the V histogram (dimension R) is normalized for dimension unification. p Convert to a histogram with dimension R. p The feature map is dimension-compatible. The transformed V histogram and L histogram are concatenated to reshape a q×r dimension reshaped feature map. This reshaped feature map is then compared with the ab histogram (dimension R). q×r The fusion is performed to obtain a fused feature map (dimension R). q×r×(p+1) ).

[0035] In one implementation, the V channel of the HSV color space and the L channel of the Lab color space are independent channels dedicated to carrying brightness information. They can directly extract the brightness distribution features of the image (without color interference), and describe brightness from the perspective of different color models. They can complement each other to cover the brightness details of low-light images, providing an accurate data foundation for subsequent brightness enhancement and avoiding color damage during brightness adjustment. Meanwhile, the ab channels of the Lab color space are pure color information carriers (unrelated to brightness), which can completely preserve the original color distribution rules of low-light images and avoid the problem of brightness enhancement accompanied by color distortion caused by brightness and color coupling in the traditional RGB space.

[0036] In one embodiment, local feature extraction is performed on the initial image to obtain local features, including: The initial image is sequentially input into six convolutional layers to obtain the first feature map, and the first feature map is sequentially input into four residual modules to obtain the second feature map; The second feature map is sequentially input into six transposed convolutional layers to obtain the third feature map. The dimensions of the third feature map are then adjusted to obtain local features.

[0037] In one implementation, see [link to implementation details]. Figure 2 , Figure 2 This invention provides a model structure diagram for a patrol and monitoring method based on night vision images. An initial image is sequentially input into six convolutional layers to obtain a first feature map, which can progressively extract local low-level features from the image, such as edges and textures. The layer-by-layer processing of the convolutional layers refines the features from simple to complex, for example, from the most basic edge contours to more detailed local features such as edge grayscale gradients, providing a foundation for subsequent feature optimization and image processing. The first feature map is further optimized through four residual modules to obtain a second feature map. The residual modules effectively alleviate the gradient vanishing problem in deep networks, ensuring that weak local features in low-light images (such as edges and textures of objects in dark areas) are preserved in deep networks, avoiding the loss of details during feature transfer, thereby enhancing the representation of local image details.

[0038] In one implementation, the second feature map is sequentially input into six transposed convolutional layers to obtain the third feature map. The transposed convolutional layers gradually restore the spatial dimension of the feature map, compensating for the dimensionality compression caused by the pooling operation in the previous encoder. During the process of restoring spatial resolution, the transposed convolutional layers can not only enlarge the spatial dimension of the feature map to match that of the original image, but also preserve the correlation of local features during the layer-by-layer restoration process, avoiding distortion of local details during dimensionality enlargement. For example, it can restore the grayscale transition of edges, the details of textures, etc. The final third feature map has both a spatial dimension that matches that of the original image and completely preserves local detail features, providing a high-quality local feature foundation for subsequent image processing and feature fusion, thereby improving the overall effect and quality of image processing.

[0039] In one implementation, the third feature map is dimensionally adjusted to obtain local features. In subsequent processing, the global features and local features need to be fused, such as through feature concatenation or element-wise addition. Fusion can only be performed when the number of channels and spatial dimensions of the two are completely consistent. The dimensionality of the third feature map is adjusted by 1×1 convolution operations to ensure dimensionality matching between the local and global feature maps during fusion.

[0040] In one implementation, the initial image is fed into the L-Net encoder (containing 6 convolutional layers, each followed by RelU activation). Through convolution operations, the spatial dimension is gradually compressed and local low-level features (such as edges and textures) are extracted, finally outputting the first feature map of the encoder. Four residual modules are inserted between the encoder and the decoder to perform feature transfer and optimization on the first feature map, alleviating the gradient vanishing problem, accelerating training convergence, and preserving the local detail features extracted by the encoder to avoid detail loss caused by deep networks. The output of the residual modules (the third feature map) is fed into the decoder, and the spatial dimension is gradually restored through transposed convolution to generate local features that match the resolution of the input image.

[0041] In one embodiment, patrol and supervision based on fusion characteristics includes: Calculate the cosine similarity value between the fused feature and the features stored in the preset database, obtain the data feature corresponding to the maximum cosine similarity value, and determine the inspection and supervision plan based on the data feature; the preset database contains multiple data features, and each data feature corresponds to an inspection and supervision plan.

[0042] In one implementation, multiple data features in the preset database are features (data features) obtained after processing historical night view images. That is, the data features and fused feature structures are the same, and each type of night view image corresponds to a patrol and supervision scheme.

[0043] In one implementation, the patrol and monitoring scheme includes, for example: sending a real-time alarm to the park's security center to notify security personnel to conduct on-site inspections; automatically activating surveillance cameras near the area where intruders are trespassing to record real-time video and preserve evidence; using the park's monitoring system or other technical means to locate and track intruders; and recording in detail information such as the time, location, and characteristics of the intruders involved in the intrusion.

[0044] Based on the same inventive concept, this invention also provides a patrol and monitoring device based on night vision images. See also Figure 3 , Figure 3 A schematic diagram of a patrol and monitoring device based on night vision images provided in an embodiment of the present invention includes: The quality assessment module is used to acquire night view images of the target area, assess the quality of the night view images, and obtain assessment results; the assessment results include: low-quality images and high-quality images; The preprocessing module is used to preprocess the night view image to obtain an initial image if the night view image is a low-quality image; The feature extraction module is used to extract global features from the initial image and extract local features from the initial image to obtain local features. The feature fusion module is used to fuse global and local features to obtain fused features, and to conduct inspection and supervision based on the fused features.

[0045] The night vision image-based patrol and monitoring device provided in this invention uses an automated process to perform quality judgment, preprocessing, global and local feature extraction and fusion on the night vision images. This effectively distinguishes image quality and optimizes low-quality images, comprehensively captures image information, improves decision-making accuracy, and achieves efficient and precise patrol and monitoring of industrial parks.

[0046] In one embodiment, the quality judgment module includes: The feature quantization module is used to map the night vision image through the encoder to obtain a feature vector set; The feature replacement module is used to obtain a preset codebook and replace each feature vector in the feature vector set according to the preset codebook to obtain the target feature vector set. The image reconstruction module is used to reconstruct the target feature vector set through the decoder to obtain the reconstructed image, and to calculate the similarity value between the reconstructed image and the night view image. The quality assessment module is used to determine that a night view image is a low-quality image if the similarity value is less than the similarity threshold.

[0047] In one embodiment, the feature extraction module includes: The channel extraction module is used to convert the initial image to obtain an HSV color space map and a Lab color space map, extract the V channel from the HSV color space map, and extract the L channel and ab channel from the Lab color space map. The channel calculation module is used to calculate the histograms of the V channel, L channel, and ab channel to obtain the V histogram, L histogram, and ab histogram. The global stitching module is used to normalize the V histogram, L histogram, and ab histogram, and then stitch and fuse them to obtain global features.

[0048] In one embodiment, the feature extraction module further includes: The feature extraction module is used to sequentially input the initial image into six convolutional layers to obtain the first feature map, and then sequentially input the first feature map into four residual modules to obtain the second feature map; The local extraction module is used to sequentially input the second feature map into six transposed convolutional layers to obtain the third feature map, and then adjust the dimensions of the third feature map to obtain local features.

[0049] In one embodiment, the feature fusion module is further configured to calculate the cosine similarity value between the fused feature and the features stored in the preset database, obtain the data feature corresponding to the maximum cosine similarity value, and determine the inspection and supervision plan based on the data feature; the preset database contains multiple data features, and each data feature corresponds to an inspection and supervision plan.

[0050] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A patrol and monitoring method based on night vision images, characterized in that, The method includes: Acquire night view images of the target area, and perform quality assessment on the night view images to obtain assessment results; the assessment results include: low-quality images and high-quality images; If the night view image is a low-quality image, then the night view image is preprocessed to obtain an initial image; Global features are obtained by performing global feature extraction on the initial image, and local features are obtained by performing local feature extraction on the initial image. The global features and the local features are fused to obtain fused features, and patrol and supervision are carried out based on the fused features.

2. The patrol and monitoring method based on night vision images according to claim 1, characterized in that, The quality of the night view image is assessed to obtain the assessment result, including: The night vision image is mapped by an encoder to obtain a feature vector set; Obtain a preset codebook, and replace each feature vector in the feature vector set according to the preset codebook to obtain the target feature vector set; The target feature vector set is reconstructed by a decoder to obtain a reconstructed image, and the similarity between the reconstructed image and the night view image is calculated to obtain a similarity value; If the similarity value is less than the similarity threshold, the night view image is determined to be a low-quality image.

3. The patrol and monitoring method based on night vision images according to claim 1, characterized in that, Global features are extracted from the initial image to obtain global features, including: The initial image is converted to obtain an HSV color space map and a Lab color space map. Channels are extracted from the HSV color space map to obtain the V channel, and channels are extracted from the Lab color space map to obtain the L channel and ab channel. Calculate the histograms of the V channel, the L channel, and the ab channel to obtain the V histogram, L histogram, and ab histogram; The global features are obtained by normalizing the V histogram, L histogram, and ab histogram and then splicing them together.

4. The patrol and monitoring method based on night vision images according to claim 1, characterized in that, Local features are extracted from the initial image to obtain local features, including: The initial image is sequentially input into six convolutional layers to obtain the first feature map, and the first feature map is sequentially input into four residual modules to obtain the second feature map; The second feature map is sequentially input into six transposed convolutional layers to obtain the third feature map, and the dimensions of the third feature map are adjusted to obtain local features.

5. The patrol and monitoring method based on night vision images according to claim 1, characterized in that, Inspection and supervision based on the aforementioned fusion features include: The cosine similarity value between the fused feature and the features stored in the preset database is calculated, and the data feature corresponding to the maximum cosine similarity value is obtained. The inspection and supervision plan is determined based on the data feature. The preset database contains multiple data features, and each data feature corresponds to an inspection and supervision plan.

6. A patrol and monitoring device based on night vision images, characterized in that, The device includes: The quality assessment module is used to acquire night view images of the target area, and to assess the quality of the night view images to obtain assessment results; the assessment results include: low-quality images and high-quality images; A preprocessing module is used to preprocess the night view image to obtain an initial image if the night view image is a low-quality image; The feature extraction module is used to perform global feature extraction on the initial image to obtain global features, and to perform local feature extraction on the initial image to obtain local features; The feature fusion module is used to fuse the global features and the local features to obtain fused features, and to perform inspection and supervision based on the fused features.

7. A patrol and monitoring device based on night vision images according to claim 6, characterized in that, The quality judgment module includes: The feature quantization module is used to map the night view image through the encoder to obtain a feature vector set; The feature replacement module is used to obtain a preset codebook and replace each feature vector in the feature vector set according to the preset codebook to obtain a target feature vector set; The image reconstruction module is used to reconstruct the target feature vector set through the decoder to obtain a reconstructed image, and to calculate the similarity between the reconstructed image and the night view image to obtain a similarity value; The quality determination module is used to determine that the night view image is a low-quality image if the similarity value is less than the similarity threshold.

8. A patrol and monitoring device based on night vision images according to claim 6, characterized in that, The feature extraction module includes: The channel extraction module is used to convert the initial image to obtain an HSV color space map and a Lab color space map, extract the V channel from the HSV color space map, and extract the L channel and ab channel from the Lab color space map. The channel calculation module is used to calculate the histograms of the V channel, the L channel, and the ab channel to obtain the V histogram, the L histogram, and the ab histogram; The global stitching module is used to normalize the V histogram, the L histogram, and the ab histogram, and then stitch and fuse them to obtain global features.

9. A patrol and monitoring device based on night vision images according to claim 6, characterized in that, The feature extraction module further includes: The feature extraction module is used to sequentially input the initial image into six convolutional layers to obtain a first feature map, and sequentially input the first feature map into four residual modules to obtain a second feature map; The local extraction module is used to sequentially input the second feature map into six transposed convolutional layers to obtain the third feature map, and to adjust the dimensions of the third feature map to obtain local features.

10. A patrol and monitoring device based on night vision images according to claim 6, characterized in that, The feature fusion module is also used to calculate the cosine similarity value between the fused feature and the features stored in the preset database, obtain the data feature corresponding to the maximum cosine similarity value, and determine the inspection and supervision plan based on the data feature; the preset database contains multiple data features, and each data feature corresponds to an inspection and supervision plan.

Images