Automatic sewage draining exit identification method and device under unmanned aerial vehicle low-altitude oblique photography

Abstract

The invention relates to an automatic sewage draining exit identification method and device under unmanned aerial vehicle low-altitude oblique photography, and the method comprises the steps: generating enhanced image information based on at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic puzzle enhancement and copying and pasting of a data enhancement strategy; introducing a multi-head self-attention mechanism into a backbone network of a pre-constructed sewage draining exit identification model, and introducing a channel attention mechanism into a neck network to determine a bounding box area of the sewage draining exit; and according to the bounding box area of the sewage draining exit, cutting an image which is obliquely photographed by the low-altitude unmanned aerial vehicle, and inputting the cut image into a pre-constructed sewage draining state discrimination model to identify the actual discharging state of the sewage draining exit. Therefore, the problems that small and medium-sized hidden drain outlets in narrow drainage basins such as ditches and the like are difficult to effectively identify by the existing method, the detection precision and the calculation efficiency of the existing model are difficult to balance, and the discharge state is not effectively distinguished are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of pollution outlet identification, and particularly relates to a pollution outlet automatic identification method and device under low-altitude oblique photography of a UAV. BACKGROUND

[0002] The pollution outlet is the last barrier for pollutants in wastewater before entering the natural water body. At present, the pollution outlet investigation mainly relies on manual field investigation. However, the manual field investigation has high cost, long time consumption, high missing rate and limited coverage, and is not suitable for large-scale basin pollution outlet investigation. As a supplementary means, the satellite remote sensing investigation method can monitor the sewage discharge of the pollution outlet through water quality inversion and other technical means, but this method is limited by the spatial resolution, repetition period and identification method accuracy, and it is difficult to effectively monitor the periodic discharge and small and concealed pollution outlet objects. In recent years, the UAV remote sensing investigation method has gradually become an important means in the field of environmental monitoring due to its high mobility, high controllability, low cost and small influence of natural conditions. The UAV can go to the ditch and other places that are difficult for humans to reach, and can fly close to accurately identify the pollution outlet hidden by vegetation and the like.

[0003] In the related art, the UAV aerial photography identification of the pollution outlet relies on the visual interpretation of experts from left to right and from top to bottom, which is time-consuming and inefficient, and each image usually takes about 10 minutes. In order to improve the interpretation efficiency, scholars have proposed an automatic detection method for pollution outlets based on deep learning technology. Huang et al. proposed an improved geographic deep learning method based on the Faster R-CNN architecture, integrated with a geographic classification module with digital surface model enhancement and spatial activation function, which expanded the spectral information of the remote sensing picture and improved the identification accuracy of the pollution outlet. Yu et al. proposed a lightweight Transformer network, integrated with a separate down-sampling self-attention module and a convolutional feedforward network with a shortcut, which established a long-term dependency relationship between the pollution outlet and its surrounding environment, so that the model could understand the target and background information of the pollution outlet.

[0004] However, in the related art, only the flight efficiency is considered, and most studies focus on the UAV aerial images taken at high altitudes. Such methods can only effectively identify relatively large pollution outlets in rivers and lakes. For small and concealed pollution outlets in narrow waters such as ditches, low-altitude close-range flight of the UAV is usually required for investigation, but the research on identification models in such scenarios is relatively scarce. The balance between efficiency and accuracy of the existing identification models is still insufficient. Most models pursue higher detection accuracy by using complex network structures and a large number of parameters, which increases the calculation cost and slows down the processing speed, making it difficult to meet the high efficiency requirement of real-time inspection of the UAV. The current mainstream pollution outlet investigation method only focuses on the identification of the pollution outlet, and lacks the discrimination of the discharge state of the pollution outlet, which needs to be improved. SUMMARY

[0005] The application provides a sewage outlet automatic identification method and device under low-altitude oblique photography of a UAV to solve the problem that existing methods mainly rely on high-altitude aerial images and cannot effectively identify small and hidden sewage outlets in narrow valleys, and that existing models cannot balance between detection accuracy and calculation efficiency and cannot meet the real-time inspection requirements of UAVs.

[0006] The first aspect of the application provides a sewage outlet automatic identification method under low-altitude oblique photography of a UAV, comprising the following steps: performing at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic picture enhancement and copy-pasting based on a data enhancement strategy on an image obtained by low-altitude oblique photography of a UAV to generate enhanced image information; introducing a multi-head self-attention mechanism into a backbone network of a pre-constructed sewage outlet identification model and introducing a channel attention mechanism into a neck network based on the enhanced image information to determine a boundary box region of a sewage outlet; cropping the image obtained by low-altitude oblique photography of the UAV according to the boundary box region of the sewage outlet to obtain a cropped image, and inputting the cropped image into a pre-constructed sewage state discrimination model to identify the actual discharge state of the sewage outlet.

[0007] Optionally, in one embodiment of the application, the at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic picture enhancement and copy-pasting based on a data enhancement strategy on an image obtained by low-altitude oblique photography of a UAV to generate enhanced image information comprises: in a model training phase, randomly rotating the image obtained by low-altitude oblique photography of the UAV within a preset angle range to simulate changes in the flight angle of the UAV, and transforming the image obtained by low-altitude oblique photography of the UAV by cutting to simulate distortion of a lens; applying Gaussian blurring and motion blurring processing to the image to simulate imaging jitter when the UAV vibrates, and using a mosaic picture enhancement method to splice images of different scenes to generate background samples that meet a preset high complexity condition; using a copy-pasting strategy to cut and paste a sewage outlet region into a target image to generate multi-target high-density samples; and performing data enhancement on the image obtained by low-altitude oblique photography of the UAV based on the changes in the flight angle, the distortion of the lens, the imaging jitter, the background samples and the multi-target high-density samples to generate the enhanced image information.

[0008] Optionally, in an embodiment of the present application, the multi-head self-attention mechanism is introduced into the backbone network of the pre-constructed pollution outlet identification model, and the channel attention mechanism is introduced into the neck network to determine the boundary box region of the pollution outlet, comprising: constructing the pollution outlet identification model using the enhanced image information, and introducing the multi-head self-attention mechanism into the backbone network of the pollution outlet identification model to capture at least one regional feature of the pollution outlet that is blocked or confused in the low-altitude unmanned aerial vehicle oblique photography image; introducing the channel attention mechanism into the neck network to adaptively weight the response intensity of each feature channel, generating at least one semantic feature corresponding to the pollution outlet; using the pollution outlet identification model to output the position box of the pollution outlet according to the at least one regional feature and the at least one semantic feature, and determining the boundary box region of the pollution outlet according to the position box.

[0009] Optionally, in an embodiment of the present application, the image of the low-altitude unmanned aerial vehicle oblique photography is cropped according to the boundary box region of the pollution outlet to obtain a cropped image, and the cropped image is input into a pre-constructed pollution state discrimination model to identify the actual emission state of the pollution outlet, comprising: in the case of cropping the image of the low-altitude unmanned aerial vehicle oblique photography, the boundary box region of the pollution outlet is expanded outward according to a preset width and a preset height to generate the cropped image containing background environment information; inputting the cropped image into the pre-constructed pollution state discrimination model to identify the actual emission state of the pollution outlet, wherein the pollution state discrimination model is obtained by adjusting the pollution category and outputting the pollution state or the non-pollution state.

[0010] Optionally, in an embodiment of the present application, after identifying the actual emission state of the pollution outlet, further comprising: obtaining a real image of the pollution outlet and identifying a state coordinate of the pollution outlet; uploading the real image of the pollution outlet, the state coordinate and the actual emission state to a target environmental supervision platform to monitor the actual emission state using the target environmental supervision platform.

[0011] The second aspect embodiment of the application provides an automatic discharge outlet identification device under low-altitude oblique photography of a UAV, comprising: a data enhancement module, configured to perform at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic puzzle enhancement and copy-paste based on a data enhancement strategy, to perform data enhancement on images of low-altitude UAV oblique photography, to generate enhanced image information; a determination module, configured to introduce a multi-head self-attention mechanism in a backbone network of a pre-constructed discharge outlet identification model based on the enhanced image information, and introduce a channel attention mechanism in a neck network, to determine a boundary box region of a discharge outlet; and an identification module, configured to crop the images of low-altitude UAV oblique photography according to the boundary box region of the discharge outlet, to obtain cropped images, and input the cropped images into a pre-constructed discharge state discrimination model, to identify an actual discharge state of the discharge outlet.

[0012] Optionally, in an embodiment of the application, the data enhancement module comprises: a first simulation unit, configured to, in a model training stage, randomly rotate the images of low-altitude UAV oblique photography within a preset angle range to simulate flight angle changes of a UAV, and perform cutting transformation on the images of low-altitude UAV oblique photography to simulate distortion of a lens; a second simulation unit, configured to apply Gaussian blurring and motion blurring processing to the images to simulate imaging jitter when the UAV vibrates, and use a mosaic puzzle enhancement method to splice images of different scenes to generate background samples meeting a preset high complexity condition; a generation unit, configured to use a copy-paste strategy to cut and paste a discharge outlet region into a target image to generate multi-target high-density samples; and a data enhancement unit, configured to perform data enhancement on the images of low-altitude UAV oblique photography based on the flight angle changes, the distortion of the lens, the imaging jitter, the background samples and the multi-target high-density samples, to generate the enhanced image information.

[0013] Optionally, in an embodiment of the application, the determination module comprises: a capture unit, configured to use the enhanced image information to construct the discharge outlet identification model, and introduce the multi-head self-attention mechanism in a backbone network of the discharge outlet identification model to capture at least one regional feature of the discharge outlet being blocked or confused in the low-altitude UAV oblique photography images; an introduction unit, configured to introduce the channel attention mechanism in the neck network to adaptively weight a response intensity of each feature channel, to generate at least one semantic feature corresponding to the discharge outlet; and a determination unit, configured to use the discharge outlet identification model to output a position box of the discharge outlet according to the at least one regional feature and the at least one semantic feature, and determine a boundary box region of the discharge outlet according to the position box.

[0014] Optionally, in an embodiment of the present application, the identification module comprises: an expansion unit, configured to, in the case of cropping the image of the low-altitude unmanned aerial vehicle oblique photography, expand the boundary box region of the sewage outlet according to a preset width and a preset height to generate the cropped image containing background environment information; and an identification unit, configured to input the cropped image into a pre-constructed sewage state discrimination model to identify the actual discharge state of the sewage outlet, wherein the sewage state discrimination model is obtained by adjusting a sewage category and outputting a sewage state or a non-sewage state.

[0015] Optionally, in an embodiment of the present application, the method further comprises: an acquisition module, configured to, after identifying the actual discharge state of the sewage outlet, acquire a real image of the sewage outlet and identify a state coordinate of the sewage outlet; and a monitoring module, configured to upload the real image of the sewage outlet, the state coordinate and the actual discharge state to a target environmental supervision platform to monitor the actual discharge state by using the target environmental supervision platform.

[0016] An electronic device is provided in a third aspect of the present application, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes the program to implement the automatic sewage outlet identification method under low-altitude oblique photography of a UAV as described in the above embodiments.

[0017] A computer readable storage medium is provided in a fourth aspect of the present application, and the computer readable storage medium stores a computer program, which is executed by a processor to implement the automatic sewage outlet identification method under low-altitude oblique photography of a UAV as described above.

[0018] A computer program product is provided in a fifth aspect of the present application, and the computer program product stores a computer program, which is executed by a processor to implement the automatic sewage outlet identification method under low-altitude oblique photography of a UAV as described above.

[0019] The embodiments of the present application can face the low-altitude close-range oblique aerial photography scene, adopt a targeted data enhancement strategy, improve the sewage outlet identification capability of the model in a complex terrain and a shielding environment, optimize the model structure, significantly reduce the calculation complexity while ensuring the identification accuracy, improve the detection speed, and be more suitable for actual patrol deployment requiring real-time performance, introduce a sewage state discrimination model, realize intelligent judgment of the running state of the sewage outlet, and provide technical support for identification of illegal sewage behavior. Thus, the problems that the existing method mainly relies on high-altitude aerial images, cannot effectively identify small and hidden sewage outlets in narrow waters such as ditches, the existing model is difficult to balance between detection accuracy and calculation efficiency, and cannot meet the real-time inspection needs of a UAV, and the mainstream sewage outlet investigation method only focuses on sewage outlet position identification and lacks effective discrimination of discharge state are solved.

[0020] Additional aspects and advantages of the present application will be made apparent from the following description with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and be more readily understood from the following description, by reference to the drawings, wherein:

[0022] Figure 1 A flow chart of a method for automatic identification of a sewage outlet under low-altitude oblique photography of a UAV according to an embodiment of the present application;

[0023] Figure 2 A schematic diagram of a technical solution of a method for automatic identification of a sewage outlet under low-altitude oblique photography of a UAV according to an embodiment of the present application;

[0024] Figure 3 A structural schematic diagram of a device for automatic identification of a sewage outlet under low-altitude oblique photography of a UAV according to an embodiment of the present application;

[0025] Figure 4 A structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Embodiments of the present application are described in detail below with reference to the accompanying drawings, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0027] An unmanned aerial vehicle low-altitude oblique photography-based automatic identification method and device for a pollution outlet are described below with reference to the accompanying drawings. In view of the fact that the existing method mainly relies on high-altitude aerial images and cannot effectively identify small and concealed pollution outlets in narrow waters such as ditches, and the existing model cannot balance between detection accuracy and computational efficiency and cannot meet the real-time inspection requirements of unmanned aerial vehicles, and the mainstream pollution outlet investigation method only focuses on the identification of the position of the pollution outlet and lacks effective discrimination of the discharge state, the present application provides an unmanned aerial vehicle low-altitude oblique photography-based automatic identification method. In this method, the model can be used for low-altitude close-range oblique aerial photography scenarios, and a targeted data enhancement strategy can be used to improve the model's ability to identify pollution outlets in complex terrain and occluded environments. The model structure is optimized to significantly reduce the computational complexity while ensuring the identification accuracy, thereby improving the detection speed and making it more suitable for real-time actual inspection deployment. A pollution discharge state discrimination model is introduced to intelligently judge the running state of the pollution outlet and provide technical support for the identification of illegal pollution behavior. Thus, the problems of the existing method mainly relying on high-altitude aerial images, being difficult to effectively identify small and concealed pollution outlets in narrow waters such as ditches, the existing model being difficult to balance between detection accuracy and computational efficiency, being difficult to meet the real-time inspection requirements of unmanned aerial vehicles, the mainstream pollution outlet investigation method only focusing on the identification of the position of the pollution outlet, and lacking effective discrimination of the discharge state are solved.

[0028] Specifically, Figure 1 A flowchart of an unmanned aerial vehicle low-altitude oblique photography-based automatic identification method provided by an embodiment of the present application is shown in the figure.

[0029] As Figure 1 shown, the unmanned aerial vehicle low-altitude oblique photography-based automatic identification method includes the following steps:

[0030] In step S101, at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic puzzle enhancement, and copy-paste in the data enhancement strategy is used to perform data enhancement on the images of the low-altitude unmanned aerial vehicle oblique photography to generate enhanced image information.

[0031] It can be understood that the present application is applicable to the detection and monitoring of small and easily occluded pollution outlets in complex water body environments such as rivers, ditches, and lakes. As Figure 2 shown, the method is divided into three parts: data enhancement, and subsequent construction of a pollution outlet identification model and a pollution state discrimination model.

[0032] In actual execution, the embodiment of the present application can introduce a data enhancement strategy for low-altitude unmanned aerial vehicle oblique photography in the model training stage. Considering that the images obtained by the unmanned aerial vehicle under different angles, postures, lightings and motion conditions may have problems such as rotation, distortion, blur and the like, random rotation, random cutting, image blur and the like are adopted as enhancement means to simulate complex situations in actual flight scenes. At the same time, in order to enhance the adaptability of the model to multiple targets and complex backgrounds, a mosaic enhancement and copy-paste strategy is further adopted, the former fuses four images into one to improve the multi-scale learning ability, and the latter increases multiple pollution outlet targets in the image to improve the recognition ability of the model to high-density pollution outlet scenes. All enhancement operations are enabled only in the training stage, and the originality of the image is maintained in the verification and test stages to ensure the consistency and accuracy of model evaluation.

[0033] The embodiment of the present application can adapt to the improvement of the small pollution outlet recognition ability of the low-altitude close-range aerial photography scene: in view of the problem of difficult identification of hidden and small pollution outlet targets in narrow waters such as ditches, the oblique view image and the targeted designed data enhancement strategy are innovatively adopted to improve the target perception ability of the model in complex terrain and shielding environment.

[0034] Optionally, in an embodiment of the present application, at least one image enhancement operation of random rotation, random cutting, image blur, mosaic puzzle enhancement and copy-paste in the data enhancement strategy is used to perform data enhancement on the image of low-altitude unmanned aerial vehicle oblique photography to generate enhanced image information, including: in the model training stage, the image of low-altitude unmanned aerial vehicle oblique photography is randomly rotated within a preset angle range to simulate the change of flight angle of the unmanned aerial vehicle, and the image of low-altitude unmanned aerial vehicle oblique photography is transformed by cutting to simulate the distortion of the lens; Gaussian blur and motion blur processing are applied to the image to simulate the imaging jitter when the unmanned aerial vehicle vibrates, and a mosaic puzzle enhancement method is used to splice images of different scenes to generate background samples meeting a preset high complexity condition; a copy-paste strategy is used to cut and paste the pollution outlet region into the target image to generate multi-target high-density samples; based on the change of flight angle, the distortion of the lens, the imaging jitter, the background samples and the multi-target high-density samples, the image of low-altitude unmanned aerial vehicle oblique photography is data enhanced to generate enhanced image information.

[0035] It can be understood that the preset angle range in the embodiment of the present application can be ±30 degrees clockwise / counterclockwise.

[0036] In the model training stage, image rotation processing is adopted to rotate the image clockwise / counterclockwise within a range of ±30 degrees to simulate the change of flight angle; shear transformation is used to simulate lens distortion; Gaussian blur and motion blur processing is applied to the image to simulate the imaging jitter problem caused by unmanned aerial vehicle vibration; mosaic puzzle enhancement technology is introduced to splice four different scene images to generate high complexity background samples and improve the multi-background generalization ability of the model; the copy and paste strategy is introduced to cut and paste the artificially labeled sewage outlet area to other images to construct multi-target high-density samples and improve the adaptability of the model to heavy pollution scenes. These enhancement strategies are only used in the training stage, and the original state of the image is maintained in the verification and test stages, that is, the image taken in the actual patrol will not use the above data enhancement strategies.

[0037] The embodiment of the present application faces the low-altitude close-range oblique aerial photography scene, adopts targeted data enhancement strategies, and improves the sewage outlet recognition ability of the model in complex terrain and occlusion environment.

[0038] In step S102, based on the enhanced image information, a multi-head self-attention mechanism is introduced into the backbone network of the pre-constructed sewage outlet recognition model, and a channel attention mechanism is introduced into the neck network to determine the boundary box region of the sewage outlet.

[0039] In actual execution, the embodiment of the present application constructs a lightweight sewage outlet recognition model based on enhanced image information. On the basis of the YOLO network structure, the present application further introduces an attention mechanism to improve the recognition ability of the model for small sewage outlet targets in complex backgrounds. Specifically, a multi-head self-attention mechanism is embedded in the backbone network to enhance the global perception ability of the model for the spatial position of the sewage outlet in the image, so that the model can capture the dependency relationship between the target and its surrounding environment as a whole; a channel attention mechanism is introduced into the neck network to adaptively adjust the response strength of different feature channels, strengthen the key channels related to the sewage outlet semantics, and suppress redundant or interference information. Through the synergistic effect of the two attention mechanisms, the embodiment of the present application effectively improves the feature extraction ability of the model in complex scenes and enhances the recognition robustness of occlusion, blur and multi-scale targets.

[0040] The embodiment of the present application can design a lightweight sewage outlet detection model with consideration of both precision and efficiency: under the premise of maintaining the lightweight model structure, an attention mechanism is introduced to improve the feature extraction ability of the model, making it more suitable for sewage outlet patrol scenes with high recognition accuracy requirements.

[0041] Optionally, in an embodiment of the present application, a multi-head self-attention mechanism is introduced into the backbone network of the pre-constructed pollution outlet identification model, and a channel attention mechanism is introduced into the neck network to determine the boundary box area of the pollution outlet, comprising: constructing a pollution outlet identification model using enhanced image information, and introducing a multi-head self-attention mechanism into the backbone network of the pollution outlet identification model to capture at least one regional feature of the pollution outlet that is blocked or confused in the low-altitude unmanned aerial vehicle oblique photography image; introducing a channel attention mechanism into the neck network to adaptively weight the response intensity of each feature channel, generating at least one semantic feature corresponding to the pollution outlet; outputting the position box of the pollution outlet according to the at least one regional feature and the at least one semantic feature using the pollution outlet identification model, and determining the boundary box area of the pollution outlet according to the position box.

[0042] Specifically, in the embodiment of the present application, the enhanced image is sent into the pollution outlet detection model. The backbone network part of the model introduces a multi-head self-attention mechanism to capture the regional features of the pollution outlet that may be blocked or confused in the image. The principle is to calculate the correlation between different regions of the whole image in parallel through multiple independent attention heads, thereby forming a strong spatial perception of the target position of the fuzzy edge and unclear texture of the pollution outlet. A channel attention mechanism is introduced in the neck layer of the network to adaptively weight the response intensity of each feature channel, effectively extracting semantic features related to the pollution outlet, while reducing the attention to interference factors such as water grass, stones, shorelines, etc. Finally, the identification model outputs the position box of a pollution outlet in the image, labeled as a set of boundary coordinates, and gives a confidence degree to judge whether it meets the effectiveness requirement.

[0043] The embodiment of the present application optimizes the model structure, significantly reduces the computational complexity while ensuring the recognition accuracy, and improves the detection speed, which is more suitable for real-time actual patrol deployment.

[0044] In step S103, the image of the low-altitude unmanned aerial vehicle oblique photography is cropped according to the boundary box area of the pollution outlet to obtain a cropped image, and the cropped image is input into the pre-constructed pollution state discrimination model to identify the actual discharge state of the pollution outlet.

[0045] It can be understood that the identification of the discharge state of the pollution outlet is of great significance to the real-time and legality of the pollution behavior, and is a key link to realize fine supervision.

[0046] In actual execution, an emission state discrimination model is designed in the embodiment of the application, which is used for classifying and judging the emission state of the identified emission port. The input of the model is the boundary box region of the emission port output by the emission port recognition model. The embodiment of the application can crop the image of low-altitude unmanned aerial vehicle oblique photography according to the boundary box region of the emission port to obtain a cropped image, and input the cropped image into the pre-constructed emission state discrimination model to identify the actual emission state of the emission port. The cropped image is input into the emission state classification model. The model is adjusted and retrained on the basis of the emission port recognition model, retains the original backbone network and attention mechanism setting, but is specially adapted and optimized for emission state categories in terms of loss function and sample label design. The state discrimination model also uses consistent data enhancement strategies in the training stage to ensure stable emission state judgment performance in actual inspection.

[0047] The embodiment of the application can integrate the emission state discrimination module into the intelligent monitoring system of emission behavior: based on the recognition of the emission port, the emission state discrimination model is innovatively introduced to realize dynamic classification and judgment of the emission behavior, make up for the blank of the prior art in emission state monitoring, and improve the effectiveness of system supervision.

[0048] Optionally, in an embodiment of the application, the image of low-altitude unmanned aerial vehicle oblique photography is cropped according to the boundary box region of the emission port to obtain a cropped image, and the cropped image is input into a pre-constructed emission state discrimination model to identify the actual emission state of the emission port, comprising: in the case of cropping the image of low-altitude unmanned aerial vehicle oblique photography, the boundary box region of the emission port is expanded outward according to a preset width and a preset height to generate a cropped image containing background environment information; the cropped image is input into a pre-constructed emission state discrimination model to identify the actual emission state of the emission port, wherein the emission state discrimination model is obtained by adjusting the emission category and outputting the emission state or the non-emission state.

[0049] As a possible implementation, the embodiment of the application can crop the image according to the boundary box position output by the above-mentioned step in the case of cropping the image of low-altitude unmanned aerial vehicle oblique photography. To contain sufficient background environment information and prevent truncation of water flow or emission traces, the embodiment expands the boundary box by 10% in width and height respectively for image extraction. The cropped image is input into the emission state discrimination model. The model structure is adjusted for task adaptation on the basis of the detection model, such as removing the neck layer, adjusting the category output to the state of “emission / non-emission”, and finally outputting the state of the emission port.

[0050] The embodiment of the application introduces the emission state discrimination model to realize intelligent judgment of the running state of the emission port, and provides technical support for identification of illegal emission behavior.

[0051] Optionally, in an embodiment of the present application, after identifying the actual emission state of the pollution outlet, further comprising: obtaining a real image of the pollution outlet, and identifying a state coordinate of the pollution outlet; uploading the real image of the pollution outlet, the state coordinate and the actual emission state to a target environmental supervision platform, so as to monitor the actual emission state by using the target environmental supervision platform.

[0052] Specifically, in the actual pollution outlet patrol, the pollution outlet state coordinate, the pollution state and the real image identified by the positioning system of the unmanned aerial vehicle can be packaged as the final result, which can be uploaded to the environmental supervision platform for supporting subsequent on-site verification and law enforcement decision.

[0053] The pollution outlet automatic identification method under low-altitude oblique photography of an unmanned aerial vehicle according to the embodiment of the present application can face the low-altitude close-range oblique aerial photography scene, adopt a targeted data enhancement strategy, improve the pollution outlet identification capability of the model in a complex terrain and a shielding environment, optimize the model structure, significantly reduce the calculation complexity while ensuring the identification accuracy, improve the detection speed, and be more suitable for actual patrol deployment requiring real-time performance, and introduce a pollution state discrimination model to realize intelligent judgment of the running state of the pollution outlet and provide technical support for identification of illegal pollution behavior. Thus, the problems that the existing method mainly relies on high-altitude aerial images and cannot effectively identify small and hidden pollution outlets in narrow valleys, the existing model cannot balance between detection accuracy and calculation efficiency, and cannot meet the real-time patrol needs of the unmanned aerial vehicle, and the mainstream pollution outlet investigation method only focuses on the identification of the position of the pollution outlet and lacks effective discrimination of the emission state are solved.

[0054] Secondly, the unmanned aerial vehicle low-altitude oblique photography pollution outlet automatic identification device according to the embodiment of the present application is described with reference to the accompanying drawings.

[0055] Figure 3 is a structural schematic diagram of the unmanned aerial vehicle low-altitude oblique photography pollution outlet automatic identification device according to the embodiment of the present application.

[0056] As Figure 3 shown, the unmanned aerial vehicle low-altitude oblique photography pollution outlet automatic identification device 10 includes a data enhancement module 100, a determination module 200 and an identification module 300.

[0057] Specifically, the data enhancement module 100 is configured to perform at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic puzzle enhancement and copy-paste in a data enhancement strategy, to perform data enhancement on the image of the low-altitude unmanned aerial vehicle oblique photography, to generate enhanced image information.

[0058] The determination module 200 is configured to introduce a multi-head self-attention mechanism in a backbone network of a pre-constructed sewage outlet identification model and introduce a channel attention mechanism in a neck network based on the enhanced image information, so as to determine a boundary box region of the sewage outlet.

[0059] The identification module 300 is configured to crop the image of the low-altitude unmanned aerial vehicle oblique photography according to the boundary box region of the sewage outlet to obtain a cropped image, and input the cropped image into a pre-constructed sewage state discrimination model to identify an actual discharge state of the sewage outlet.

[0060] Optionally, in an embodiment of the present application, the data enhancement module 100 comprises a first simulation unit, a second simulation unit, a generation unit and a data enhancement unit.

[0061] The first simulation unit is configured to randomly rotate the image of the low-altitude unmanned aerial vehicle oblique photography within a preset angle range to simulate the flight angle change of the unmanned aerial vehicle, and cut the image of the low-altitude unmanned aerial vehicle oblique photography to simulate the distortion of the lens in the model training stage.

[0062] The second simulation unit is configured to apply Gaussian blur and motion blur processing to the image to simulate the imaging jitter when the unmanned aerial vehicle vibrates, and use a mosaic puzzle enhancement method to splice images of different scenes to generate background samples meeting a preset high complexity condition.

[0063] The generation unit is configured to use a copy and paste strategy to cut and paste the sewage outlet region into a target image to generate multi-target high-density samples.

[0064] The data enhancement unit is configured to perform data enhancement on the image of the low-altitude unmanned aerial vehicle oblique photography based on the flight angle change, the distortion of the lens, the imaging jitter, the background samples and the multi-target high-density samples to generate enhanced image information.

[0065] Optionally, in an embodiment of the present application, the determination module 200 comprises a capture unit, an introduction unit and a determination unit.

[0066] The capture unit is configured to use the enhanced image information to construct a sewage outlet identification model, and introduce a multi-head self-attention mechanism in a backbone network of the sewage outlet identification model to capture at least one region feature of the sewage outlet that is blocked or confused in the image of the low-altitude unmanned aerial vehicle oblique photography.

[0067] The introduction unit is configured to introduce a channel attention mechanism in a neck network to adaptively weight the response intensity of each feature channel to generate at least one semantic feature corresponding to the sewage outlet.

[0068] The determining unit is configured to determine a bounding box region of the pollution outlet according to the position box by using a pollution outlet identification model according to the at least one regional feature and the at least one semantic feature.

[0069] Optionally, in an embodiment of the present application, the identification module 300 comprises an expansion unit and an identification unit.

[0070] The expansion unit is configured to expand the bounding box region of the pollution outlet according to a preset width and a preset height to generate a cropped image containing background environment information in the case of cropping the image of the low-altitude unmanned aerial vehicle oblique photography.

[0071] The identification unit is configured to input the cropped image into a pre-constructed pollution state discrimination model to identify the actual emission state of the pollution outlet, wherein the pollution state discrimination model is obtained by adjusting the pollution category and outputting the pollution state or the non-pollution state.

[0072] Optionally, in an embodiment of the present application, the automatic pollution outlet identification device 10 under unmanned aerial vehicle low-altitude oblique photography further comprises an acquisition module and a monitoring module.

[0073] The acquisition module is configured to acquire a real image of the pollution outlet and identify the state coordinates of the pollution outlet after identifying the actual emission state of the pollution outlet.

[0074] The monitoring module is configured to upload the real image, the state coordinates and the actual emission state of the pollution outlet to a target environmental supervision platform to monitor the actual emission state by using the target environmental supervision platform.

[0075] It should be noted that the foregoing explanation and description of the embodiment of the automatic pollution outlet identification method under unmanned aerial vehicle low-altitude oblique photography also applies to the embodiment of the automatic pollution outlet identification device under unmanned aerial vehicle low-altitude oblique photography, which will not be described here.

[0076] The automatic pollution outlet identification device under unmanned aerial vehicle low-altitude oblique photography according to the embodiment of the present application can face the low-altitude close-range oblique aerial photography scene, adopt a targeted data enhancement strategy, improve the pollution outlet identification capability of the model in a complex terrain and a shielding environment, optimize the model structure, significantly reduce the calculation complexity while ensuring the identification accuracy, improve the detection speed, and be more suitable for actual patrol deployment requiring real-time performance, introduce a pollution state discrimination model, realize intelligent judgment of the running state of the pollution outlet, and provide technical support for identification of illegal pollution behavior. Therefore, the problems that the existing method mainly relies on high-altitude aerial photography images, cannot effectively identify small and hidden pollution outlets in narrow waters such as ditches, the existing model cannot balance between detection accuracy and calculation efficiency, and cannot meet the real-time patrol requirements of unmanned aerial vehicles, and the mainstream pollution outlet investigation method only focuses on the identification of the position of the pollution outlet and lacks effective discrimination of the emission state are solved.

[0077] Figure 4 The structure schematic diagram of the electronic device provided by the embodiment of the present application is provided. The electronic device can include:

[0078] The memory 401, the processor 402 and the computer program stored in the memory 401 and executable on the processor 402.

[0079] The processor 402 implements the automatic identification method of the sewage outlet under the low-altitude oblique photography of the unmanned aerial vehicle provided in the above embodiment when executing the program.

[0080] Further, the electronic device further includes:

[0081] The communication interface 403 is used for the communication between the memory 401 and the processor 402.

[0082] The memory 401 is used for storing the computer program executable on the processor 402.

[0083] The memory 401 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0084] If the memory 401, the processor 402 and the communication interface 403 are independently implemented, the communication interface 403, the memory 401 and the processor 402 can be connected with each other through a bus and complete the communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 4 In the figure, only one thick line is used to represent, but it does not mean that there is only one bus or only one type of bus.

[0085] Optionally, in the specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can complete the communication between each other through an internal interface.

[0086] The processor 402 can be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to perform the operations of the embodiments of the application.

[0087] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method for automatically identifying a sewage outlet in low-altitude oblique photography of a UAV.

[0088] The embodiments of the present application also provide a computer program product, which stores a computer program. The computer program is executed by a processor to implement the method for automatically identifying a sewage outlet in low-altitude oblique photography of a UAV.

[0089] In the description of the present application, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or N embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0090] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0091] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for performing steps associated with the functions described in the flow charts or elsewhere herein, and that the various embodiments of preferred implementations can perform these functions in the sequence indicated, in an electrical circuit, an ASIC, or by other means, as will occur to those skilled in the art. The description of a process or method that is described as following a flow chart, flow diagram, or described elsewhere herein can be understood as representing one or more code modules, segments, or portions of code that include one or more executable instructions for performing steps associated with the functions described in the flow charts or elsewhere herein.

[0092] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0093] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0094] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0095] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0096] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for automatically identifying a sewage outlet under low-altitude oblique photography of a UAV, characterized in that, The method comprises the following steps: performing data enhancement on the low-altitude unmanned aerial oblique photography image based on at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic puzzle enhancement and copy-paste in a data enhancement strategy to generate enhanced image information; introducing a multi-head self-attention mechanism into a backbone network of a pre-constructed sewage outlet identification model and introducing a channel attention mechanism into a neck network based on the enhanced image information to determine a boundary box region of the sewage outlet; cropping the low-altitude unmanned aerial oblique photography image according to the boundary box region of the sewage outlet to obtain a cropped image, and inputting the cropped image into a pre-constructed sewage state discrimination model to identify an actual discharge state of the sewage outlet.

2. The method of claim 1, wherein, The method of performing data enhancement on the low-altitude unmanned aerial oblique photography image based on at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic puzzle enhancement and copy-paste in a data enhancement strategy to generate enhanced image information comprises: in a model training phase, performing random rotation on the low-altitude unmanned aerial oblique photography image within a preset angle range to simulate flight angle changes of the unmanned aerial vehicle, and performing shear transformation on the low-altitude unmanned aerial oblique photography image to simulate lens distortion; applying Gaussian blurring and motion blurring processing to the image to simulate imaging jitter when the unmanned aerial vehicle vibrates, and using a mosaic puzzle enhancement method to splice images of different scenes to generate background samples meeting a preset high complexity condition; using a copy-paste strategy to cut and paste the sewage outlet region into a target image to generate multi-target high-density samples; performing data enhancement on the low-altitude unmanned aerial oblique photography image based on the flight angle changes, the lens distortion, the imaging jitter, the background samples and the multi-target high-density samples to generate the enhanced image information.

3. The method of claim 1, wherein, The method of introducing a multi-head self-attention mechanism into a backbone network of a pre-constructed sewage outlet identification model and introducing a channel attention mechanism into a neck network to determine a boundary box region of the sewage outlet comprises: constructing the sewage outlet identification model using the enhanced image information, and introducing the multi-head self-attention mechanism into the backbone network of the sewage outlet identification model to capture at least one regional feature of the sewage outlet that is blocked or confused in the low-altitude unmanned aerial oblique photography image; introducing the channel attention mechanism into the neck network to adaptively weight the response intensity of each feature channel to generate at least one semantic feature corresponding to the sewage outlet; outputting a position box of the sewage outlet from the sewage outlet identification model according to the at least one regional feature and the at least one semantic feature, and determining the boundary box region of the sewage outlet according to the position box.

4. The method of claim 1, wherein, The method of cropping the low-altitude unmanned aerial oblique photography image according to the boundary box region of the sewage outlet to obtain a cropped image, and inputting the cropped image into a pre-constructed sewage state discrimination model to identify an actual discharge state of the sewage outlet comprises: In the case of cropping the image of the low-altitude unmanned aerial oblique photography, the boundary box region of the sewage outlet is expanded outward according to a preset width and a preset height to generate the cropped image containing background environment information; The cropped image is input into a pre-constructed sewage state discrimination model to identify the actual discharge state of the sewage outlet, wherein the sewage state discrimination model is obtained by adjusting the sewage category and outputting the sewage state or the non-sewage state.

5. The method of claim 1, wherein, After identifying the actual discharge state of the sewage outlet, further comprising: obtaining a real image of the sewage outlet and identifying the state coordinates of the sewage outlet; uploading the real image of the sewage outlet, the state coordinates and the actual discharge state to a target environmental supervision platform to monitor the actual discharge state by using the target environmental supervision platform.

6. An automatic discharge outlet identification device under low-altitude oblique photography of a UAV, characterized in that, comprising: a data enhancement module for performing at least one image enhancement operation of random rotation, random cutting, image blurring, mosaic puzzle enhancement and copy-paste based on a data enhancement strategy to enhance the image of low-altitude unmanned aerial oblique photography to generate enhanced image information; a determination module for introducing a multi-head self-attention mechanism in the backbone network of a pre-constructed sewage outlet recognition model based on the enhanced image information, and introducing a channel attention mechanism in the neck network to determine the boundary box region of the sewage outlet; an identification module for cropping the image of the low-altitude unmanned aerial oblique photography according to the boundary box region of the sewage outlet to obtain a cropped image, and inputting the cropped image into a pre-constructed sewage state discrimination model to identify the actual discharge state of the sewage outlet.

7. The apparatus of claim 6, wherein, The data enhancement module comprises: a first simulation unit for simulating the flight angle change of the unmanned aerial vehicle by randomly rotating the image of the low-altitude unmanned aerial oblique photography within a preset angle range, and simulating the distortion of the lens by cutting the image of the low-altitude unmanned aerial oblique photography; a second simulation unit for applying Gaussian blur and motion blur processing to the image to simulate the imaging jitter when the unmanned aerial vehicle vibrates, and using a mosaic puzzle enhancement method to splice images of different scenes to generate background samples that meet a preset high complexity condition; a generation unit for cutting and pasting the sewage outlet region into a target image using a copy-paste strategy to generate multi-target high-density samples; a data enhancement unit for enhancing the image of the low-altitude unmanned aerial oblique photography based on the flight angle change, the distortion of the lens, the imaging jitter, the background samples and the multi-target high-density samples to generate the enhanced image information.

8. An electronic device, comprising: comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the automatic identification method of the sewage outlet under the low-altitude oblique photography of the unmanned aerial vehicle according to any one of claims 1-5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the automatic identification method of the sewage outlet under the low-altitude oblique photography of the unmanned aerial vehicle according to any one of claims 1-5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed to implement the automatic identification method of sewage outlet under low-altitude oblique photography of the unmanned aerial vehicle according to any one of claims 1-5.

Images