Method for controlling UAV image quality with automatic selection of repeat shooting, optical-electronic system parameters and processing mode

Abstract

FIELD: unmanned aerial vehicles; image processing.SUBSTANCE: invention relates to the processing of images obtained by the UAV's optical electronic system. The claimed method for improving image quality includes: acquiring an image, processing it and using it for observation, as well as the following stages: generating an image quality passport (IQP) with assessments of motion blur, line-by-line geometric distortions, noise level, illumination, dynamic range, atmospheric degradation, spatial detail, overlap with adjacent images, and image suitability for object recognition; selecting an on-board processing mode based on the IQP: correction of motion blur and line-by-line distortions, local improvement of low illumination, compensation for atmospheric degradation and multi-frame increase of spatial resolution; recalculating the image suitability indicator for recognition; setting a repeat shooting if the suitability threshold is not reached; transmitting the IQP and processing information to the object recognition module.EFFECT: increase in the reliability of object recognition.4 cl

Description

[0001] The invention relates to unmanned aerial systems, survey control systems, optical-electronic surveillance equipment, digital image processing and automated quality control of aerial photography.

[0002] A known UAV (patent for invention RU No. 2798604) and a method for performing aerial photography are disclosed, comprising a flight controller, a camera, an onboard computer, a camera control module, and an image quality determination module, all configured in a specific manner. To perform aerial photography, images are acquired from the camera at intervals specified in the flight mission. The number of images from the camera is determined and compared with a predetermined value. If this number is reached and the image quality metric is less than a predetermined threshold, a computer-assisted decision is made on the need for recalibration with a return to the survey point. This improves image quality, accelerates and reduces resource costs for aerial photography of a given area to the required quality, improves orthomosaic accuracy, and accelerates and reduces resource costs for generating orthomosaics of the required quality.

[0003] Also considered as analogs are modern solutions for onboard UAV image enhancement, low-light processing, atmospheric degradation removal, multi-frame resolution enhancement, and image stabilization. A common drawback of known solutions is that they enhance existing images and, as a rule, do not form a closed technical loop in which the frame quality certificate controls subsequent shooting parameters, camera mode, sensor channel selection, and onboard processing mode.

[0004] Disadvantages of existing technical solutions:

[0005] Known systems do not integrate frame quality assessment with re-shooting and camera parameters control.

[0006] The processing mode is often pre-selected and does not depend on the underlying cause of degradation.

[0007] The recognition module receives an image without a quality technical passport.

[0008] The UAV may leave the area before obtaining a usable image.

[0009] The technical challenge is to create a closed-loop method for improving the quality of UAV images, in which frame degradation is automatically classified, a technically appropriate processing mode is selected, and, if the quality is insufficient, a task for re-shooting with changes in the observation parameters is generated.

[0010] The solution to the technical problem is ensured due to the fact that the Method for closed-loop enhancement of the quality of images obtained by the optoelectronic system of an unmanned aerial vehicle includes obtaining an image, digitally processing it and using the result for observation, characterized in that after obtaining the image, an image quality passport is formed containing assessments of at least blur, line-by-line geometric distortions, noise level, illumination, dynamic range, atmospheric degradation, spatial detail, overlap with adjacent images, the significance of objects and the suitability of the image for recognizing objects, according to the said passport the leading cause of image degradation is determined, an on-board processing mode is selected from a set of modes including correction of blur and line-by-line distortions, local improvement of low illumination, compensation for atmospheric degradation and multi-frame increase in spatial resolution,the selected processing mode is performed only for the areas requiring improvement, after processing the image suitability index for recognition is recalculated, if the specified suitability threshold is not reached, a re-shooting task is automatically generated with a change in at least one parameter from the group: flight altitude, speed, angle, course, frame overlap, shutter speed, gain, frame rate, stabilization mode or sensor channel, in the presence of energy and time constraints, the minimum sufficient action for processing or re-shooting is selected taking into account the significance of the object, after which the improved image, quality certificate and information about the completed processing are transmitted to the object recognition module and / or the mission control module.

[0011] The technical result consists in increasing the proportion of suitable images in a mission, reducing the number of unsuitable frames, increasing the reliability of object recognition, reducing repetitive flights and rationally using onboard computing and energy resources.

[0012] The essence of the invention.

[0013] Image quality passport is a structured set of technical frame indicators, including assessments of blur, noise, illumination, atmospheric degradation, detail, and suitability for recognition.

[0014] Leading cause of degradation - the main factor of image deterioration, determined by the quality passport.

[0015] Closed loop quality control is a sequence of operations in which image quality assessment controls the processing mode and parameters of subsequent shooting.

[0016] Minimum sufficient action - a processing or re-capture action that provides the required quality with a minimum expenditure of time, energy and computing resources.

[0017] The essence of the invention lies in the transition from isolated image enhancement to a closed-loop technical quality control system. During a mission, a UAV acquires frames and generates a quality certificate for each frame. The quality certificate contains assessments of blur, distortion, noise, illumination, dynamic range, atmospheric degradation, spatial resolution, overlap with adjacent frames, suitability for object recognition, and the importance of objects contained in the frame. These assessments are generated both based on the image itself and on telemetry data: speed, altitude, vibration, angular velocity, exposure, gimbal position, and shooting conditions.

[0018] After generating a quality certificate, the system determines the primary cause of degradation. If the frame is degraded due to blur, inertial-consistent correction is selected. If low light is the cause, local exposure correction with noise control is selected. If haze is the cause, atmospheric-consistent restoration is selected. If insufficient detail is the cause and there are adjacent frames with useful overlap, multi-frame super-resolution is selected. If the degradation is critical and restoration is unreliable, the frame is not aggressively improved, but a retake is created.

[0019] The retake task is technically specific. Depending on the cause of degradation, the system adjusts the altitude, speed, angle, course, overlap, exposure, shutter speed, gain, frame rate, stabilization mode, or sensor channel. For example, if there's blur, reduce the speed or shutter speed; in low light, change the exposure or reschedule the shoot; in hazy conditions, change the altitude or angle; and if there's insufficient detail, increase the overlap or decrease the altitude. Given energy and time constraints, the system evaluates the object's significance and selects the minimum required action: local processing, retake of a single section, or a complete repeat of the route.

[0020] Unlike known methods, the proposed solution generates not only an improved image but also a management decision aimed at obtaining a usable image. The processed image is transmitted for recognition along with a quality certificate and an indicator of the quality's origin: restored by processing, confirmed by re-capture, or requiring verification. This allows the recognition module to take into account the quality of the initial observation, and the UAV control system to continue the mission until critical objects are acquired with the required reliability. This approach ensures a technical result due to the functional connection between quality assessment, processing selection, image capture management, and recognition.

[0021] The quality passport can be generated separately for the entire frame and for local areas of the frame.

[0022] The leading cause of degradation can be selected based on the maximum contribution of the corresponding indicator to the reduction of recognition suitability.

[0023] The minimum sufficient action can be selected according to the criterion of minimizing energy and time consumption when reaching the suitability threshold.

[0024] Repeat shooting can only be assigned to the area containing the high-value object.

[0025] The information about the completed processing can be used by the recognition module to change the object confirmation threshold.

[0026] The claimed solution is a technical method for controlling image acquisition, camera, processing modes, flight parameters and re-shooting of the UAV, and not a computer program as such.

[0027] The quality certificate identifies the cause of degradation; the selection of the processing mode based on the cause prevents incorrect filter application; re-evaluation of suitability eliminates unreliable results; the assignment of a repeat shoot ensures the acquisition of a new technically suitable frame; taking into account the energy and significance of the object makes the method practically feasible.

[0028] The method is applicable to industrial UAVs for inspection of energy, oil and gas infrastructure, transportation, construction, agriculture, facility security, geodesy, and emergency monitoring. Implementation is possible by adding a hardware and software module for quality assessment and repeat survey management.

[0029] Implementation example

[0030] A UAV inspects a solar power plant. For some frames, the quality certificate shows blur due to wind, for others, overexposed due to glare, and for distant rows, insufficient detail. The system applies inertial-consistent correction to the blurred frames, local tone mapping to the overexposed ones, and a retake with a lower altitude and greater overlap to the distant rows. If, after processing, the object's quality falls below the threshold, a command is generated to retake the image from a different angle. The operator receives technically verified images with a quality certificate.

Claims

1. A method for managing the quality of images obtained by an optoelectronic system of an unmanned aerial vehicle, including obtaining an image, digitally processing it and using the result for observation, characterized in that after obtaining the image, an image quality passport is generated containing assessments of blur, line-by-line geometric distortions, noise level, illumination, dynamic range, atmospheric degradation, spatial detail, overlap with adjacent images and the suitability of the image for recognizing objects, according to said passport, an on-board processing mode is selected from a set of modes including correction of blur and line-by-line distortions, local improvement of low illumination, compensation for atmospheric degradation and multi-frame increase in spatial resolution, and when blur or line-by-line geometric distortions are detected, their correction is selected,in low light conditions, local low light enhancement with noise control is selected; in case of atmospheric degradation, atmospheric degradation compensation is selected; and in case of insufficient spatial detail and the presence of adjacent overlapping images, multi-frame spatial resolution enhancement is selected; after completing the selected processing mode, the image suitability indicator for recognition is recalculated; if the specified suitability threshold is not reached, a re-shooting task is automatically generated with a change in one or more parameters from a group including flight altitude, speed, angle, course, frame overlap, shutter speed, gain, frame rate, stabilization mode, or sensor channel; after which the processed image, quality certificate, and information about the completed processing are transmitted to the object recognition module, to the mission control module, or to both of the specified modules.

2. The method according to paragraph 1, characterized in that the image quality passport is generated separately for the entire frame and for local areas of the frame.

3. The method according to paragraph 1, characterized in that when forming a task for repeated shooting in the case of blur, the flight speed or shutter speed is reduced, in the case of atmospheric degradation, the flight altitude or viewing angle is changed, and in the case of insufficient spatial detail, the flight altitude is reduced or the overlap of frames is increased.

4. The method according to paragraph 1, characterized in that the processed image is transmitted to the recognition module together with a quality certificate and a quality origin indicator indicating that the image quality has been restored by processing, confirmed by re-shooting, or requires verification.