Low-altitude economic unmanned aerial vehicle photography image enhancement method

Abstract

The application discloses a low-altitude economy unmanned aerial vehicle photography image enhancement method, which comprises the following steps: emitting a laser pulse and acquiring an echo signal of the laser pulse to obtain echo power data varying with echo time; collecting a first image under active illumination and a second image under passive illumination in the same observation period; calculating atmospheric extinction coefficient distribution on a path from an observation point to a scene point according to the echo power data, and constructing a three-dimensional atmospheric particle field based on the atmospheric extinction coefficient distribution; calculating a target direct illumination component of the scene, and simultaneously acquiring a spectral signal of the second image as a target spectral signal; acquiring a target intrinsic material parameter under passive illumination and a target illumination parameter after interference elimination; and synthesizing and outputting an enhanced image through a graphics renderer. The application enables the final enhanced image to be established on a real atmospheric physical model, solves the problem of scene information loss caused by the prior art, and improves the quality of the enhanced image.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and more specifically to an image enhancement method based on low-altitude economical UAV photography. Background Technology

[0002] Low-altitude unmanned aerial vehicles (UAVs) generally refer to drones flying in low-altitude and ultra-low-altitude airspace below 1000 m (inclusive). They serve as excellent carriers for diverse technological equipment, and high-performance, intelligent low-altitude UAVs will become a backbone force supporting the development of the low-altitude economy. With the rapid development of artificial intelligence technology, UAVs are gradually moving towards intelligence. Strengthening research on key technology systems such as autonomous intelligent technology for low-altitude UAVs, and completing the transformation from remote-controlled flight to autonomous flight in complex environments, is of great significance for the innovative application of low-altitude UAVs and the high-quality development of the low-altitude economy.

[0003] Photography, as an important function of low-altitude drones, places increasingly higher demands on video images. However, atmospheric particles such as aerosols, fog, haze, and dust in the low-altitude environment severely scatter and absorb light, leading to problems such as reduced contrast, color distortion, and blurred details in the captured images.

[0004] Current image enhancement techniques can be broadly divided into two categories. One category is based on traditional image processing methods (such as histogram equalization, Retinex algorithm, etc.). These methods operate only at the pixel level, lack physical basis, and are prone to over-enhancement or the introduction of noise. The other category is based on deep learning methods, which use intelligently generated physical scenes to enhance images. However, such physical scenes are prone to information loss, resulting in distortion of the final enhanced image. Summary of the Invention

[0005] This invention provides a method for enhancing images captured by a low-altitude economic unmanned aerial vehicle (UAV) to at least partially solve the aforementioned technical problems.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for enhancing images captured by a low-altitude economic unmanned aerial vehicle (UAV), comprising:

[0007] A laser pulse is emitted and the echo signal of the laser pulse is acquired to obtain echo power data that varies with echo time, wherein the echo time is the time from the emission of the laser pulse to the receipt of the echo signal.

[0008] Acquire the first image of active illumination and the second image of passive illumination within the same observation period;

[0009] Based on the echo power data, the atmospheric extinction coefficient distribution along the path from the observation point to the scene point is calculated using an atmospheric inversion algorithm, and a three-dimensional atmospheric particle field is constructed based on the atmospheric extinction coefficient distribution.

[0010] Calculate the pixel-level difference between the first image and the second image to obtain the target direct illumination component, and simultaneously acquire the spectral signal of the second image as the target spectral signal;

[0011] Using the three-dimensional atmospheric particle field as the physical constraint environment for photon propagation, and the target direct illumination component and target spectral signal as the joint target, a reverse simulation process based on Monte Carlo path tracing is performed to obtain the target intrinsic material parameters under passive illumination and the target illumination parameters after interference removal.

[0012] Based on the intrinsic material parameters of the target, the target lighting parameters are used as lighting conditions, and an enhanced image is synthesized and output through a graphics renderer.

[0013] In conjunction with the first aspect above, in one possible implementation, the atmospheric inversion algorithm is the Klett inversion algorithm, and the construction of the three-dimensional atmospheric particle field includes:

[0014] Calculate the distance correction signal based on the echo power data;

[0015] Based on the distance correction signal, the extinction coefficient distributed with distance is obtained by the Klett inversion algorithm;

[0016] The extinction coefficient is associated with the three-dimensional coordinates of the scene to construct the three-dimensional atmospheric particle field.

[0017] In conjunction with the first aspect above, in one possible implementation, the expression for the echo power is: Where t represents the echo time; Let t represent the echo power at time t; P represent the laser emission power; c represent the speed of light; Δt represent the pulse width; A represent the receiving area of ​​the lidar telescope; z(t) represent the detection distance, calculated by z(t) = (c*t) / 2. T(z(t)) represents the backscattering coefficient at a distance z(t); T(z(t)) represents the one-way atmospheric transmittance of the laser radar at a distance z(t).

[0018] In conjunction with the first aspect above, in one possible implementation, the process of using the three-dimensional atmospheric particle field as the physical constraint environment for photon propagation, and using the target direct illumination component and the target spectral signal as the joint target, to perform a Monte Carlo path tracing-based inverse simulation to obtain the target intrinsic material parameters under passive illumination and the target illumination parameters after interference removal includes:

[0019] Assign initial values ​​to the intrinsic material parameters and lighting parameters of the scene, wherein the initial values ​​are random values ​​or preset values;

[0020] Virtual sampling rays are emitted from the pixels of a virtual camera to simulate the scattering of light in the three-dimensional atmospheric particle field and its reflection and refraction with the scene surface;

[0021] The path contribution is calculated using the Monte Carlo integral algorithm to obtain the simulated spectral signal and direct illumination component;

[0022] The difference between the simulated spectral signal and the target spectral signal, and the difference between the simulated direct illumination component and the target direct illumination component, are used together as the loss function to update the intrinsic material parameters and illumination parameters through the gradient descent algorithm.

[0023] The intrinsic material parameters and illumination parameters are continuously adjusted. When the loss function converges to below a preset threshold, the current intrinsic material parameters and illumination parameters are determined as the intrinsic material parameters under passive illumination and the illumination parameters after interference removal.

[0024] In conjunction with the first aspect above, in one possible implementation, the step of calculating the path contribution using the Monte Carlo integration algorithm to obtain the simulated spectral signal and direct illumination component includes:

[0025] Calculate the contribution of a single pixel to a single path, and add it to the total path contribution. At the same time, determine whether the path is a direct path. If it is, add the contribution of the path to the direct path contribution.

[0026] After the path tracing of all rays in a single pixel is completed, the average value of the total path contribution is calculated to obtain the spectral signal pixel value of that pixel. At the same time, the average value of the direct path contribution is calculated to obtain the direct illumination component pixel value of that pixel.

[0027] By integrating the spectral signal pixel values ​​and direct illumination component pixel values ​​of all pixels in the virtual camera, a simulated spectral signal and a simulated direct illumination component are obtained.

[0028] In conjunction with the first aspect above, in one possible implementation, the condition for determining that the path is a direct path is: the path starts from the virtual camera, undergoes only one reflection or refraction on the scene surface, and then directly hits the active lighting source defined by the lighting parameters.

[0029] In conjunction with the first aspect above, in one possible implementation, the expression for the loss function is: Where R and L represent the intrinsic material parameters and lighting parameters to be solved, respectively; Represents the target spectral signal; This represents the simulated spectral signal obtained based on the current intrinsic material parameters R and illumination parameters L. Indicates the direct illumination component of the target. λ represents the direct illumination component obtained from the simulation based on the current intrinsic material parameters R; λ represents the regularization coefficient.

[0030] In conjunction with the first aspect above, in one possible implementation, the step of using the target lighting parameters as lighting conditions based on the target intrinsic material parameters, and synthesizing and outputting the enhanced image through a graphics renderer, includes:

[0031] The intrinsic material parameters of the target are assigned to a virtual three-dimensional scene model to define the optical reflection properties of each surface in the scene model.

[0032] The target illumination parameters are selected as illumination conditions to provide the light source required for the enhanced image;

[0033] In a graphics renderer, the physical process of simulating light rays emanating from the selected lighting conditions, illuminating the virtual 3D scene model, and entering the virtual camera after being reflected by the scene surface is simulated. The parameters of the virtual camera are consistent with the parameters of the camera that acquires the first and second images.

[0034] Calculate the radiance value received by each pixel on the imaging plane of the virtual camera, quantize these radiance values ​​into digital signals, and output the enhanced image.

[0035] In conjunction with the first aspect above, one possible implementation also includes a preset lighting model, which can replace the target lighting parameters as lighting conditions;

[0036] The preset lighting model uses uniform parallel light as the basic lighting model and is configured with preset parameters including light source direction, light intensity and spectrum.

[0037] Secondly, the present invention provides a low-altitude economic drone photography image enhancement system, including the drone body and a control center;

[0038] The drone body includes:

[0039] A lidar module is used to emit laser pulses and receive their echo signals to obtain echo power data that varies with echo time.

[0040] The multispectral imaging module is used to acquire a first image under active illumination and a second image under passive illumination within the same observation period;

[0041] A synchronization module is electrically connected to both the lidar module and the multispectral imaging module to ensure that the laser pulse emission is synchronized with the first image acquisition in time.

[0042] The first communication module is used to send the collected data to the control center;

[0043] The control center includes:

[0044] The second communication module is used to receive echo power data, a first image, and a second image from the UAV body.

[0045] The construction module, connected to the second communication module, is used to calculate the atmospheric extinction coefficient distribution and construct a three-dimensional atmospheric particle field based on the echo power data using an atmospheric inversion algorithm.

[0046] The target signal extraction module, connected to the second communication module, is used to calculate the pixel-level difference between the first image and the second image, obtain the direct illumination component of the target, and acquire the spectral signal of the second image as the target spectral signal.

[0047] The solution module is connected to the construction module and the target signal extraction module respectively; it is used to perform a reverse simulation process based on Monte Carlo path tracing with the three-dimensional atmospheric particle field as the physical constraint environment and the direct illumination component and the target spectral signal as the joint target, to solve the intrinsic material parameters of the target under passive illumination and the target illumination parameters after interference removal.

[0048] The image synthesis module, connected to the solution module, is used to synthesize and output an enhanced image based on the intrinsic material parameters and lighting parameters of the target using a graphics renderer.

[0049] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to realize the functions of the various modules of the control center described above.

[0050] Fourthly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor enables the processor to perform the functions of the modules of the control center described above.

[0051] In summary, the low-altitude economic UAV photography image enhancement method provided by the embodiments of the present invention provides precise physical constraints for image enhancement by constructing a three-dimensional atmospheric particle field; by using the direct illumination component and the target spectral signal as joint targets, the intrinsic material parameters of the scene and the illumination parameters after interference removal are solved in reverse, so that the final enhanced image is based on a real atmospheric physical model, which solves the problem of scene information loss caused by the prior art and improves the quality of the enhanced image. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0053] To better understand the present invention and its beneficial effects, the following description will be made in conjunction with the accompanying drawings, wherein the same reference numerals denote the same parts in the following description.

[0054] Figure 1 This is a flowchart of an image enhancement method based on low-altitude economic drone photography in one embodiment of the present invention.

[0055] Figure 2 This is an architecture diagram of a low-altitude economic drone-based image enhancement system according to one embodiment of the present invention.

[0056] Figure 3 This is a schematic diagram of the structure of the drone body in one embodiment of the present invention.

[0057] Figure 4 This is a schematic diagram of the control center in one embodiment of the present invention.

[0058] Figure 5 This is a schematic diagram of the structure of a computer device in one embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the protection scope of the present invention.

[0060] Figure 1 This is an exemplary embodiment of the low-altitude economic drone photography image enhancement method provided by the present invention. The method's steps are merely one possible implementation of the present invention. Figure 1 As shown, the image enhancement method based on low-altitude economic UAV photography provided in this embodiment of the invention includes the following steps:

[0061] S10, emit a laser pulse and acquire the echo signal of the laser pulse to obtain echo power data that varies with echo time, wherein the echo time is the time from emitting the laser pulse to receiving the echo signal.

[0062] In this embodiment, the laser pulse is emitted by a lidar module mounted on the drone, and its specific model and style are not limited.

[0063] S20: Acquire the first image of active illumination and the second image of passive illumination within the same observation period.

[0064] In this embodiment, the same observation period can be understood as the time spent completing one active illumination image acquisition and one passive illumination image acquisition process. Active illumination shooting refers to shooting under active light source illumination, while passive illumination shooting refers to shooting without active light source illumination. In this embodiment, the active light source is a laser pulse, that is, the first image is contributed by laser illumination + natural light illumination + atmospheric scattered light, and the second image is contributed by natural light illumination + atmospheric scattered light.

[0065] It should be noted that although the steps above are numbered as S10 and S20, this is only for the convenience of description and does not mean that the two steps are sequential. In fact, the laser pulse emission and the first image acquisition are synchronized in time, ensuring the contribution of laser illumination.

[0066] S30, based on the echo power data, calculate the atmospheric extinction coefficient distribution along the path from the observation point to the scene point using an atmospheric inversion algorithm, and construct a three-dimensional atmospheric particle field based on the atmospheric extinction coefficient distribution.

[0067] In this embodiment, the atmospheric extinction coefficient is a physical quantity that describes the degree to which the intensity of light attenuates due to scattering and absorption when it propagates in the atmosphere. In this embodiment, instead of guessing or estimating the atmospheric conditions as in the prior art, the distribution of the atmospheric extinction coefficient is actively calculated.

[0068] S40, calculate the pixel-level difference between the first image and the second image to obtain the target direct illumination component, and simultaneously acquire the spectral signal of the second image as the target spectral signal.

[0069] In this embodiment, the pixel value difference refers to the subtraction operation of grayscale value or color value performed on each pixel of the first image and the second image.

[0070] S50, using the three-dimensional atmospheric particle field as the physical constraint environment for photon propagation, and using the target direct illumination component and the target spectral signal as the joint target, a reverse simulation process based on Monte Carlo path tracing is executed to obtain the target intrinsic material parameters under passive illumination and the target illumination parameters after interference removal.

[0071] In this embodiment, the reverse simulation process is relative to the forward Monte Carlo method. The forward Monte Carlo method is a process in which the cause (parameters) is known, the simulation (path tracing) is performed, and the result (synthesized image) is obtained. The reverse Monte Carlo method is a process in which the result (observed image) is known, the iterative optimization is performed, and then the cause (parameters) is obtained. That is, the final image is known, but the real material and lighting of the scene are unknown, and they need to be solved in reverse.

[0072] S60, based on the intrinsic material parameters of the target, the target lighting parameters are used as lighting conditions, and an enhanced image is synthesized and output through a graphics renderer.

[0073] In this embodiment, the graphics renderer is a specialized software or hardware engine whose core function is to transform a mathematically described, virtual three-dimensional scene into a realistic two-dimensional digital image.

[0074] In summary, the low-altitude economic UAV photography image enhancement method provided by the embodiments of the present invention provides precise physical constraints for image enhancement by constructing a three-dimensional atmospheric particle field; by using the direct illumination component and the target spectral signal as joint targets, the intrinsic material parameters of the scene and the illumination parameters after interference removal are solved in reverse, so that the final enhanced image is based on a real atmospheric physical model, which solves the problem of scene information loss caused by the prior art and improves the quality of the enhanced image.

[0075] To provide a more intuitive understanding of the methods in the above embodiments, this invention also provides a low-altitude economic unmanned aerial vehicle (UAV) image enhancement system.

[0076] Figure 2 This is an exemplary embodiment of the low-altitude economic drone photography image enhancement system provided by the present invention. For example... Figure 2 As shown, the low-altitude economic drone photography image enhancement system provided in this embodiment of the invention includes the drone body and a control center.

[0077] Figure 3 This is a schematic diagram of the structure of the UAV body provided in an exemplary embodiment of the present invention, such as... Figure 3 As shown, the drone body includes:

[0078] A lidar module is used to emit laser pulses and receive their echo signals to obtain echo power data that varies with echo time.

[0079] The multispectral imaging module is used to acquire a first image under active illumination and a second image under passive illumination within the same observation period;

[0080] A synchronization module is electrically connected to both the lidar module and the multispectral imaging module to ensure that the laser pulse emission is synchronized with the first image acquisition in time.

[0081] The first communication module is used to send the collected data to the control center;

[0082] Figure 4 This is a schematic diagram of the control center provided in an exemplary embodiment of the present invention, such as... Figure 4 As shown, the control center includes:

[0083] The second communication module is used to receive echo power data, a first image, and a second image from the UAV body.

[0084] The construction module, connected to the second communication module, is used to calculate the atmospheric extinction coefficient distribution and construct a three-dimensional atmospheric particle field based on the echo power data using an atmospheric inversion algorithm.

[0085] The target signal extraction module, connected to the second communication module, is used to calculate the pixel-level difference between the first image and the second image, obtain the direct illumination component of the target, and acquire the spectral signal of the second image as the target spectral signal.

[0086] The solution module is connected to the construction module and the target signal extraction module respectively; it is used to perform a reverse simulation process based on Monte Carlo path tracing with the three-dimensional atmospheric particle field as the physical constraint environment and the direct illumination component and the target spectral signal as the joint target, to solve the intrinsic material parameters of the target under passive illumination and the target illumination parameters after interference removal.

[0087] The image synthesis module, connected to the solution module, is used to synthesize and output an enhanced image based on the intrinsic material parameters and lighting parameters of the target using a graphics renderer.

[0088] In summary, the low-altitude economic UAV photography image enhancement system provided by the embodiments of the present invention explains the relationship between the modules from an architectural perspective, thereby making it easier to understand the above-mentioned UAV photography image enhancement method.

[0089] Furthermore, in one embodiment of the present invention, the atmospheric inversion algorithm is the Klett inversion algorithm. The Klett algorithm is a method in the field of lidar atmospheric detection, used to invert the vertical distribution of atmospheric extinction coefficient from the laser echo signal.

[0090] In this embodiment, constructing a three-dimensional atmospheric particle field based on the Klett algorithm includes the following steps:

[0091] S301, Calculate the distance correction signal based on the echo power data.

[0092] S302, Based on the distance correction signal, the extinction coefficient distributed with distance is obtained by the Klett inversion algorithm.

[0093] S303, The extinction coefficient is associated with the three-dimensional coordinates of the scene to construct the three-dimensional atmospheric particle field.

[0094] Specifically, to further explain the above steps, the expression for the echo power is as follows: Where t represents the echo time; Let t represent the echo power at time t; P represent the laser emission power; c represent the speed of light; Δt represent the pulse width; A represent the receiving area of ​​the lidar telescope; z(t) represent the detection distance, calculated by z(t) = (c*t) / 2. T(z(t)) represents the backscattering coefficient at a distance z(t); T(z(t)) represents the one-way atmospheric transmittance of the laser radar at a distance z(t).

[0095] Furthermore, in one embodiment of the present invention, the process of using the three-dimensional atmospheric particle field as the physical constraint environment for photon propagation, and using the target direct illumination component and the target spectral signal as the joint target, to perform an inverse simulation process based on Monte Carlo path tracing to obtain the target intrinsic material parameters under passive illumination and the target illumination parameters after interference removal includes the following steps:

[0096] S501, assign initial values ​​to the intrinsic material parameters and lighting parameters of the scene, wherein the initial values ​​are random values ​​or preset values.

[0097] In this embodiment, the scene includes a physical constraint environment and scene surface geometry and material. The physical constraint environment is the three-dimensional atmospheric particle field, while the scene surface geometry and material refer to the three-dimensional models of physical objects such as terrain, buildings, and vegetation, as well as the optical properties of their surfaces. The optical properties are the intrinsic material parameters.

[0098] S502 emits virtual sampling rays from the pixels of a virtual camera to simulate the scattering of light in the three-dimensional atmospheric particle field and its reflection and refraction with the scene surface.

[0099] S503 calculates the path contribution using the Monte Carlo integration algorithm to obtain the simulated spectral signal and direct illumination component.

[0100] In this embodiment, the expression for calculating the path contribution using the Monte Carlo integral algorithm is as follows: in, This represents the contribution of the i-th path to the final pixel value; Represents lighting parameters; Represents the chain multiplication operator; Represents intrinsic material parameters; P represents the volume scattering term determined by the three-dimensional atmospheric particle field.

[0101] In this embodiment, the contribution of each path is an independent Monte Carlo sample. According to the Monte Carlo integral theory, the physical observation value (pixel value) of a pixel can be determined by the arithmetic mean of these samples.

[0102] S504, using the difference between the simulated spectral signal and the target spectral signal, and the difference between the simulated direct illumination component and the target direct illumination component, as a loss function, the intrinsic material parameters and illumination parameters are updated through the gradient descent algorithm.

[0103] S505, continuously adjust the intrinsic material parameters and illumination parameters and repeatedly perform reverse path tracing. When the loss function converges to below a preset threshold, the current intrinsic material parameters and illumination parameters are determined as the intrinsic material parameters under passive illumination and the illumination parameters after interference removal.

[0104] In this embodiment, the preset threshold is a preset value, which aims to achieve a balance between image quality, computational efficiency and robustness. Those skilled in the art can determine the most suitable threshold for specific needs.

[0105] Furthermore, in one embodiment of the present invention, obtaining the simulated spectral signal and the simulated direct illumination component by calculating the average Monte Carlo integral of all path contributions includes the following steps:

[0106] S5031 Calculate the contribution of a single pixel to a single path, and add it to the total path contribution. At the same time, determine whether the path is a direct path. If so, add the contribution of the path to the direct path contribution.

[0107] S5032: After the path tracing of all rays of a single pixel is completed, the average value of the total path contribution is calculated to obtain the spectral signal pixel value of the pixel. At the same time, the average value of the direct path contribution is calculated to obtain the direct illumination component pixel value of the pixel.

[0108] S5033, integrate the spectral signal pixel values ​​and direct illumination component pixel values ​​of all pixels of the virtual camera to obtain the simulated spectral signal and simulated direct illumination component.

[0109] Specifically, in this embodiment, the condition for determining that the path is a direct path is: the path starts from the virtual camera, undergoes only one reflection or refraction on the scene surface, and directly hits the active lighting source defined by the lighting parameters.

[0110] Specifically, to further explain step S504, the expression for the loss function is as follows: Where R and L represent the intrinsic material parameters and lighting parameters to be solved, respectively; Represents the target spectral signal; This represents the simulated spectral signal obtained based on the current intrinsic material parameters R and illumination parameters L. Indicates the direct illumination component of the target. λ represents the direct illumination component obtained from the simulation based on the current intrinsic material parameters R; λ represents the regularization coefficient.

[0111] Furthermore, in one embodiment of the present invention, the step of synthesizing and outputting an enhanced image using a graphics renderer based on the target intrinsic material parameters and the target lighting parameters as lighting conditions includes the following steps:

[0112] S601, The intrinsic material parameters of the target are assigned to a virtual three-dimensional scene model to define the optical reflection properties of each surface in the scene model.

[0113] S602, Select the target illumination parameters as illumination conditions to provide the light source required for the enhanced image.

[0114] S603, in the graphics renderer, simulates the physical process of light rays emanating from the selected lighting conditions, illuminating the virtual 3D scene model, and entering the virtual camera after being reflected by the scene surface. The parameters of the virtual camera are consistent with the parameters of the camera that acquires the first and second images.

[0115] S604, calculate the radiance value received by each pixel on the imaging plane of the virtual camera, quantize these radiance values ​​into digital signals, and output the enhanced image.

[0116] Furthermore, in one embodiment of the present invention, a preset illumination model is also included. The illumination model can replace the target illumination parameters as illumination conditions. In the implementation of the above method in this embodiment, under certain extremely complex atmospheric conditions, or when the input data quality is poor, the retrieved target illumination parameters may be inaccurate or noisy. In this case, the preset illumination model can ensure that at least a visually reasonable and enhanced image can be output, thereby improving the robustness of the system. At the same time, it can meet the different needs of different application scenarios, improving flexibility and practicality.

[0117] Specifically, in this embodiment, the preset lighting model uses uniform parallel light as the basic lighting model and is configured with preset parameters including light source direction, light intensity and spectrum. The uniform parallel light model is one of the common light source models in computer graphics. Uniform parallel light refers to light rays with exactly the same intensity, which can simulate natural sunlight under clear weather. At the same time, this model can be flexibly configured through a set of parameters to adapt to different simulation needs.

[0118] Furthermore, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0119] To this end, the present invention provides a computer-readable storage medium storing a computer program thereon, the computer program being loaded by a processor to implement the functions of the various modules of the control center described above, specifically including the following steps:

[0120] Receive echo power data, a first image, and a second image from the UAV itself;

[0121] Based on the echo power data, the atmospheric extinction coefficient distribution is calculated using an atmospheric inversion algorithm, and a three-dimensional atmospheric particle field is constructed.

[0122] Calculate the pixel-level difference between the first image and the second image to obtain the direct illumination component of the target, and obtain the spectral signal of the second image as the target spectral signal;

[0123] Using the three-dimensional atmospheric particle field as the physical constraint environment and the direct illumination component and the target spectral signal as the joint target, a reverse simulation process based on Monte Carlo path tracing is performed to solve the intrinsic material parameters of the target under passive illumination and the target illumination parameters after interference removal.

[0124] Based on the intrinsic material parameters and lighting parameters of the target, an enhanced image is synthesized and output using a graphics renderer.

[0125] Figure 5 This is a schematic diagram of the structure of the computer device provided in an embodiment of the present invention, such as... Figure 5 As shown, the computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, enables the processor to perform the functions of the various modules of the control center, specifically including the following steps:

[0126] Receive echo power data, a first image, and a second image from the UAV itself;

[0127] Based on the echo power data, the atmospheric extinction coefficient distribution is calculated using an atmospheric inversion algorithm, and a three-dimensional atmospheric particle field is constructed.

[0128] Calculate the pixel-level difference between the first image and the second image to obtain the direct illumination component of the target, and obtain the spectral signal of the second image as the target spectral signal;

[0129] Using the three-dimensional atmospheric particle field as the physical constraint environment and the direct illumination component and the target spectral signal as the joint target, a reverse simulation process based on Monte Carlo path tracing is performed to solve the intrinsic material parameters of the target under passive illumination and the target illumination parameters after interference removal.

[0130] Based on the intrinsic material parameters and lighting parameters of the target, an enhanced image is synthesized and output using a graphics renderer.

[0131] The computer program instructions described in this application embodiment may be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, the instruction means being implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0132] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0133] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0134] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0135] Computer-readable media include both permanent and non-permanent, removable and non-removable media, which can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated communication signals and carrier waves.

[0136] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0137] The embodiments, implementation methods, and related technical features of this invention can be combined and substituted for each other without conflict.

[0138] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for enhancing images captured by low-altitude economic unmanned aerial vehicles (UAVs), characterized in that, include: A laser pulse is emitted and the echo signal of the laser pulse is acquired to obtain echo power data that varies with echo time, wherein the echo time is the time from the emission of the laser pulse to the receipt of the echo signal. Acquire the first image of active illumination and the second image of passive illumination within the same observation period; Based on the echo power data, the atmospheric extinction coefficient distribution along the path from the observation point to the scene point is calculated using an atmospheric inversion algorithm, and a three-dimensional atmospheric particle field is constructed based on the atmospheric extinction coefficient distribution. Calculate the pixel-level difference between the first image and the second image to obtain the target direct illumination component, and simultaneously acquire the spectral signal of the second image as the target spectral signal; Using the three-dimensional atmospheric particle field as the physical constraint environment for photon propagation, and the target direct illumination component and target spectral signal as the joint target, a reverse simulation process based on Monte Carlo path tracing is performed to obtain the target intrinsic material parameters under passive illumination and the target illumination parameters after interference removal. Based on the intrinsic material parameters of the target, the lighting parameters of the target are used as lighting conditions, and an enhanced image is synthesized and output through a graphics renderer. The process of obtaining the intrinsic material parameters of the target under passive illumination and the target illumination parameters after interference removal includes: Assign initial values ​​to the intrinsic material parameters and lighting parameters of the scene, wherein the initial values ​​are random values ​​or preset values; Virtual sampling rays are emitted from the pixels of a virtual camera to simulate the scattering of light in the three-dimensional atmospheric particle field and its reflection and refraction with the scene surface; The path contribution is calculated using the Monte Carlo integral algorithm to obtain the simulated spectral signal and direct illumination component; The difference between the simulated spectral signal and the target spectral signal, and the difference between the simulated direct illumination component and the target direct illumination component, are used together as the loss function to update the intrinsic material parameters and illumination parameters through the gradient descent algorithm. The intrinsic material parameters and illumination parameters are continuously adjusted. When the loss function converges to below a preset threshold, the current intrinsic material parameters and illumination parameters are determined as the intrinsic material parameters under passive illumination and the illumination parameters after interference removal. The calculation of path contribution using the Monte Carlo integration algorithm to obtain the simulated spectral signal and direct illumination component includes: Calculate the contribution of a single pixel and a single path, and add them to the total path contribution. At the same time, determine whether the path is a direct path. If it is, add the contribution of the path to the direct path contribution. The condition for determining that the path is a direct path is: the path starts from the virtual camera, undergoes only one reflection or refraction on the scene surface, and directly hits the active lighting source defined by the lighting parameters. After the path tracing of all rays in a single pixel is completed, the average value of the total path contribution is calculated to obtain the spectral signal pixel value of that pixel. At the same time, the average value of the direct path contribution is calculated to obtain the direct illumination component pixel value of that pixel. By integrating the spectral signal pixel values ​​and direct illumination component pixel values ​​of all pixels in the virtual camera, a simulated spectral signal and a simulated direct illumination component are obtained.

2. The image enhancement method based on low-altitude economic UAV photography according to claim 1, characterized in that, The atmospheric inversion algorithm is the Klett inversion algorithm, and the construction of the three-dimensional atmospheric particle field includes: Calculate the distance correction signal based on the echo power data; Based on the distance correction signal, the extinction coefficient distributed with distance is obtained by the Klett inversion algorithm; The extinction coefficient is associated with the three-dimensional coordinates of the scene to construct the three-dimensional atmospheric particle field.

3. The image enhancement method based on low-altitude economic UAV photography according to claim 2, characterized in that, The expression for the echo power is: Where t represents the echo time; Let t represent the echo power at time t; P represent the laser emission power; c represent the speed of light; Δt represent the pulse width; A represent the receiving area of ​​the lidar telescope; z(t) represent the detection distance, calculated by z(t) = (c*t) / 2. T(z(t)) represents the backscattering coefficient at a distance z(t); T(z(t)) represents the one-way atmospheric transmittance of the laser radar at a distance z(t).

4. The image enhancement method based on low-altitude economic UAV photography according to claim 1, characterized in that, The expression for the loss function is: Where R and L represent the intrinsic material parameters and lighting parameters to be solved, respectively; Represents the target spectral signal; This represents the simulated spectral signal obtained based on the current intrinsic material parameters R and illumination parameters L. Indicates the direct illumination component of the target. λ represents the direct illumination component obtained from the simulation based on the current intrinsic material parameters R; λ represents the regularization coefficient.

5. The image enhancement method based on low-altitude economic UAV photography according to claim 1, characterized in that, The step of synthesizing and outputting an enhanced image using a graphics renderer based on the target intrinsic material parameters and the target lighting parameters as lighting conditions includes: The intrinsic material parameters of the target are assigned to a virtual three-dimensional scene model to define the optical reflection properties of each surface in the scene model. The target illumination parameters are selected as illumination conditions to provide the light source required for the enhanced image; In a graphics renderer, the physical process of simulating light rays emanating from the selected lighting conditions, illuminating the virtual 3D scene model, and entering the virtual camera after being reflected by the scene surface is simulated. The parameters of the virtual camera are consistent with the parameters of the camera that acquires the first and second images. Calculate the radiance value received by each pixel on the imaging plane of the virtual camera, quantize these radiance values ​​into digital signals, and output the enhanced image.

6. The image enhancement method based on low-altitude economic UAV photography according to claim 1, characterized in that, It also includes a preset lighting model, which can be used as a lighting condition to replace the target lighting parameters; The preset lighting model uses uniform parallel light as the basic lighting model and is configured with preset parameters including light source direction, light intensity and spectrum.

Images