AI model construction system and method for unmanned aerial vehicle image recognition

Abstract

The invention relates to the field of unmanned aerial vehicle control, in particular to an AI model construction system and method for unmanned aerial vehicle image recognition, and the system comprises a positioning entity module, an angular point training module, a distortion correction module, an image fusion module and a color restoration module. The angular point training module is used for training an AI model and inputting original calibration parameters, the distortion correction module is used for establishing a calibration model and carrying out image preliminary calibration, the image fusion module is used for shooting an image set and generating a fused image, and the color restoration module is used for carrying out color correction on an original image. The space precision of the image is optimized, the stability and availability of the image under different environment conditions are improved, meanwhile, the overlapping error between the images is eliminated, the accuracy of image data is ensured, and subsequent analysis and application of the unmanned aerial vehicle image are facilitated.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle control, and in particular to an AI model building system and method for unmanned aerial vehicle image recognition. Background Art

[0002] Drone images refer to images or video data taken from the air by cameras or other sensors carried by unmanned aerial vehicles (UAVs), which can realize functions such as agricultural land analysis, environmental monitoring, and real-time early warning. To accurately capture images, the drone must hover above the target and shoot downward through the camera. During this process, it is necessary to calibrate the shooting through equipment such as gyroscopes.

[0003] Since drones operate at high altitudes, the pictures taken by the digital cameras they carry are prone to nonlinear optical distortion, causing some pixels to shift in position, resulting in imaging distortion and fisheye effect, making the image unable to reflect the actual state of the object well.

[0004] Image restoration by building an AI model is one of the means to solve this problem. However, since the only source of information for high-altitude drones is the camera itself, there is a lack of objective positioning points when building the AI ​​model, which makes model construction difficult.

[0005] In addition, due to shooting problems such as exposure and focus, the color of drone images will also be offset. Image color restoration requires multiple focus and aperture adjustments, which affects the drone's shooting efficiency and cannot well reflect the true color of the image. Summary of the invention

[0006] The purpose of the present invention is to provide an AI model building system and method for drone image recognition to solve the problems raised in the above background technology.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: an AI model construction system for drone image recognition, comprising: a positioning entity module, a corner point training module, a distortion correction module, an image fusion module and a color restoration module; The positioning entity module is used to set a retractable positioning plumb bob at a fixed position on the drone, and before starting to shoot, the drone is hovered above the object and the plumb bob is released. The camera shoots an image and records the position of the plumb bob in the captured image. The field of view of the image is determined according to the flight altitude of the drone, the plumb bob hoisting length and the position of the plumb bob in the image. The corner point training module is used to take the pixel at the center of the plumb bob as the corner point, extract pixels within a fixed range outside the corner point, make the extracted pixel range larger than the plumb bob range, smooth the pixels beyond the plumb bob range by a mean filter method, and store them in a training library together with the plumb bob pixels. The AI ​​model learns the content in the training library, and inputs the original data of the plumb bob, the field of view of the picture, and the flight altitude of the drone as exogenous parameters into the AI ​​model. The trained AI model outputs the image principal point offset value of the camera, and enters the offset value into the database as the original calibration parameter; The distortion correction module is used for, after the plumb bob is put away, the camera takes an image again, adopts a weighted algorithm to identify the corner point in the image, takes the corner point as the center, introduces radial deformation and tangential deformation into the bundle method regional block adjustment, establishes a distortion correction mathematical model, calculates the adjustment value of the corner point by using the distortion correction mathematical model, establishes a calibration model according to the adjusted corner point, maps the transition pixels between the corner points in the image into the calibration model, and completes the preliminary calibration of the image after mapping; The image fusion module is used to control the drone to fly in a preset direction at the same altitude and continuously take images to generate an image set until the images taken in the image set contain the features of all the original images, fuse the image set using the trained AI model to obtain a fused image, and use the nearest neighbor method to determine the position of the original image in the fused image; The color restoration module is used to calculate the contrast between the original image and the edge image in different areas, calculate the adjustment value according to the contrast of adjacent areas, and use the adjustment value as a parameter to adjust the color gradient vector between pixels in the original image, thereby performing color correction on the original image and outputting an image after shape calibration and color correction processing.

[0008] Further, the positioning entity module includes: a plumb bob hoisting unit and a camera unit; The plumb bob hoisting unit is used to set a plumb bob and a hoisting device at a fixed position under the drone camera, so that the plumb bob can be raised or lowered according to the control command, and the camera is kept from being blocked when the plumb bob is raised; The camera unit is used to capture images of the scenery below the drone, obtain original images, and transmit the original images to a data processing device.

[0009] Furthermore, the corner point training module includes: an image processing unit, an AI modeling unit and a parameter input unit; The image processing unit is used for pixel recognition, pixel extraction and pixel mean filtering and smoothing to remove noise in the image; The AI ​​modeling unit is used to introduce an AI model to learn the original image and provide exogenous parameters, and the AI ​​model includes: an SSD model, a DCGAN model, a Mask R-CNN model and a SimCLR self-supervised learning model; The parameter input unit is used to calculate the principal point offset value of the camera, and generate and input the image calibration parameters.

[0010] Further, the distortion correction module includes: a feature extraction unit, a model adjustment unit and a pixel mapping unit; The feature extraction unit is used to identify corner points in the image by using a weighted algorithm, wherein the weighted algorithm includes: a SUSAN feature extraction algorithm, a Harris feature extraction algorithm, and a SIFT transformation algorithm; The model adjustment unit is used to construct a beam method regional network with the corner point as the center, calculate the calibration amount and generate a calibration model; The pixel mapping unit is used to map non-corner pixels in the original image to the calibration model to correct image deviation.

[0011] Furthermore, the image fusion module includes: an image storage unit and an intelligent fusion unit; The image storage unit is used to control the drone to capture images, store all captured images, and analyze the coverage of the images; The intelligent fusion unit is used to identify and splice all images in the image set to form a fused image with a higher number of pixels than the original image.

[0012] Furthermore, the color restoration module includes: a superposition contrast unit and a gradient restoration unit; The superposition contrast unit is used to divide the pixel areas of the fused image and the original image, and calculate the contrast of the image in each area; The gradient restoration unit is used to calculate the contrast adjustment amount, adjust the pixel colors in the original image, and output the restored image.

[0013] A method for building an AI model for drone image recognition, comprising the following steps: Step S1. A plumb bob and a hoisting device are set under the drone camera, the drone is hovered above the object and the plumb bob is released, an image of the scene below the drone is captured, a calibration image is obtained, and the sway parameter and maximum field of view of the drone are determined according to the plumb bob position in the adjacent calibration image and the drone height; Step S2. Take the pixel at the center of the plumb bob as the corner point, extract the pixels outside the corner point, smooth the grayscale and store them in the training library, use the training library as the endogenous data, and the plumb bob data, the height of the drone and the field of view as the exogenous parameters to train the AI ​​model, so that the AI ​​model outputs the radial deformation and tangential deformation of the image; Step S3. put away the plumb bob, shoot the original image with the camera, use the weighted algorithm to identify the corner points in the original image, build the bundle method regional block adjustment with the corner points as the center, calculate the calibration amount and generate the calibration model, provide the non-corner point pixels in the original image for calibration model adjustment, and correct the nonlinear distortion of the digital camera lens when the drone is unstable in flight; Step S4. Make the drone fly in a preset direction and shoot an image set until the image set contains the features of all the original images, identify and stitch all the images in the image set to obtain a fused image; Step S5. Calculate the contrast between the fused image and each image area in the original image, adjust the contrast between pixels in each area of ​​the original image to make it consistent with the fused image, and repair the lens color distortion caused by water droplets or fog.

[0014] Further, step S1 includes: Step S11. A plumb bob and a hoisting device are arranged below the camera of the drone, so that the plumb bob can be raised or lowered according to the control command, and the camera is not blocked when the plumb bob is raised or completely lowered, the plumb bob has an upper surface and a lower surface in a regular circular or elliptical shape, the upper surface is made of a reflective material or a light-absorbing material, and the length of the hoisting line is greater than the vertical distance between the plumb bob and the camera; Step S12. The UAV arrives above the shooting point and hovers at a fixed height at the shooting point. After the plumb bob is completely released, a calibration image is taken, and the upper surface projection of the plumb bob in the calibration image is obtained. Images are taken continuously, and the shaking direction and amplitude of the UAV are determined according to the change of the plumb bob center point in adjacent images. The UAV shaking parameters are input into a digital camera to obtain non-orthogonal distortion coefficients and non-square ratio coefficients. Step S13: Calculate the visual field range by: ; Among them, α represents the angle between the edge of the image and the central axis of the image, D α represents the widest field of view of the image in the α direction, S1 and S2 represent the surface area of ​​the plumb bob in the calibration image and the actual plumb bob respectively, P represents the proportion of the plumb bob surface area in the calibration image to the overall image, H1 and H2 represent the length of the plumb bob’s hoisting line and the UAV’s flight altitude respectively, and cosθ represents the focus offset angle of the plumb bob’s surface in the α direction.

[0015] Further, step S2 includes: Step S21. Identify the center point of the upper surface of the plumb bob, take the center point as the corner point, extract pixels within a range of n·n outside the corner point, where n is the side length of the extracted pixel area, and keep the extracted pixel range larger than the plumb bob range. Smooth the pixels beyond the plumb bob range by a mean filter method to correct the pixel color. Step S22. The extracted pixels are stored in the training library, the data in the AI ​​model learning training library is introduced, the plumb bob data, the height of the drone and the field of view are input as exogenous parameters into the AI ​​model for combined training, and the trained AI model is obtained. The selected AI models include: SSD model, DCGAN model, Mask R-CNN model and SimCLR self-supervised learning model; Step S23: Instruct the trained AI model to analyze the principal point offset value of the camera, and enter the offset value into the database as the original calibration parameter.

[0016] Further, step S3 includes: Step S31. After the plumb bob is recovered, the camera retakes the image and stores it as the original image. A weighted algorithm is used to identify corner points in the original image. The corner points represent pixels in the image whose color or texture difference with adjacent pixels is greater than a threshold. The weighted algorithm includes: SUSAN feature extraction algorithm, Harris feature extraction algorithm and SIFT transformation algorithm. Step S32. Construct a bundle method block adjustment with the corner point as the center, calculate the calibration amount and generate a calibration model. The calibration amount calculation method is: ; Where xe and ye represent the horizontal and vertical calibration values, respectively; x0 and y0 represent the horizontal and vertical principal point offset values, respectively; Dx and Dy represent the widest field of view in the horizontal and vertical directions, respectively; x and y represent the pixel coordinates; a represents the non-square scale coefficient; and b represents the non-orthogonal distortion coefficient, which is obtained from the camera parameters; Step S33. Adjust each corner point according to the calibration amount, generate a calibration model, map the non-corner point pixels in the original image to the calibration model, and output the calibration result.

[0017] Further, step S4 includes: Step S41. Keep the drone at the same altitude, fly in each preset direction, and continue to shoot the scene below, so that the captured image contains at least a part of the original image, store the captured image and form an image set, until the image set contains all the features of the original image; Step S42. Use the trained AI model to fuse the image set to form a fused image with a higher number of pixels than the original image, and use the nearest neighbor method to determine the position of the original image in the fused image.

[0018] Further, step S5 includes: Step S51. Divide the portion of the edge image containing the original image into a preset number of sub-regions, and use image recognition software to calculate the average contrast in each sub-region; Step S52. Calculate the average contrast of the adjacent areas of each sub-area, subtract the average contrast in the sub-area from the average contrast of the adjacent areas to obtain an adjustment value, process the color gradient of each pixel in the sub-area according to the adjustment value, complete color calibration, and output the image after shape calibration and color correction.

[0019] Compared with the prior art, the beneficial effects achieved by the present invention are: The present invention sets a retractable positioning plumb bob on the drone and records the position of the plumb bob in the captured image, thereby determining the field of view of the picture, and trains the AI ​​model with the plumb bob as the corner point, enters the principal point offset value into the original calibration parameters, adds objective positioning points, simplifies the AI ​​training process, helps adjust the drone position, and improves the spatial accuracy of the image.

[0020] The present invention continuously shoots images, adopts a weighted algorithm to identify corner points in the images, introduces radial deformation and tangential deformation into bundle method regional block adjustment with the corner points as the center, establishes a distortion correction mathematical model, maps each pixel in the image into the model, thereby completing the initial calibration of the image, which can help eliminate image distortion, ensure the consistency between different sensor data, facilitate subsequent analysis and application of the image, and improve the stability and availability of the image under different environmental conditions.

[0021] The present invention captures images at different positions and forms an image set, so that the image set contains all the features of the original image, adjusts the pixel gradient vector in the original image according to the contrast, performs color correction on the image, eliminates the overlapping errors between images, makes it closer to the real scene, improves the spatial accuracy of the image, and ensures the accuracy of the image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 It is a structural schematic diagram of an AI model building system for drone image recognition of the present invention; Figure 2 It is a schematic diagram of the steps of a method for building an AI model for drone image recognition in the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1 ,The present invention provides a technical solution: an AI model building system for drone image recognition, comprising: a positioning entity module, a corner point training module, a distortion correction module, an image fusion module and a color restoration module; The positioning entity module is used to set a retractable positioning plumb bob at a fixed position on the drone, and before starting to shoot, the drone is hovered above the object and the plumb bob is released. The camera shoots an image and records the position of the plumb bob in the captured image. The field of view of the image is determined according to the flight altitude of the drone, the plumb bob hoisting length and the position of the plumb bob in the image. The positioning entity module includes: a plumb bob hoisting unit and a camera unit; The plumb bob hoisting unit is used to set a plumb bob and a hoisting device at a fixed position under the drone camera, so that the plumb bob can be raised or lowered according to the control command, and the camera is kept from being blocked when the plumb bob is raised; The camera unit is used to capture images of the scenery below the drone, obtain original images, and transmit the original images to a data processing device.

[0025] The corner point training module is used to take the pixel at the center of the plumb bob as the corner point, extract pixels within a fixed range outside the corner point, make the extracted pixel range larger than the plumb bob range, smooth the pixels beyond the plumb bob range by a mean filter method, and store them in a training library together with the plumb bob pixels. The AI ​​model learns the content in the training library, and inputs the original data of the plumb bob, the field of view of the picture, and the flight altitude of the drone as exogenous parameters into the AI ​​model. The trained AI model outputs the image principal point offset value of the camera, and enters the offset value into the database as the original calibration parameter; The corner point training module includes: an image processing unit, an AI modeling unit and a parameter input unit; The image processing unit is used for pixel recognition, pixel extraction and pixel mean filtering and smoothing to remove noise in the image; The AI ​​modeling unit is used to introduce an AI model to learn the original image and provide exogenous parameters, and the AI ​​model includes: an SSD model, a DCGAN model, a Mask R-CNN model and a SimCLR self-supervised learning model; The parameter input unit is used to calculate the principal point offset value of the camera, and generate and input the image calibration parameters.

[0026] The distortion correction module is used for, after the plumb bob is put away, the camera takes an image again, adopts a weighted algorithm to identify the corner point in the image, takes the corner point as the center, introduces radial deformation and tangential deformation into the bundle method regional block adjustment, establishes a distortion correction mathematical model, calculates the adjustment value of the corner point by using the distortion correction mathematical model, establishes a calibration model according to the adjusted corner point, maps the transition pixels between the corner points in the image into the calibration model, and completes the preliminary calibration of the image after mapping; The distortion correction module includes: a feature extraction unit, a model adjustment unit and a pixel mapping unit; The feature extraction unit is used to identify corner points in the image by using a weighted algorithm, wherein the weighted algorithm includes: a SUSAN feature extraction algorithm, a Harris feature extraction algorithm, and a SIFT transformation algorithm; The model adjustment unit is used to construct a beam method regional network with the corner point as the center, calculate the calibration amount and generate a calibration model; The pixel mapping unit is used to map non-corner pixels in the original image to the calibration model to correct image deviation.

[0027] The image fusion module is used to control the drone to fly in a preset direction at the same altitude and continuously take images to generate an image set until the images taken in the image set contain the features of all the original images, fuse the image set using the trained AI model to obtain a fused image, and use the nearest neighbor method to determine the position of the original image in the fused image; The image fusion module includes: an image storage unit and an intelligent fusion unit; The image storage unit is used to control the drone to capture images, store all captured images, and analyze the coverage of the images; The intelligent fusion unit is used to identify and splice all images in the image set to form a fused image with a higher number of pixels than the original image.

[0028] The color restoration module is used to calculate the contrast between the original image and the edge image in different areas, calculate the adjustment value according to the contrast of adjacent areas, and use the adjustment value as a parameter to adjust the color gradient vector between pixels in the original image, thereby performing color correction on the original image and outputting an image after shape calibration and color correction processing.

[0029] The color restoration module includes: a superposition contrast unit and a gradient restoration unit; The superposition contrast unit is used to divide the pixel areas of the fused image and the original image, and calculate the contrast of the image in each area; The gradient restoration unit is used to calculate the contrast adjustment amount, adjust the pixel colors in the original image, and output the restored image.

[0030] like Figure 2As shown, a method for building an AI model for drone image recognition includes the following steps: Step S1. A plumb bob and a hoisting device are set at a fixed position under the drone camera. Before shooting, the drone is hovered above the object and the plumb bob is released. An image of the scene below the drone is captured to obtain a calibration image. The camera's field of view is determined based on the plumb bob position, drone height, and plumb bob data in the calibration image. Step S1 includes: Step S11. A plumb bob and a hoisting device are arranged below the camera of the drone, so that the plumb bob can be raised or lowered according to the control command, and the camera is not blocked when the plumb bob is raised or completely lowered, the plumb bob has an upper surface and a lower surface in a regular circular or elliptical shape, the upper surface is made of a reflective material or a light-absorbing material, and the length of the hoisting line is greater than the vertical distance between the plumb bob and the camera; Step S12. The drone arrives above the shooting point and hovers at a fixed height at the shooting point. After the plumb bob is completely released, a calibration image is taken, and the upper surface projection of the plumb bob in the calibration image is obtained. The field of view is calculated according to the plumb bob position, the drone height, and the plumb bob data. The plumb bob data includes the upper surface area of ​​the plumb bob, the length of the hoisting line, and the upper surface focus. The calculation method is: ; Among them, α represents the angle between the edge of the image and the central axis of the image, D α represents the widest field of view of the image in the α direction, S1 and S2 represent the surface area of ​​the plumb bob in the calibration image and the actual plumb bob respectively, P represents the proportion of the plumb bob surface area in the calibration image to the overall image, H1 and H2 represent the length of the plumb bob’s hoisting line and the UAV’s flight altitude respectively, and cosθ represents the focus offset angle of the plumb bob’s surface in the α direction.

[0031] Step S2. Take the pixel at the center of the plumb bob as the corner point, extract the pixels within a fixed range outside the corner point, perform grayscale smoothing with the mean filter method, and store them in the training library. Use the training library as the endogenous data, and use the plumb bob data, drone height, and field of view as the exogenous parameters to train the AI ​​model, so that the AI ​​model outputs the calibration parameters of the image; Step S2 includes: Step S21. Identify the center point of the upper surface of the plumb bob, take the center point as the corner point, extract pixels within a range of n·n outside the corner point, where n is the side length of the extracted pixel area, and keep the extracted pixel range larger than the plumb bob range. Smooth the pixels beyond the plumb bob range by a mean filter method to correct the pixel color. Step S22. The extracted pixels are stored in the training library, the data in the AI ​​model learning training library is introduced, the plumb bob data, the height of the drone and the field of view are input as exogenous parameters into the AI ​​model for combined training, and the trained AI model is obtained. The selected AI models include: SSD model, DCGAN model, Mask R-CNN model and SimCLR self-supervised learning model; Step S23: Instruct the trained AI model to analyze the principal point offset value of the camera, and enter the offset value into the database as the original calibration parameter.

[0032] Step S3. Put away the plumb bob, shoot the original image with the camera, use the weighted algorithm to identify the corner points in the original image, construct the bundle method regional block adjustment with the corner points as the center, calculate the calibration amount and generate the calibration model, map the non-corner point pixels in the original image to the calibration model, and correct the shape deviation; Step S3 includes: Step S31. After the plumb bob is recovered, the camera retakes the image and stores it as the original image. A weighted algorithm is used to identify corner points in the original image. The corner points represent pixels in the image whose color or texture difference with adjacent pixels is greater than a threshold. The weighted algorithm includes: SUSAN feature extraction algorithm, Harris feature extraction algorithm and SIFT transformation algorithm. Step S32. Construct a bundle method block adjustment with the corner point as the center, calculate the calibration amount and generate a calibration model. The calibration amount calculation method is: ; Where xe and ye represent the horizontal and vertical calibration values, respectively; x0 and y0 represent the horizontal and vertical principal point offset values, respectively; Dx and Dy represent the widest field of view in the horizontal and vertical directions, respectively; x and y represent the pixel coordinates; a represents the non-square scale coefficient; and b represents the non-orthogonal distortion coefficient, which is obtained from the camera parameters; Step S33. Adjust each corner point according to the calibration amount, generate a calibration model, map the non-corner point pixels in the original image to the calibration model, and output the calibration result.

[0033] Step S4. Make the drone fly in all directions and take pictures of the image set until the image set contains the features of all the original images, identify and stitch all the images in the image set to obtain a fused image; Step S4 includes: Step S41. Keep the drone at the same altitude, fly in each preset direction, and continue to shoot the scene below, so that the captured image contains at least a part of the original image, store the captured image and form an image set, until the image set contains all the features of the original image; Step S42. Use the trained AI model to fuse the image set to form a fused image with a higher number of pixels than the original image, and use the nearest neighbor method to determine the position of the original image in the fused image.

[0034] Step S5. Calculate the contrast of each image area in the fused image and the original image, calculate the difference in contrast between each area and all adjacent areas to obtain an adjustment value, adjust the color gradient between pixels in each area of ​​the original image, and output the original image after color restoration.

[0035] Step S5 includes: Step S51. Divide the portion of the edge image containing the original image into a preset number of sub-regions, and use image recognition software to calculate the average contrast in each sub-region; Step S52. Calculate the average contrast of the adjacent areas of each sub-area, subtract the average contrast in the sub-area from the average contrast of the adjacent areas to obtain an adjustment value, process the color gradient of each pixel in the sub-area according to the adjustment value, complete color calibration, and output the image after shape calibration and color correction.

[0036] Example: A circular plumb bob with a radius of 5 cm is released from a drone. The length of the hoisting line is 10 m. The drone is hovering at a height of 100 m. In the calibration image, the plumb bob has a radius of 1 cm and occupies 20% of the entire image area. The horizontal and vertical offset angles are 2° and 3° respectively. The image resolution is 1000*1000, so the field of view is approximately 49 cm. 2 , after recording the data, put away the plumb bob and calculate that the horizontal image principal point offset value between unit pixels is 0.007sin2°cm, and the vertical image principal point offset value is 0.007sin3°cm; The image was taken by a drone. It is known that the non-orthogonal distortion coefficient of the digital camera caused by the shaking of the drone is 0.05, and the non-square ratio coefficient is 0.01. The pixel at the exact center of the image is used as the corner point to correct the nonlinear distortion in the original image. Taking the pixel point (-5,10) as an example, the horizontal and vertical calibration amounts in the lens are -0.014 cm and 0.026 cm respectively. The drone is kept at the same altitude for extended shooting, and the captured images are fused. The fused images are used for color restoration. Taking the pixels (-5,10) and (-5,9) as an example, the contrast between the pixels (-5,10) and (-5,9) in the fused image is -0.01, and the contrast in the captured image is -0.008. The color gradient between the pixels is adjusted to -0.002, and the image is output after all pixels are processed to complete the camera distortion correction and color correction.

[0037] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0038] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for constructing an AI model for drone image recognition, characterized in that: The method comprises the following steps: Step S1. A plumb bob and a hoisting device are set under the drone camera, the drone is hovered above the object and the plumb bob is released, an image of the scene below the drone is captured, a calibration image is obtained, and the sway parameter and maximum field of view of the drone are determined according to the plumb bob position in the adjacent calibration image and the drone height; Step S2. Take the pixel at the center of the plumb bob as the corner point, extract the pixels outside the corner point, smooth the grayscale and store them in the training library, use the training library as the endogenous data, and the plumb bob data, the height of the drone and the field of view as the exogenous parameters to train the AI ​​model, so that the AI ​​model outputs the radial deformation and tangential deformation of the image; Step S3. put away the plumb bob, shoot the original image with the camera, construct the bundle method regional block adjustment with the center of the image as the corner point, calculate the calibration amount and generate the calibration model, provide the non-corner pixels in the original image to the calibration model for adjustment, and correct the nonlinear distortion of the digital camera lens when the drone is unstable in flight; Step S4. Make the drone fly in a preset direction and shoot an image set until the image set contains the features of all the original images, identify and stitch all the images in the image set to obtain a fused image; Step S5. Calculate the contrast between the fused image and each image area in the original image, adjust the contrast between pixels in each area of ​​the original image to make it consistent with the fused image, and repair the lens color distortion caused by water droplets or fog.

2. The AI ​​model construction method for drone image recognition according to claim 1, characterized in that: Step S1 includes: Step S11. A plumb bob and a hoisting device are arranged below the camera of the drone, so that the plumb bob is raised or lowered according to the control command, and the camera is not blocked when the plumb bob is raised or completely lowered, the plumb bob has an upper surface and a lower surface in a regular circular or elliptical shape, the upper surface is made of a reflective material or a light-absorbing material, and the length of the hoisting line is greater than the vertical distance between the plumb bob and the camera; Step S12. The UAV arrives above the shooting point and hovers at a fixed height at the shooting point. After the plumb bob is completely released, a calibration image is taken, and the upper surface projection of the plumb bob in the calibration image is obtained. Images are taken continuously, and the shaking direction and amplitude of the UAV are determined according to the change of the plumb bob center point in adjacent images. The UAV shaking parameters are input into a digital camera to obtain non-orthogonal distortion coefficients and non-square ratio coefficients. Step S13: Calculate the visual field range by: ； Among them, α represents the angle between the edge of the image and the central axis of the image, D α represents the widest field of view of the image in the α direction, S1 and S2 represent the surface area of ​​the plumb bob in the calibration image and the actual plumb bob respectively, P represents the proportion of the plumb bob surface area in the calibration image to the overall image, H1 and H2 represent the length of the plumb bob’s hoisting line and the UAV’s flight altitude respectively, and cosθ represents the focus offset angle of the plumb bob’s surface in the α direction.

3. The AI ​​model construction method for drone image recognition according to claim 2 is characterized in that: Step S2 includes: Step S21. Identify the center point of the upper surface of the plumb bob, take the center point as the corner point, extract pixels within a range of n·n outside the corner point, where n is the side length of the extracted pixel area, and keep the extracted pixel range larger than the plumb bob range. Smooth the pixels beyond the plumb bob range by a mean filter method to correct the pixel color. Step S22. The extracted pixels are stored in the training library, the data in the AI ​​model learning training library is introduced, the plumb bob data, the height of the drone and the field of view are input as exogenous parameters into the AI ​​model for combined training, and the trained AI model is obtained. The selected AI models include: SSD model, DCGAN model, Mask R-CNN model and SimCLR self-supervised learning model; Step S23: Instruct the trained AI model to analyze the principal point offset value of the camera, and enter the offset value into the database as the original calibration parameter.

4. The AI ​​model construction method for drone image recognition according to claim 3 is characterized in that: Step S3 includes: Step S31. After the plumb bob is recovered, the camera retakes the image and stores it as the original image. A weighted algorithm is used to identify corner points in the original image. The corner points represent pixels in the image whose color or texture difference with adjacent pixels is greater than a threshold. The weighted algorithm includes: SUSAN feature extraction algorithm, Harris feature extraction algorithm and SIFT transformation algorithm. Step S32. Construct a bundle method block adjustment with the corner point as the center, calculate the calibration amount and generate a calibration model. The calibration amount calculation method is: ； Where xe and ye represent the horizontal and vertical calibration values, respectively; x0 and y0 represent the horizontal and vertical principal point offset values, respectively; Dx and Dy represent the widest field of view in the horizontal and vertical directions, respectively; x and y represent the pixel coordinates; a represents the non-square scale coefficient; and b represents the non-orthogonal distortion coefficient, which is obtained from the camera parameters; Step S33. Adjust each corner point according to the calibration amount, generate a calibration model, map the non-corner point pixels in the original image to the calibration model, and output the calibration result.

5. The AI ​​model construction method for drone image recognition according to claim 4 is characterized in that: Step S4 includes: Step S41. Keep the drone at the same altitude, fly in each preset direction, and continue to shoot the scene below, so that the captured image contains at least a part of the original image, store the captured image and form an image set, until the image set contains all the features of the original image; Step S42. Using the trained AI model to fuse the image set, a fused image with a higher number of pixels than the original image is formed, and the position of the original image in the fused image is determined by using the nearest neighbor method; Step S5 includes: Step S51. Divide the portion of the edge image containing the original image into a preset number of sub-regions, and use image recognition software to calculate the average contrast in each sub-region; Step S52. Calculate the average contrast of the adjacent areas of each sub-area, subtract the average contrast in the sub-area from the average contrast of the adjacent areas to obtain an adjustment value, process the color gradient of each pixel in the sub-area according to the adjustment value, complete color calibration, and output the image after shape calibration and color correction.

6. An AI model building system for drone image recognition, characterized in that: The system includes the following modules: entity positioning module, corner point training module, distortion correction module, image fusion module and color restoration module; The positioning entity module is used to set a retractable positioning plumb bob at a fixed position on the drone, and before starting to shoot, the drone is hovered above the object and the plumb bob is released. The camera shoots an image and records the position of the plumb bob in the captured image. The field of view of the image is determined according to the flight altitude of the drone, the plumb bob hoisting length and the position of the plumb bob in the image. The corner point training module is used to take the pixel at the center of the plumb bob as the corner point, extract pixels within a fixed range outside the corner point, make the extracted pixel range larger than the plumb bob range, smooth the pixels beyond the plumb bob range by a mean filter method, and store them in a training library together with the plumb bob pixels. The AI ​​model learns the content in the training library, and inputs the original data of the plumb bob, the field of view of the picture, and the flight altitude of the drone as exogenous parameters into the AI ​​model. The trained AI model outputs the image principal point offset value of the camera, and enters the offset value into the database as the original calibration parameter; The distortion correction module is used for, after the plumb bob is put away, the camera takes an image again, adopts a weighted algorithm to identify the corner point in the image, takes the corner point as the center, introduces radial deformation and tangential deformation into the bundle method regional block adjustment, establishes a distortion correction mathematical model, calculates the adjustment value of the corner point by using the distortion correction mathematical model, establishes a calibration model according to the adjusted corner point, maps the transition pixels between the corner points in the image into the calibration model, and completes the preliminary calibration of the image after mapping; The image fusion module is used to control the drone to fly in a preset direction at the same altitude and continuously take images to generate an image set until the images taken in the image set contain the features of all the original images, fuse the image set using the trained AI model to obtain a fused image, and use the nearest neighbor method to determine the position of the original image in the fused image; The color restoration module is used to calculate the contrast between the original image and the edge image in different areas, calculate the adjustment value according to the contrast of adjacent areas, and use the adjustment value as a parameter to adjust the color gradient vector between pixels in the original image, thereby performing color correction on the original image and outputting an image after shape calibration and color correction processing.

7. The AI ​​model building system for drone image recognition according to claim 6, characterized in that: The positioning entity module includes: a plumb bob hoisting unit and a camera unit; The plumb bob hoisting unit is used to set a plumb bob and a hoisting device at a fixed position under the drone camera, so that the plumb bob is raised or lowered according to the control command, and the camera is kept from being blocked when the plumb bob is raised; The camera unit is used to capture images of the scenery below the drone, obtain original images, and transmit the original images to a data processing device.

8. The AI ​​model building system for drone image recognition according to claim 7, characterized in that: The corner point training module includes: an image processing unit, an AI modeling unit and a parameter input unit; The image processing unit is used for pixel recognition, pixel extraction and pixel mean filtering and smoothing to remove noise in the image; The AI ​​modeling unit is used to introduce an AI model to learn the original image and provide exogenous parameters, and the AI ​​model includes: an SSD model, a DCGAN model, a Mask R-CNN model and a SimCLR self-supervised learning model; The parameter input unit is used to calculate the principal point offset value of the camera, and generate and input the image calibration parameters.

9. The AI ​​model building system for drone image recognition according to claim 8, characterized in that: The distortion correction module includes: a feature extraction unit, a model adjustment unit and a pixel mapping unit; The feature extraction unit is used to identify corner points in an image by using a weighted algorithm, wherein the weighted algorithm includes: a SUSAN feature extraction algorithm, a Harris feature extraction algorithm, and a SIFT transformation algorithm; The model adjustment unit is used to construct a beam method regional network with the corner point as the center, calculate the calibration amount and generate a calibration model; The pixel mapping unit is used to map non-corner pixels in the original image to the calibration model to correct image deviation.

10. The AI ​​model building system for drone image recognition according to claim 9, characterized in that: The image fusion module includes: an image storage unit and an intelligent fusion unit; The image storage unit is used to control the drone to capture images, store all captured images, and analyze the coverage of the images; The intelligent fusion unit is used to identify and stitch all images in the image set to form a fused image with a higher number of pixels than the original image; The color restoration module includes: a superposition contrast unit and a gradient restoration unit; The superposition contrast unit is used to divide the pixel areas of the fused image and the original image, and calculate the contrast of the image in each area; The gradient restoration unit is used to calculate the contrast adjustment amount, adjust the pixel colors in the original image, and output the restored image.

Images