UAV visual recognition method based on artificial intelligence

Abstract

The present invention belongs to the technical field of unmanned aerial vehicle visual recognition, and specifically discloses an artificial intelligence-based unmanned aerial vehicle visual recognition method, which comprises: using an unmanned aerial vehicle to collect various pattern image sets corresponding to various markers in a target area, and screening out invalid sub-images after collection from the aspects of lens distortion rate and illumination lens variation rate, so as to ensure that the image set finally obtained has a low degree of distortion; by identifying the lens deviation type of various pattern image sets corresponding to various markers, performing targeted image correction and collection on each marker, which helps to reduce the collection frequency of invalid images, and at the same time can ensure that the collected images are closer to the real scene; by performing marker behavior recognition on the image set after correction and collection, the influence of factors such as lens deviation and illumination on the image quality is reduced, and then performing marker behavior recognition on these high-quality images, which can significantly improve the recognition accuracy.

Description

Technical Field

[0001] The invention belongs to the technical field of unmanned aerial vehicle visual recognition, and relates to a unmanned aerial vehicle visual recognition method based on artificial intelligence. Background Art

[0002] Traditional video acquisition methods mainly rely on manual inspections and fixed cameras. Although fixed cameras can shoot continuously, they are limited by the viewing angle and installation position and cannot fully capture the dynamic behavior of markers. Therefore, they have certain limitations in marker recognition and behavior analysis. In order to solve the shortcomings of traditional marker monitoring methods in terms of coverage and behavior analysis, and to improve the efficiency and accuracy of marker monitoring, drone video has emerged. The drone video method mainly uses drones equipped with high-definition cameras to achieve comprehensive monitoring of markers in the target area through flight control and camera technology.

[0003] There are also some solutions in the prior art for analyzing the behavior of markers by collecting images through drones, but they still have the following shortcomings: 1. In the prior art, there is a lack of specific identification of the environmental conditions and lens distortion conditions for lens collection caused by the lens deviation caused by drone image collection, and there is a lack of effective means to automatically screen and eliminate invalid sub-images caused by poor lighting conditions.

[0004] 2. In the existing technology, image correction and acquisition usually lacks specificity, and no differentiated correction strategy is adopted for different types of lens deviations, which limits the accuracy of marker behavior recognition. This may lead to errors in the recognition results, affecting subsequent analysis and judgment. Summary of the invention

[0005] In view of this, in order to solve the problems raised in the above background technology, an artificial intelligence-based drone visual recognition method is proposed.

[0006] The purpose of the present invention can be achieved through the following technical solutions: The present invention provides an artificial intelligence-based drone visual recognition method, comprising the following steps: Step 1, using a drone to collect various pattern image sets corresponding to various markers in the target area, numbering each marker as 1, 2, ...i..., a, numbering each pattern image set as 1, 2, ...j..., b, and numbering each sub-image in the pattern image set as 1, 2, ...k..., c.

[0007] Step 2: annotate information on each set of images corresponding to each marker, and filter out invalid sub-images in each set of images corresponding to each marker.

[0008] Step 3: Identify the lens deviation type of each pattern image set corresponding to each marker, and perform image correction and acquisition on it.

[0009] Step 4: Identify whether there are any abnormalities in the corresponding behaviors of each marker and provide early warning feedback.

[0010] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention collects various image sets corresponding to various markers in the target area through drones, and screens out invalid sub-images after collection from the perspective of lens distortion rate and light mirror variation rate. When drones collect images, lens collection deviations may occur due to focal length parameter errors and lens jitter errors set by the lens. At the same time, excessive or weak lighting may cause image blur, overexposure and other problems, thereby affecting the accuracy and authenticity of image collection. By screening invalid sub-images with higher lens distortion rates, it can be ensured that the final image set has a lower degree of distortion, thereby improving the accuracy and availability of the image.

[0011] (2) The present invention identifies the lens deviation type of each marker corresponding to each pattern image set, and performs targeted image correction and acquisition for each marker. Different types of lens deviations will have different effects on image accuracy and authenticity. By identifying and correcting these deviations, targeted image correction and acquisition can help reduce the frequency of invalid image acquisition, and at the same time ensure that the acquired images are closer to the real scene.

[0012] (3) The present invention performs marker behavior recognition on the corrected image set. The corrected image set has been optimized to reduce the impact of factors such as lens deviation and lighting on image quality. Therefore, performing marker behavior recognition on these high-quality images can significantly improve recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0014] Figure 1 The present invention is a schematic flow chart of the steps for implementing the method. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] See also Figure 1As shown, the present invention provides an artificial intelligence-based UAV visual recognition method, which includes the following steps: Step 1, using a UAV to collect various pattern image sets corresponding to various markers in the target area, numbering each marker as 1, 2, ...i..., a, numbering each pattern image set as 1, 2, ...j..., b, and numbering each sub-image in the pattern image set as 1, 2, ...k..., c.

[0017] In a preferred embodiment, the various pattern image sets include various flight angle images, various flight altitude images, and various light brightness images.

[0018] The brightness of each light is specifically a percentage of daylight brightness.

[0019] Step 2: annotate information on each set of images corresponding to each marker, and filter out invalid sub-images in each set of images corresponding to each marker.

[0020] In a preferred embodiment, the annotation information in the image set includes a marker outline area and a marker inertial behavior area.

[0021] The marker inertial behavior area is, for example, a pedestrian inertial walking area or a vehicle inertial circulation area.

[0022] In another preferred embodiment, the method of screening out invalid sub-images in the image sets of various patterns corresponding to each marker includes: extracting the scene model of each sub-image in the image sets of various patterns corresponding to each marker, comparing it with the pre-stored standard scene model corresponding to each marker, and analyzing the lens distortion rate of each sub-image in the image sets of various patterns corresponding to each marker.

[0023] Through environmental recognition, the illumination mirror variation rate of each sub-image in each pattern image set corresponding to each marker is analyzed.

[0024] The lens distortion rate and illumination mirror variation rate of each sub-image in each pattern image set corresponding to each marker are compared with the preset lens distortion rate and illumination mirror variation rate corresponding thresholds respectively, and the sub-images whose lens distortion rate exceeds the preset lens distortion rate corresponding threshold or the illumination mirror variation rate exceeds the preset illumination mirror variation rate corresponding threshold in each pattern image set corresponding to each marker are screened out, determined as invalid sub-images, and screened out.

[0025] In another preferred embodiment, the lens distortion rate of each sub-image in each pattern image set corresponding to each marker is analyzed in the following specific manner: the corresponding annotation contour of the annotation information of the scene model of each sub-image in each pattern image set corresponding to each marker is identified by an image processing algorithm, and the annotation contour is compared with the pre-stored standard scene model corresponding to the corresponding marker, and the contour line distortion degree D of each sub-image in each pattern image set corresponding to each marker is identified. ijk .

[0026] The contour line distortion index information indicates the ratio of the deformed contour area between the contour annotated corresponding to the information and the contour of the corresponding area in the standard scene model.

[0027] Obtain the acquisition angle and height of each sub-image in each pattern image set corresponding to each marker, determine the expected extraction range of the contour of each sub-image in each pattern image set corresponding to each marker according to the preset lens extraction range, compare it with the contour range of the scene model of each sub-image in each pattern image set, and identify the contour layer difference C of each sub-image in each pattern image set corresponding to each marker ijk .

[0028] The preset lens extraction range is specifically a designated image acquisition region area under the combination of each designated angle and each designated height.

[0029] The contour layer difference index annotation information corresponds to the redundancy and missing area ratio between the marked contour and the contour of the area that should actually be collected in the standard scene model.

[0030] Analyze the lens distortion rate (ABR) of each sub-image in each image set corresponding to each marker ijk = l 1 *D ijk +l 2 *C ijk +l 0 , where l 1 , l 2 They are the preset contour line distortion, contour layer difference and corresponding distortion ratio, l 0 To set a constant.

[0031] In another preferred embodiment, the analysis of the illumination mirror variability of each sub-image in each pattern image set corresponding to each marker is specifically carried out as follows: the light brightness belonging to each sub-image in each pattern image set corresponding to each marker is obtained, and it is compared with a preset light brightness threshold, and sub-images in each pattern image set corresponding to each marker whose light brightness exceeds the preset light brightness threshold are screened out, and are marked as illumination sub-images in each pattern image set corresponding to each marker, and sub-images in each pattern image set corresponding to each marker whose light brightness does not exceed the preset light brightness threshold are marked as meteorological sub-images in each pattern image set corresponding to each marker.

[0032] The annotation contour corresponding to the annotation information of the scene model of each sub-image in the various pattern image sets corresponding to each marker is compared with the pre-stored standard scene model of the corresponding marker habitat, and the halo area of ​​each sub-image in the various pattern image sets corresponding to each marker is identified, and then the halo area of ​​each illumination sub-image in the various pattern image sets corresponding to each marker is extracted, and it is compared with the preset reference halo area to obtain the halo coefficient of each illumination sub-image in the various pattern image sets corresponding to each marker.

[0033] The halo area is a highlighted area in the sub-image with a brightness higher than that of the surrounding area.

[0034] The annotation contour corresponding to the annotation information of the scene model of each sub-image in the image set of each pattern corresponding to each marker is compared with the pre-stored standard scene model of the corresponding marker, and the fuzzy contour area of ​​each sub-image in the image set of each pattern corresponding to each marker is identified, and then the fuzzy contour area of ​​each meteorological sub-image in the image set of each pattern corresponding to each marker is extracted, and compared with the preset reference fuzzy contour area to obtain the fuzzy coefficient of each illumination sub-image in the image set of each pattern corresponding to each marker.

[0035] The fuzzy contour is a contour whose corresponding annotation contour of the annotation information matches the contour of the corresponding area in the standard scene model but whose grayscale value does not reach the expected grayscale value.

[0036] Obtain the corresponding categories of each sub-image in each pattern image set corresponding to each marker, the corresponding categories of sub-images include illumination sub-images and meteorological sub-images. When a sub-image in a pattern image set corresponding to a marker is an illumination sub-image, obtain the halo coefficient λ of the illumination sub-image in the pattern image set corresponding to the marker from the halo coefficients of the illumination sub-images in the pattern image set corresponding to the marker. 0 ,by is the illumination mirror variability of the sub-image in the pattern image set corresponding to the marker; when the sub-image in the pattern image set corresponding to the marker is a meteorological sub-image, the halo coefficient γ of the illumination sub-image in the pattern image set corresponding to the marker is obtained from the fuzzy coefficient of each illumination sub-image in each pattern image set corresponding to each marker 0 ,by is the illumination mirror variation rate of the sub-image in the pattern image set corresponding to the marker, and the illumination mirror variation rate of each sub-image in each pattern image set corresponding to each marker is obtained in this way, and e is a natural constant.

[0037] The present invention uses a drone to collect various pattern image sets corresponding to various markers in the target area, and screens out invalid sub-images after collection from the perspective of lens distortion rate and illumination lens variation rate. When the drone collects images, lens collection deviation may occur due to focal length parameter errors and lens jitter errors set by the lens. At the same time, too strong or too weak illumination may cause image blur, overexposure and other problems, thereby affecting the accuracy and authenticity of image collection. By screening invalid sub-images with higher lens distortion rates, it can be ensured that the final image set has a lower degree of distortion, thereby improving the accuracy and availability of the image.

[0038] Step 3: Identify the lens deviation type of each pattern image set corresponding to each marker, and perform image correction and acquisition on it.

[0039] In a preferred implementation, the lens deviation types include focus type, jitter type, and meteorological type.

[0040] In another preferred embodiment, the lens deviation type of each pattern image set corresponding to each marker is identified by recording each sub-image in each pattern image set corresponding to each marker after filtering out invalid sub-images as each marker sub-image in each pattern image set corresponding to each marker.

[0041] The lens distortion rate of each sub-image in each marker image set corresponding to each marker is extracted from the lens distortion rate of each sub-image in each marker image set corresponding to each marker, and is used as the fuzzy distortion index χ1 of each marker sub-image in each marker image set corresponding to each marker ijh , h is the number of the identification subgraph, h = 1, 2,…, w.

[0042] The corresponding annotation contour of the annotation information of the scene model of each sub-image in the image set corresponding to each marker is compared with the pre-stored standard scene model of the corresponding marker, and the ghost area of ​​each sub-image in the image set corresponding to each marker is identified, and then the ghost area of ​​each marker sub-image in the image set corresponding to each marker is extracted, and the ghost area is compared with the preset reference ghost area to obtain the ghost phenomenon index χ2 of each marker sub-image in the image set corresponding to each marker. ijh .

[0043] The ghost area is an area where the corresponding annotation contour of the annotation information appears repeatedly in the sub-image.

[0044] Extract the illumination mirror variation rate of each sub-image in each image set corresponding to each marker from the illumination mirror variation rate of each sub-image in each image set corresponding to each marker, and use it as ... ijh .

[0045] The preset reference blur distortion index, reference ghosting index, and reference light printing index are respectively recorded as pass The maximum value index of each marker sub-image in each pattern image set corresponding to each marker is obtained, recorded as the key index of each sub-image in each pattern image set corresponding to each marker, and then the lens deviation type corresponding to the key index is obtained.

[0046] Specifically, the key indicators correspond to lens deviation types: the blur distortion indicator corresponds to the focus type, the ghosting phenomenon indicator corresponds to the jitter type, and the light printing phenomenon indicator corresponds to the meteorological type.

[0047] The number of identification sub-images with lens deviation types of focus type, jitter type, and meteorological type in each pattern image set corresponding to each marker is counted, and the lens deviation type to which the maximum number of identification sub-images belongs is selected by comparison and recorded as the lens deviation type of each pattern image set corresponding to each marker.

[0048] In another preferred embodiment, the image correction and acquisition includes: extracting the marker outline area belonging to the standard information of each marker sub-image in each pattern image set corresponding to each marker, constructing the marker body contour belonging to each pattern image set corresponding to each marker by three-dimensional contour reconstruction, comparing it with the preset shape contour of the corresponding marker to identify whether they are consistent; if the marker body contour belonging to a pattern image set corresponding to a certain marker does not match the preset shape contour of the corresponding marker, then marking the information belonging to the pattern image set corresponding to the marker as lacking integrity, and accordingly screening out the various pattern image sets corresponding to each marker that lack integrity.

[0049] Based on the lens deviation type of each pattern image set corresponding to each marker, image correction and collection are performed on each pattern image set corresponding to each marker that lacks integrity.

[0050] Specifically, the method of performing image correction and acquisition for each pattern image set corresponding to each marker that lacks integrity is: obtaining the lens deviation type of each pattern image set corresponding to each marker, when the lens deviation type of a certain pattern image set corresponding to a certain marker is a focus type, adjusting the camera acquisition focal length of the certain pattern image set corresponding to the marker.

[0051] When the lens deviation type of a certain marker corresponding to a certain pattern image set is a jitter type, the camera acquisition frequency of the certain pattern image set corresponding to the marker is adjusted, and the acquisition frequency is the interval duration of image acquisition.

[0052] When the lens deviation type of a certain marker corresponding to a certain pattern image set is a meteorological type, the marker movement trajectory area corresponding to the marker is circled, and the camera acquisition position of the marker corresponding to the certain pattern image set is adjusted, so as to keep the image acquisition flight trajectory of the UAV consistent with the marker movement trajectory.

[0053] The present invention identifies the lens deviation type of each marker corresponding to each pattern image set, and performs targeted image correction and acquisition for each marker. Different types of lens deviations (such as distortion, etc.) will have different effects on image accuracy and authenticity. By identifying and correcting these deviations, targeted image correction and acquisition can help reduce the frequency of invalid image acquisition, and at the same time ensure that the acquired images are closer to the real scene.

[0054] Step 4: Identify whether there are any abnormalities in the corresponding behaviors of each marker and provide early warning feedback.

[0055] In a preferred embodiment, the identification of whether there is an abnormality in the corresponding behavior of each marker is specifically carried out as follows: a set of images of various patterns corresponding to each marker after image correction and acquisition is obtained, a dynamic behavior corresponding to each marker is obtained by a dynamic recognition method, and the dynamic behavior is compared with the pre-stored dynamic behaviors corresponding to the marker; if the dynamic behavior corresponding to a marker does not match the pre-stored dynamic behaviors corresponding to the corresponding marker, the dynamic behavior of the marker is recorded as an abnormal behavior, and then an early warning feedback is given for the dynamic behavior of the marker.

[0056] The present invention performs marker behavior recognition on the corrected and collected image set. The corrected and collected image set has been optimized to reduce the impact of factors such as lens deviation and lighting on image quality. Therefore, performing marker behavior recognition on these high-quality images can significantly improve recognition accuracy.

[0057] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. The UAV visual recognition method based on artificial intelligence is characterized by: The following steps are involved: Step 1: Use drones to collect various image sets corresponding to various markers in the target area. Each image set includes images of various flight angles, altitudes, and light brightness, and each light brightness is a percentage of daylight brightness. Number each marker as 1, 2, ..., i, ..., a, number each pattern image set as 1, 2, ..., j, ..., b, and number each sub-image in the pattern image set as 1, 2, ..., k, ..., c; Step 2: annotate the information of each set of images corresponding to each marker, and filter out invalid sub-images in each set of images corresponding to each marker, including: Extract the scene model of each sub-image in the image set corresponding to each marker, compare it with the pre-stored standard scene model corresponding to each marker, and obtain the contour line distortion degree D of each sub-image. ijk and contour layer difference C ijk , D ijk With C ijk The weighted summation is used to obtain the lens distortion rate (ABR) of each sub-image. ijk ; Through environmental recognition, the illumination mirror change rate (CDR)0 of each sub-image is analyzed, including: Obtain the light brightness of each sub-image, compare it with the preset light brightness threshold, filter out the sub-images whose light brightness exceeds the preset light brightness threshold, and mark them as illumination sub-images, otherwise mark them as meteorological sub-images; Identify the halo area of ​​each illumination sub-image, compare it with the preset reference halo area, and obtain the halo coefficient of each illumination sub-image; Identify the fuzzy contour area of ​​each meteorological sub-image, compare it with the preset reference fuzzy contour area, and obtain the fuzzy coefficient of each meteorological sub-image; Get the corresponding category of each sub-image, which includes illumination sub-image and meteorological sub-image. When a sub-image is an illumination sub-image, get the halo coefficient λ0 of the illumination sub-image. When the sub-image is a meteorological sub-image, the fuzzy coefficient γ0 of the meteorological sub-image is obtained from the fuzzy coefficients of each meteorological sub-image. e is a natural constant; (ABR) ijk and (CDR)0 are compared with the preset lens distortion rate and illumination mirror change rate corresponding thresholds respectively, and each sub-graph whose lens distortion rate exceeds the preset lens distortion rate corresponding threshold or illumination mirror change rate exceeds the preset illumination mirror change rate corresponding threshold is screened out, and is determined as an invalid sub-graph and screened out; Step 3: Identify the lens deviation type of each image set corresponding to each marker, and perform image correction and collection. The lens deviation types include focus type, jitter type, and meteorological type. The identification method is: Each subgraph after filtering out invalid subgraphs is recorded as each identification subgraph; Extract the lens distortion rate of each logo sub-image and use it as the blur distortion index χ1 of each logo sub-image ijh , h is the number of the identification subgraph, h = 1, 2, ..., w; Identify the ghosting area of ​​each sub-image, compare it with the preset reference ghosting area, and obtain the ghosting phenomenon index χ2 of each sub-image ijh ; Extract the illumination mirror variation rate of each sub-image as the light printing phenomenon index χ3 of each sub-image ijh ; The preset reference blur distortion index, reference ghosting index, and reference light printing index are respectively recorded as pass The maximum value index of each identified sub-image is obtained as the key index of each sub-image, and the key index corresponds to the lens deviation type, the blur distortion index corresponds to the focus type, the ghosting phenomenon index corresponds to the jitter type, and the light printing phenomenon index corresponds to the meteorological type; The number of identification sub-images of lens deviation types of focus type, jitter type, and meteorological type is counted, and the lens deviation type to which the maximum number of identification sub-images belongs is selected by comparison and recorded as the lens deviation type; Step 4: Identify whether there are any abnormalities in the corresponding behaviors of each marker and provide early warning feedback.

2. The method for visual recognition of unmanned aerial vehicles based on artificial intelligence according to claim 1, characterized in that: The annotation information in the image set includes a marker outline area and a marker inertial behavior area, and the marker inertial behavior area includes a pedestrian inertial walking area and a vehicle inertial circulation area.

3. The method for visual recognition of unmanned aerial vehicles based on artificial intelligence according to claim 1, characterized in that: The specific method of analyzing the lens distortion rate of each sub-image in each pattern image set corresponding to each marker is as follows: The image processing algorithm is used to identify the corresponding annotation contour of the scene model of each sub-image in the image set of each pattern corresponding to each marker, and it is compared with the pre-stored standard scene model corresponding to the corresponding marker to identify the contour line distortion degree D of each sub-image in the image set of each pattern corresponding to each marker. ijk ; The proportion of the deformed contour area between the contour marked by the contour line distortion index information and the contour of the corresponding area in the standard scene model; Obtain the acquisition angle and height of each sub-image in each pattern image set corresponding to each marker, determine the expected extraction range of the contour of each sub-image in each pattern image set corresponding to each marker according to the preset lens extraction range, compare it with the contour range of the scene model of each sub-image in each pattern image set, and identify the contour layer difference C of each sub-image in each pattern image set corresponding to each marker ijk ; The redundancy and missing area ratio between the contour marked by the corresponding annotation information of the contour layer and the contour of the area to be collected in the standard scene model; Analyze the lens distortion rate (ABR) of each sub-image in each image set corresponding to each marker ijk =l1*D ijk +l2*C ijk +l0, where l1 and l2 are the preset contour line distortion and contour layer difference corresponding distortion ratios, and e0 is a set constant.

4. The method for visual recognition of unmanned aerial vehicles based on artificial intelligence according to claim 1, characterized in that: The image correction and acquisition includes: constructing the physical contour of the marker corresponding to each pattern image set of each marker by means of three-dimensional contour reconstruction, comparing it with the preset shape contour of the corresponding marker, and identifying whether they are consistent; if the physical contour of the marker corresponding to a pattern image set of a certain marker does not conform to the preset shape contour of the corresponding marker, marking the information of the marker corresponding to the pattern image set as lacking in integrity, and accordingly screening out the pattern image sets corresponding to each marker lacking in integrity; Based on the lens deviation type of each pattern image set corresponding to each marker, image correction and collection are performed on each pattern image set corresponding to each marker that lacks integrity.

5. The method for visual recognition of unmanned aerial vehicles based on artificial intelligence according to claim 1, characterized in that: The identification of whether there is an abnormality in the corresponding behavior of each marker is specifically carried out in the following manner: obtaining a set of images of various patterns corresponding to each marker after image correction and acquisition, obtaining a set of dynamic behaviors corresponding to each marker by means of dynamic recognition, and comparing them with the pre-stored dynamic behaviors corresponding to the corresponding markers; if the dynamic behavior corresponding to a marker does not match the pre-stored dynamic behaviors corresponding to the corresponding markers, the dynamic behavior of the marker is recorded as an abnormal behavior, and then a warning feedback is given for the dynamic behavior of the marker.

Images