Low-altitude inspection unmanned aerial vehicle image transmission method and system

Abstract

The application relates to the technical field of image processing, in particular to a low-altitude patrol unmanned aerial vehicle image transmission method and system, which comprises the following steps: collecting low-altitude patrol unmanned aerial vehicle images and performing pretreatment; obtaining multiple cluster classes through a clustering algorithm from the pretreated images; determining a main cluster class from the multiple cluster classes; determining adjacent pixel points according to the main cluster class connected domain; traversing and analyzing the adjacent pixel points to obtain an initial target main body; establishing a minimum circumscribed rectangle for the initial target main body, determining a final target main body according to the size of the circumscribed rectangle, and marking the final target main body; and compressing and transmitting the image according to the final target main body region. The higher the compression degree of the final target main body region is, the smaller the compression degree of other regions is, so that the file size is reduced, the compression efficiency is improved, the transmission bandwidth is saved, and the storage space is saved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a low-altitude inspection unmanned aerial vehicle image transmission method and system. BACKGROUND

[0002] Unmanned aerial vehicles play an important role in the electric tower inspection industry. Traditional electric tower inspection usually requires manual tower climbing for inspection, which is time-consuming and labor-intensive and poses a safety risk. Unmanned aerial vehicle inspection can quickly and accurately obtain electric tower state information through aerial photography, thermal imaging, and other technologies, enabling comprehensive monitoring and evaluation of electric towers and supporting facilities, improving inspection efficiency, reducing costs, and reducing personnel safety risks.

[0003] However, when an unmanned aerial vehicle transmits images to a terminal, in order to improve compression efficiency, save transmission bandwidth, and save storage space, the images are generally separated into foreground and background before compression. When using the K-means clustering algorithm (K-means algorithm) to segment images, due to color, lighting, or some technical reasons, the identification of the main body may not be complete. SUMMARY

[0004] To solve the above technical problems, the purpose of the present application is to provide a low-altitude inspection unmanned aerial vehicle image transmission method and system, and the technical solutions adopted are as follows:

[0005] A low-altitude inspection unmanned aerial vehicle image transmission method, the method comprising:

[0006] Collecting low-altitude inspection unmanned aerial vehicle images and performing preprocessing;

[0007] Obtaining multiple clusters from the preprocessed images through a clustering algorithm, and determining a main cluster from the multiple clusters;

[0008] Determining adjacent pixel points based on the main cluster connected domain, traversing and analyzing the adjacent pixel points to obtain an initial target main body;

[0009] Establishing a minimum bounding rectangle for the initial target main body, determining a final target main body based on the size of the bounding rectangle, and marking it;

[0010] Compressing and transmitting the image according to the final target main body region.

[0011] Further, the collecting low-altitude inspection unmanned aerial vehicle images and performing preprocessing further comprises:

[0012] Collecting low-altitude inspection unmanned aerial vehicle images;

[0013] Sequentially performing grayscale processing on the images.

[0014] Further, the step of obtaining a plurality of cluster classes from the pre-processed image by a clustering algorithm further comprises:

[0015] obtaining a plurality of cluster classes from the image processed by the K-means clustering algorithm;

[0016] comparing the difference between the gray value of the pixel region of the different cluster classes in the image and the gray value of the neighborhood thereof;

[0017] extracting the cluster class with a large difference between the gray value and the gray value of the neighborhood and / or a large number of neighborhoods, and segmenting it as the subject cluster class.

[0018] Further, the step of determining the adjacent pixel points according to the subject cluster class connected domain, traversing and analyzing the adjacent pixel points, and obtaining the initial target subject further comprises:

[0019] determining the adjacent pixel points according to the subject cluster class connected domain, and calculating the first possibility of the pixel point being the subject pixel point for each adjacent pixel point;

[0020] when the first possibility is greater than a first preset threshold, the pixel point is identified as the subject pixel point;

[0021] determining the initial target subject according to the subject pixel point and the subject cluster class connected domain.

[0022] Further, the step of determining the adjacent pixel points according to the subject cluster class connected domain further comprises:

[0023] determining other cluster class pixel points around the subject cluster class connected domain; wherein some of the pixel points are embedded in the subject cluster class connected domain, and some of the pixel points cut off the subject cluster class connected domain, resulting in a local area not being connected.

[0024] Further, the step of establishing a minimum bounding rectangle for the initial target subject and determining the final target subject according to the size of the bounding rectangle further comprises:

[0025] establishing a minimum bounding rectangle for the initial target subject, and setting the subject cluster class connected domain in the area represented by the largest bounding rectangle as the final target subject;

[0026] calculating a second possibility of other areas except the subject cluster class connected domain being the final target subject;

[0027] if the second possibility is greater than a second preset threshold, dividing the other areas to the final target subject.

[0028] Further, the step of compressing and transmitting the image according to the final target subject area further comprises:

[0029] Divide the whole image into a plurality of small blocks of a preset number, and perform independent discrete cosine transform on each small block to convert the image from a spatial domain to a frequency domain;

[0030] Quantize different regions of the image according to the calculated quantization matrix factors to different degrees;

[0031] The quantized data is entropy encoded using Huffman coding, and the encoding result is packaged into a file of a preset format for transmission.

[0032] Further, the quantization of different regions of the image according to the calculated quantization matrix factors to different degrees further comprises:

[0033] The quantization matrix factor is calculated according to the second possibility calculated for each region;

[0034] Quantize the region according to the quantization matrix factor.

[0035] Further, the method further comprises:

[0036] After transmitting the image to the receiving end, read the file header information to obtain the parameters and quantization table used for encoding;

[0037] Decode the data stream using the Huffman table provided by the file header to obtain the quantized coefficients, and then use the same quantization table to reversely restore the quantized coefficients to the original coefficients, and perform a preset operation on each small block to obtain the original image data block;

[0038] Convert the color space and recombine all the small blocks into a complete image.

[0039] A low-altitude patrol unmanned aerial vehicle image transmission system, the system comprises:

[0040] A preprocessing module for collecting low-altitude patrol unmanned aerial vehicle images and performing preprocessing;

[0041] An initial target object determination module for obtaining a plurality of cluster classes from the preprocessed image through a clustering algorithm, determining a subject cluster class from the plurality of cluster classes, determining adjacent pixel points according to the subject cluster class connected domain, traversing and analyzing the adjacent pixel points to obtain an initial target object;

[0042] A final target object determination module for establishing a minimum circumscribed rectangle for the initial target object, determining a final target object according to the size of the circumscribed rectangle, and marking the final target object;

[0043] A transmission module for image compression and transmission of the image of the final target object region.

[0044] The present application has the following advantages: the present application collects low-altitude inspection unmanned aerial vehicle images and pre-processes them; a plurality of clusters are obtained from the pre-processed images through a clustering algorithm, and a main cluster is determined from the plurality of clusters; adjacent pixel points are determined according to the main cluster connected domain, and the adjacent pixel points are iterated and analyzed to obtain an initial target main body; a minimum circumscribed rectangle is established for the initial target main body, the final target main body is determined according to the size of the circumscribed rectangle, and is marked; and the image is compressed and transmitted according to the final target main body region. The present application uses image processing technology to obtain a complete unmanned aerial vehicle inspection image target main body, and uses image compression technology to compress the image foreground and background to different degrees. On the basis of K-means image segmentation, the tower main body and the changes in the gray scale values of the neighborhood thereof are retained, and other different cluster pixel points on the image have some specific features, so that the tower main body is removed from the interference pixel points, a relatively complete tower main body is obtained, and technical support is provided for subsequent image compression and transmission. The higher the compression degree of the final target main body region is, the smaller the compression degree of other regions is, so that the file size is reduced, the compression efficiency is improved, the transmission bandwidth is saved, and the storage space is saved. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, a brief introduction will be given to the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on these drawings.

[0046] Figure 1 A flow chart of a low-altitude inspection unmanned aerial vehicle image transmission method provided by an embodiment of the present application;

[0047] Figure 2 An image after gray scale processing provided by an embodiment of the present application;

[0048] Figure 3 An image after marking of the final target main body provided by an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a low-altitude inspection unmanned aerial vehicle image transmission system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to further clarify the technical means and effects taken by the present application to achieve the predetermined inventive purpose, the following describes in detail the specific implementation, structure, features and effects of a low-altitude inspection unmanned aerial vehicle image transmission method and system according to the present application, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0052] The specific scheme of the low-altitude inspection unmanned aerial vehicle image transmission method provided by the present application is described in detail below in combination with the accompanying drawings.

[0053] Please refer to Figure 1 which shows a low-altitude inspection unmanned aerial vehicle image transmission method provided by an embodiment of the present application, the method comprising:

[0054] Step S110: Collecting low-altitude inspection unmanned aerial vehicle images and performing preprocessing.

[0055] This embodiment takes the application of unmanned aerial vehicles in the electric tower inspection industry as an example, that is, the low-altitude inspection unmanned aerial vehicle is mainly used to collect electric tower images. Considering that high-voltage electric towers are generally more prominent in the inspection images, in the gray image, the gray value of the main body of the electric tower is generally higher than that of the background area, and the difference in gray value with the background area is large. In the RGB color mode (red / green / blue color mode) image, the cluster after local clustering is relatively complex, resulting in an area with unclear details and lines. This area cannot be judged as foreground or background by a single color. For this area, this embodiment can take the main cluster pixel points as the guide, traverse all the pixel points in its neighborhood, and comprehensively consider the features such as the gray value, overall trend, shape, distance and connectivity of these pixel points to obtain a relatively complete tower main body.

[0056] In an optional implementation, step S110 further comprises: collecting low-altitude inspection unmanned aerial vehicle images; and sequentially performing gray processing on the images. Figure 2 The image after gray processing provided by an embodiment of the present application takes the target main body as an electric tower, and the obtained high-voltage electric tower inspection image is a color image. After performing gray processing on the image, the obtained gray image is as shown in Figure 2 .

[0057] From the observation, it can be concluded that the high-voltage tower body has the following characteristics: in the gray image, the high-voltage tower is more prominent, and the gray value difference with its neighborhood is more obvious; in the RGB image, the high-voltage tower as a whole presents the color of the main cluster class, but in some tower regions, there are mixed interference of multiple cluster classes, and the tower arm part region is missing; some regions of the high-voltage tower are in the form of uniform long strips and extend from the inside to the outside.

[0058] Therefore, in this step, the image is segmented by K-means clustering, the gray value difference of the pixel regions of different cluster classes in the image and the gray value of the neighborhood is compared, and the regions with large gray value difference with the neighborhood, large number of neighborhoods and large gray value difference of the neighborhood are extracted and segmented; since there are many adjacent pixel points of different cluster classes around the connected domain pixel points of the cluster class meeting the condition, some of these adjacent pixel points lead to incomplete main body regions, the adjacent pixel points are analyzed, and the value, overall trend, shape, distance and connectivity of the neighborhood pixel points are comprehensively considered to obtain a relatively complete tower; then a minimum circumscribed rectangle is established for all suspected target towers (connected domains composed of main cluster class pixel points), and the real main tower is determined according to the size of the rectangle.

[0059] Step S120: obtaining a plurality of cluster classes from the preprocessed image by a clustering algorithm, and determining a main cluster class from the plurality of cluster classes.

[0060] In an optional embodiment, step S120 further comprises: obtaining a plurality of cluster classes from the gray processed image by a K-means clustering algorithm; comparing the gray value difference of the pixel regions of different cluster classes in the image and the gray value of the neighborhood; extracting cluster classes with large gray value difference and / or large number of neighborhoods, and segmenting them as main cluster classes.

[0061] In this step, the main pixel point cluster class is obtained by comparing the gray value difference of the pixel regions of different cluster classes in the K-means image and the gray value of the neighborhood, and the number of neighborhoods, and the complete tower body is obtained by analyzing the characteristics and details of the tower body image.

[0062] Since there are many adjacent pixel points of different cluster classes around the connected domain pixel points of the cluster class meeting the condition, some of these adjacent pixel points lead to incomplete main body regions, the adjacent pixel points are analyzed, and the value, overall trend, shape, distance and connectivity of the neighborhood pixel points are comprehensively considered to obtain a relatively complete tower; then a minimum circumscribed rectangle is established for all suspected target towers (connected domains composed of main cluster class pixel points), and the real main tower is determined according to the size of the rectangle.

[0063] Specifically, first, the image is segmented by K-means to obtain multiple cluster classes. To determine the optimal K value, the elbow method is used for the inspection image. The specific steps are as follows: first, determine the value range of K in the clustering algorithm, set K to start from 2 and calculate every time, for each K value, calculate the sum of squared errors (SSE); plot the K value and the corresponding SSE value; wherein the abscissa is the K value and the ordinate is the SSE value. Select the K value represented by the inflection point in the curve, which indicates that at this K value, the contribution of increasing the K value to reducing the SSE gradually decreases, and the clustering effect gradually becomes stable; the K value represented by the inflection point is determined as the optimal K value. The embodiment shown in the Figure 2 When K = 4, the optimal cluster number can be obtained.

[0064] By analyzing the K = 4 RGB color segmentation image, it can be known that the tower is generally prominent in the image, so the gray value difference between the tower pixel point connected domain and its adjacent region is large; the tower is built by multiple slender metal materials, so reflected in the image, the tower is composed of multiple intersecting straight lines, resulting in multiple complete neighborhoods being segmented, thereby causing the number of adjacent regions to increase rapidly; moreover, the tower is the main body of the image, so its adjacent region will be relatively complex. Therefore, if the current cluster class and the gray value of all its adjacent regions are more different, the gray value of the adjacent regions is more discrete, and the number of adjacent regions is more, the possibility of the cluster class being the main cluster class is greater. A cluster class determines a module, and multiple regions exist in the same module, so the average value of the possibility of all regions in the module being the main region needs to be calculated, and the higher the average value, the greater the possibility of the cluster class represented by the module being the main cluster class.

[0065] Therefore, the image is divided into multiple modules by cluster classes, and in each module, the part where the pixel points are not connected is divided into multiple regions; let the gray value of the current region be p i , the gray values of all its adjacent regions be {p1, p2……pn h}, j be the jth adjacent region of the current region, n i be the number of adjacent regions of the current region, n be the average value of the number of adjacent regions of all regions, e be the e-th region represented by the current cluster class, and z be the number of regions of the current cluster class, so the average value G1 of the possibility of all regions in the current module being the main region is calculated as follows formula (1):

[0066]

[0067] In the formula, p i is the gray value of region i, p j is the gray value of the jth adjacent region of region i, n i is the number of adjacent regions of region i, and n is the average value of the number of adjacent regions of all regions. The greater the value of the area i, the more the number of adjacent areas of the area i, the greater the possibility of the area i being divided into multiple areas by a tower, and the higher the possibility of the area i being a tower. The greater the value of the area i, the greater the difference between the gray value of the area i and the gray value of its adjacent area, the more prominent the area i in the complete image, and the higher the possibility of the area i being a tower. The sum of the possibilities of all areas in the current module being a main area, so The greater the value, the greater the average value G1. The module with the maximum G1 value represents the main cluster class.

[0068] Since there are many adjacent pixel points with different cluster classes around the main cluster class connected domain pixel points, some of these adjacent pixel points cause the area to be incomplete. By traversing and analyzing these pixel points, considering the neighborhood pixel value, overall trend, shape, distance, and connectivity features, a relatively complete tower is obtained.

[0069] Step S130: Determine adjacent pixel points based on the main cluster class connected domain, traverse and analyze the adjacent pixel points, and obtain the initial target main body.

[0070] In an optional embodiment, step S130 further comprises: determining adjacent pixel points based on the main cluster class connected domain, calculating the first possibility of each adjacent pixel point being a main pixel point; when the first possibility is greater than a first preset threshold, the pixel point is identified as a main pixel point; and determining the initial target main body based on the main pixel point and the main cluster class connected domain.

[0071] As can be seen from the RGB image, there are many other cluster class pixel points around the main cluster class connected domain, some of which are embedded in the main cluster class connected domain, and some of which truncate the main cluster class connected domain, causing local areas to be not connected and affecting the judgment of the complete high-voltage tower main body. Traverse these pixel points. It is found through observation that the closer the gray value of the point to the gray value of the main cluster class, the greater the difference between the gray value of the current area background cluster class. Among them, the current area is an 8*8 rectangular frame established with the current pixel point as the center; the current area background cluster class refers to the area with the most number of pixel points of the same cluster class except the main cluster class; the greater the possibility of the point being a tower main pixel point; from the distance point of view, the closer these pixel points are to the nearest main cluster class center point, the greater the possibility of the pixel point being a tower main body; from other angles, the greater the number of adjacent pixels of the pixel point being main pixels, the greater the possibility of the pixel point being a tower main body.

[0072] Let the gray value of the current pixel point be p o , and the average gray value of the main cluster class be The average grayscale value of the background cluster in the area where the current pixel is located is The distance between the current pixel and the center point of the nearest main cluster is L. o The distance to the center point of the background cluster is L, so the probability that the current pixel o is the main pixel is K. o The calculation formula is as follows (2):

[0073]

[0074] In the formula, p o The grayscale value of pixel o. The average gray value of the main cluster. L represents the average grayscale value of the background cluster in the region where pixel o is located. o L is the distance between pixel o and the center point of the nearest connected component of the main cluster, and L is the distance between pixel o and the center point of the connected component of the background cluster. The larger the value, the closer pixel o is to the center point of the main cluster's connected domain than to the center point of the background cluster's connected domain. The closer it is to the main cluster, the greater the probability that pixel o is a main pixel when considering the distance to the pixel's affiliation. The larger the value, the closer the gray value of pixel o is to the average gray value of the main cluster, rather than the gray value of the background cluster. The closer the gray value is to the gray value of the main cluster, the greater the probability that pixel o is a main pixel.

[0075] K o The result is mapped to the range [0,1] to obtain the first probability K'1. The first preset threshold can be set according to empirical values. In this embodiment, it is set to 0.7. When K'1>0.7, the pixel is identified as the main pixel.

[0076] Once the main pixel points are determined, the initial target subject is obtained. At this point, the final target subject needs to be determined from the initial target subject.

[0077] Step S140: Establish the minimum bounding rectangle for the initial target body, determine the final target body based on the size of the bounding rectangle, and mark it.

[0078] In an optional implementation, step S140 further includes: establishing a minimum bounding rectangle for the initial target subject, setting the subject cluster connected domain within the region represented by the bounding rectangle with the largest area as the final target subject; calculating the second possibility that other regions besides the subject cluster connected domain are the final target subjects; and if the second possibility is greater than a second preset threshold, assigning other regions to the final target subjects.

[0079] It can be found from the observation image that there are hanging point problems due to K-means clustering, local image details are not clear due to dark light, and other reasons around the tower body, which cause the local main cluster to be truncated by the background cluster, resulting in the inability to be connected as a whole. The hanging point problem refers to when there is one or more abnormal points in the data set, the K-means algorithm is difficult to correctly assign it to a certain cluster, and these abnormal points are called "hanging points". However, it can be obtained from the analysis of the image that most of the truncated areas are composed of multiple long and narrow rectangles extending uniformly from the tower body, and due to stable lighting, the image is obtained by the inspection unmanned aerial vehicle. When the shooting angle is directly opposite the tower, the reflected light on the metal surface of the tower is strong, the metal looks bright, and the brightness is uniform, and the brightness difference is not obvious. Taking the current position as the starting point (0°), when the unmanned aerial vehicle is deflected to 90° in one direction, as the angle becomes larger, the area extending from the tower body will gradually become dark due to the reflection effect of the metal on the light, and when the deflection is closer to 90°, the farther the area from the unmanned aerial vehicle, the more obvious the darkening of the brightness. (The distance is taken as the standard for the distance when the unmanned aerial vehicle is deflected to 90°) Reflected on the gray scale diagram, the change of the gray scale value of the tower extension area from the starting point to the end point is: as the angle increases, the gray scale value of the tower extension area gradually decreases, and the farther the area from the unmanned aerial vehicle, the greater the degree of decrease in the gray scale value. Therefore, a minimum circumscribed rectangle is established for all connected domains of the main cluster in the background cluster area, the main cluster connected domain in the area represented by the largest circumscribed rectangle is set as the tower body, and the remaining areas are calculated as the possibility of the initial target main body. The circumscribed rectangles of all other areas are stretched in the direction of the rectangle length towards the main body area until they are connected with the tower body (if the rectangle frame is extended to the edge of the image without being connected with the tower body, it is determined that the area is not connected, which can be excluded). In the new rectangle frame, there are main cluster pixel points, background cluster pixel points (background cluster pixel points are the part with the most pixel points in the current area except the main cluster pixel points), and other cluster pixel points that are not the same as the main cluster and the background of the current area (other cluster pixel points are all pixel points except the main cluster pixel points, background cluster pixel points), if the rectangle frame is a suspected target main body, the number of main cluster pixel points and other cluster pixel points is more than the number of background cluster pixel points. Therefore, the more the number of main cluster pixel points and other cluster pixel points in the rectangle frame than the number of background cluster pixel points, the greater the possibility of the main body area in the rectangle frame. By calculating the width of the circumscribed rectangle, the closer the width to the average width of all circumscribed rectangles, the greater the possibility of the main body area in the rectangle frame, the greater the value of the first possibility of the non-main cluster pixel points in the rectangle frame, and the greater the possibility of the initial target main body in the rectangle frame.

[0080] Let the width of the current rectangle frame be w1, and the average width of all circumscribed rectangles be w m, the possibility of the pixel point being the tower main body cluster class is K2, and the maximum value of the possibility of the pixel point being the tower main body pixel point is K max , the number of the main body cluster class pixel points in the current rectangular frame is v1, the number of the background pixel points is v2, and the number of the other pixel points is v3, and the possibility D1 of the current region being the initial target main body is the following formula (3):

[0081]

[0082] In the formula, v1 is the number of the main body cluster class pixel points in the current rectangular frame, v2 is the number of the background pixel points, v3 is the number of the other pixel points, w1 is the width of the current rectangular frame, w m is the average value of the widths of all the circumscribed rectangles, K2 is the possibility of the pixel point being the tower main body cluster class, K max is the maximum value of the possibility of the pixel point being the tower main body pixel point; Sigmoid is a normalization function; and 0.01+|w1-w m | is a preset coefficient for preventing the divisor from being 0, and the smaller the coefficient is, the better, and the embodiment only takes 0.01 as an example.

[0083] The greater the value of (v1+v3)-v2 is, the more the number of the main body cluster class pixel points and the number of the other pixel points than the number of the background pixel points, the greater the possibility of the current region being affected and causing the image to be broken from the main body, and the greater the possibility of the current region being the initial target main body; the smaller the value of 0.01+|w1-w m | is, the closer the width of the current rectangular frame to the average value of the widths of all the circumscribed rectangles, the closer the width of the current region to the width of the initial target main body, and the greater the possibility of the current region being the initial target main body; The greater the value of K2 is, the closer the possibility of the pixel point being the tower main body pixel to the maximum value, the more the pixel points in the region should belong to the main body cluster class, and the greater the possibility of the current region being the initial target main body.

[0084] After the initial target main body is obtained, the possibility of the initial target main body being the final target main body is further determined by the change of the gray value of the initial target main body caused by the different angles of the unmanned aerial vehicle inspection.

[0085] Since the final target object is a relatively smooth metal, its brightness depends more on the reflection of light. The initial target object is divided into multiple small blocks of equal size (the length of the circumscribed rectangle of the initial target object is divided into multiple segments, and the width remains unchanged, so that the circumscribed rectangle is divided into small blocks of equal size). When the UAV deviates from 0o to 90o, the gray value of the same small block in the UAV inspection image at different angles decreases gradually, and the farther the small block is from the UAV, the more obvious the change in the gray value. Therefore, in the initial target object, the gray value of the same small block gradually decreases with the increase of the angle, and the farther the small block is from the UAV, the more obvious the change in the gray value, which indicates that the current initial target object is more likely to be the final target object.

[0086] Suppose that the gray values of the small block m in the initial target object A obtained at different angles are {p m1 , p m2 ……p mk}, and the gray values of the small block n obtained at different angles are {p n1 , p n2 ……p nk}. The possibility that the region A is the initial target object is D A . Therefore, the possibility that the initial target object A is the final target object is I A , which is as follows (4):

[0087]

[0088] In the formula, p ma is the gray value of the small block m of the initial target object A at the current angle, p m(a+1) is the gray value of the small block m of the initial target object A at the next angle, p na is the gray value of the small block n of the initial target object A at the current angle, and p n(a+1) is the gray value of the small block n of the initial target object A at the next angle. D A is the possibility that the initial target object A is the initial target object, is the product of the changes in the gray values of the small block m at different angles, is the product of the changes in the gray values of the small block n at different angles. When is larger, it indicates that the gray value of the small block m changes with the increase of the shooting angle, and the change is larger. When the value is greater than 0, it indicates that the gray value of the small block m gradually decreases with the change of the angle. When The smaller the value is, the more the gray value of the small block n is affected by the increase of the shooting angle, the gray value changes, but the change is small, and when the value is greater than 0, it indicates that the gray value of the small block n also has a gradual change process of gradually reducing the gray value with the change of the angle, and when The greater the value is, the more the gray value of the small block far away from the unmanned aerial vehicle changes than the gray value of the small block close to the unmanned aerial vehicle, and the greater the possibility of the region A being the final target subject is. A The greater the value is, the greater the possibility of the region A being the initial target subject is; in combination, when the region A is the initial target subject with obvious gray value change characteristics of small blocks in the region caused by different shooting angles, the greater the possibility of the region A being the real subject region is.

[0089] Map the I A result to [0, 1] to obtain a second possibility I' A , and the second preset threshold is 0.7, so when I' A > 0.7, the initial target subject is determined as the final target subject. The pixel points in the rectangular frame determined as the subject cluster are automatically changed to the subject cluster.

[0090] Thus, the complete suspected tower subject is determined.

[0091] Finally, the minimum circumscribed rectangle is established for all suspected target towers (connected domains composed of subject cluster pixel points), and the target tower is determined according to the size of the rectangle.

[0092] It can be seen from Figure 2 that in the image, there are more than one high-voltage tower, and there are two smaller towers on the left side of the image, but due to the distance between the unmanned aerial vehicle and the target tower, the target tower is much larger than the other towers, so the minimum circumscribed rectangle can be established for all suspected target towers, and the subject cluster connected domain in the rectangular frame with the largest area is determined as the target tower. Figure 3 The image after the final target subject is marked according to an embodiment of the present application is shown in Figure 3 , and the subject cluster connected domain in the rectangular frame with the largest area is determined as the target tower, that is, the final target subject.

[0093] Step S150: Compress and transmit the image according to the final target subject region.

[0094] In an alternative embodiment, step S150 further comprises: dividing the whole image into a plurality of small blocks of a preset number, and performing independent discrete cosine transform on each small block to convert the image from spatial domain to frequency domain; performing different degrees of quantization on different regions of the image according to the calculated quantization matrix factor; and performing entropy coding on the quantized data using Huffman coding, and encapsulating the coding result into a file of a preset format for transmission.

[0095] In an alternative embodiment, step S150 further comprises: calculating the quantization matrix factor according to the second possibility calculated for each region; and performing quantization on the region according to the quantization matrix factor.

[0096] In an alternative embodiment, the method further comprises: after the image is transmitted to the receiving end, reading the file header information to obtain the parameters and quantization table used for coding; decoding the data stream using the Huffman table provided by the file header to obtain the quantized coefficients, and then inversely restoring the quantized coefficients into original coefficients using the same quantization table, and performing a preset operation on each small block to obtain the original image data block; and converting the color space and recombining all the small blocks into a complete image.

[0097] In the embodiment, the image compression can adopt the JPEG (Joint Photographic Experts Group) image compression algorithm, and specifically, the JPEG image compression algorithm is used according to the final target subject mark. First, the RGB color space is converted into the YCbCr color space, and the chroma components Cb and Cr are subsampled, and then the whole image is divided into a plurality of 8*8 small blocks, and independent discrete cosine transform is performed on each small block to convert the image from spatial domain to frequency domain, and then different degrees of quantization are performed on different regions of the image. In the quantization, the quantization matrix factor determines the retention degree of the discrete cosine transform (DCT) coefficients, which also means that the larger the quantization matrix factor is, the lower the retention degree of the corresponding DCT coefficients is, and the more blurred the image details are. The minimum value of the quantization matrix factor is 1. In the embodiment, the quantization matrix factor is θ, and the relationship between the possibility D1 of the initial target subject and the quantization matrix factor is as follows (5):

[0098] θ = arccot D1 + 1 (5)

[0099] Wherein, arccot is the inverse cotangent function; the greater the possibility D1 of the initial target subject is, the greater the possibility that the region is the subject region is, the smaller the quantization matrix factor of the small block where the region is, the higher the DCT coefficient retention degree is, the lower the compression degree of the region is, and the more complete the subject detail retention is; the smaller D1 is, the smaller the possibility that the region is the subject region is, the greater the quantization matrix factor of the small block where the region is, the lower the DCT coefficient retention degree is, the higher the compression degree of the region is, the file size is reduced, the compression efficiency is improved, the transmission bandwidth is saved, and the storage space is saved.

[0100] The quantized data is entropy coded using Huffman coding. After all the steps are completed, the result is packaged into a file in JPEG format for transmission. After being transmitted to the receiving end, the JPEG file header information is first read to obtain the parameters used for coding and the quantization table, the data stream is decoded using the Huffman table provided by the JPEG file header to obtain the quantized DCT coefficients, the quantized coefficients are inversely restored into the original DCT coefficients using the same quantization table, and IDCT is performed on each 8x8 block to obtain the original image data block, the original image data block is converted from the YCbCr color space back to the RGB color space, and finally all the 8x8 blocks are recombined into a complete image.

[0101] By using the method, the target subject of the complete unmanned aerial vehicle inspection image is obtained by using image processing technology, and the image foreground and background are compressed to different degrees by using image compression technology. On the basis of K-means image segmentation, the tower subject is retained and the interference pixel points are removed according to the changes of the gray values of the tower subject and its neighborhood and some specific features possessed by other different cluster pixel points on the image, so that a relatively complete tower subject is obtained, thereby providing technical support for subsequent image compression and transmission. The higher the compression degree of the final target subject region is, the smaller the compression degree of other regions is, thereby reducing the file size, improving the compression efficiency, saving the transmission bandwidth, and saving the storage space.

[0102] The specific scheme of the low-altitude inspection unmanned aerial vehicle image transmission method provided by the present application will be described in detail below with reference to the accompanying drawings.

[0103] Referring to Figure 4 , which shows a low-altitude inspection unmanned aerial vehicle image transmission system provided by an embodiment of the present application, the system comprises a preprocessing module 410, an initial target subject determination module 420, a final target subject determination module 430, a transmission module 440, and a decompression module 450.

[0104] The preprocessing module 410 is configured to collect low-altitude inspection unmanned aerial vehicle images and perform preprocessing.

[0105] The initial target subject determining module 420 is configured to obtain a plurality of cluster classes from the preprocessed image by using a clustering algorithm, and determine a subject cluster class from the plurality of cluster classes; determine adjacent pixel points according to a connected domain of the subject cluster class, traverse and analyze the adjacent pixel points, and obtain the initial target subject.

[0106] The final target subject determining module 430 is configured to establish a minimum circumscribed rectangle for the initial target subject, determine the final target subject according to the size of the circumscribed rectangle, and mark the final target subject.

[0107] The transmission module 440 is configured to perform image compression on the image of the final target subject region and transmit the image.

[0108] In an optional implementation, the preprocessing module 410 is further configured to: acquire images of low-altitude inspection unmanned aerial vehicles; and sequentially perform grayscale processing on the images.

[0109] In an optional implementation, the initial target subject determining module 420 is further configured to: obtain a plurality of cluster classes from the image after grayscale processing by using a K-means clustering algorithm; compare the difference between the grayscale values of the pixel regions of different cluster classes in the image and the grayscale values of the neighborhoods of the pixel regions; extract cluster classes with a large difference between the grayscale values and the grayscale values of the neighborhoods and / or a large number of neighborhoods, and divide the cluster classes out as the subject cluster class.

[0110] In an optional implementation, the initial target subject determining module 420 is further configured to: determine adjacent pixel points according to a connected domain of the subject cluster class, calculate, for each adjacent pixel point, a first possibility that the pixel point is a subject pixel point, determine the pixel point as the subject pixel point when the first possibility is greater than a first preset threshold, and determine the initial target subject according to the subject pixel point and the connected domain of the subject cluster class.

[0111] In an optional implementation, the initial target subject determining module 420 is further configured to: determine other cluster class pixel points around the connected domain of the subject cluster class; wherein some of the pixel points are embedded in the connected domain of the subject cluster class, and some of the pixel points cut off the connected domain of the subject cluster class, resulting in a local region that is not connected.

[0112] In an optional implementation, the final target subject determining module 430 is further configured to: establish a minimum circumscribed rectangle for the initial target subject, set the connected domain of the subject cluster class in the region represented by the largest circumscribed rectangle as the final target subject, calculate a second possibility that other regions except the connected domain of the subject cluster class are the final target subject, and divide the other regions to the final target subject when the second possibility is greater than a second preset threshold.

[0113] In an alternative embodiment, the transmission module 440 is further configured to divide the whole image into a plurality of small blocks in a preset number, and perform independent discrete cosine transform on each small block to convert the image from a spatial domain to a frequency domain; and quantize different regions of the image according to the calculated quantization matrix factors to different degrees; and perform entropy coding on the quantized data using Huffman coding, and encapsulate the coding result into a file in a preset format for transmission.

[0114] In an alternative embodiment, the transmission module 440 is further configured to calculate the quantization matrix factor according to the second possibility calculated for each region; and quantize the region according to the quantization matrix factor.

[0115] In an alternative embodiment, the method further comprises a decompression module 450 configured to read the file header information after the image is transmitted to the receiving end to obtain the parameters and quantization table used for coding; decode the data stream using the Huffman table provided by the file header to obtain the quantized coefficients, and then use the same quantization table to reversely restore the quantized coefficients into the original coefficients, and perform a preset operation on each small block to obtain the original image data block; and convert the color space to recombine all the small blocks into a complete image.

[0116] The system of the embodiment uses image processing technology to obtain the target subject of the complete unmanned aerial vehicle inspection image, and uses image compression technology to compress the foreground and background of the image to different degrees. On the basis of K-means image segmentation, the iron tower subject and the interference pixel points are removed according to the changes of the gray values of the iron tower subject and its neighborhood and some specific features possessed by other different cluster pixel points on the image, so that a relatively complete iron tower subject is obtained, which provides technical support for subsequent image compression and transmission. The higher the compression degree of the final target subject region of the system is, the smaller the compression degree of other regions is, so as to reduce the file size, improve the compression efficiency, save the transmission bandwidth, and save the storage space.

[0117] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0118] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment mainly describes the differences from other embodiments.

Claims

1. A low-altitude inspection unmanned aerial vehicle image transmission method, characterized in that, The method comprises: Collecting low-altitude inspection unmanned aerial vehicle images and preprocessing them; Obtaining multiple clusters from the preprocessed images through a clustering algorithm, and determining a subject cluster from the multiple clusters; Determining adjacent pixel points according to the connected domain of the subject cluster, traversing and analyzing the adjacent pixel points, and obtaining an initial target subject; Establishing a minimum circumscribed rectangle for the initial target subject, determining a final target subject according to the size of the circumscribed rectangle, and marking the final target subject; Compressing and transmitting the image according to the final target subject region; collecting low-altitude inspection unmanned aerial vehicle images and preprocessing them further comprises: Collecting low-altitude inspection unmanned aerial vehicle images; Sequentially performing grayscale processing on the images; Obtaining multiple clusters from the preprocessed images through a clustering algorithm, and determining a subject cluster from the multiple clusters further comprises: Obtaining multiple clusters from the grayscale-processed images through a K-means clustering algorithm; Comparing the grayscale values of the pixel regions of different clusters in the image with the differences in grayscale values of the neighborhood; Extracting clusters with large differences in grayscale values and / or a large number of neighborhoods, and segmenting them out as subject clusters; determining adjacent pixel points according to the connected domain of the subject cluster, traversing and analyzing the adjacent pixel points, and obtaining an initial target subject further comprises: Determining adjacent pixel points according to the connected domain of the subject cluster, calculating, for each adjacent pixel point, a first possibility that the pixel point is a subject pixel point; When the first possibility is greater than a first preset threshold, the pixel point is identified as a subject pixel point; Determining an initial target subject according to the subject pixel point and the connected domain of the subject cluster; determining adjacent pixel points according to the connected domain of the subject cluster further comprises: Determining other cluster pixel points around the connected domain of the subject cluster; some of these pixel points are embedded in the connected domain of the subject cluster, and some pixel points truncate the connected domain of the subject cluster, resulting in local regions that are not connected; establishing a minimum circumscribed rectangle for the initial target subject, and determining a final target subject according to the size of the circumscribed rectangle further comprises: Establishing a minimum circumscribed rectangle for the initial target subject, and setting the connected domain of the subject cluster in the region represented by the largest circumscribed rectangle as the final target subject; Calculating a second possibility that other regions except the connected domain of the subject cluster are the final target subject; 2. The image transmission method of the low-altitude inspection unmanned aerial vehicle according to claim 1, characterized in that, If the second possibility is greater than a second preset threshold, dividing the other regions to the final target subject. Compressing and transmitting the image according to the final target subject region further comprises: Dividing the entire image into a preset number of small blocks, and performing independent discrete cosine transform on each small block to convert the image from spatial domain to frequency domain; Quantizing different regions of the image to different degrees according to the calculated quantization matrix factors; 3. The image transmission method of low-altitude inspection unmanned aerial vehicle according to claim 2, characterized in that, Using Huffman coding to perform entropy coding on the quantized data, and encapsulating the coding result into a file in a preset format for transmission. Quantizing different regions of the image to different degrees according to the calculated quantization matrix factors further comprises: Calculating the quantization matrix factors according to the second possibility calculated for each region; 4. The low-altitude inspection unmanned aerial vehicle image transmission method according to claim 3, characterized in that, Quantizing the region according to the quantization matrix factors. The method further comprises: After the image is transmitted to the receiving end, reading the file header information to obtain the parameters and quantization table used for coding; The data stream is decoded using the Huffman table provided by the file header to obtain quantized coefficients, the quantized coefficients are inversely recovered into original coefficients using the same quantization table, and each small block is subjected to a preset operation to obtain an original image data block; The color space is converted, and all the small blocks are recombined into a complete image.

5. A low-altitude inspection unmanned aerial vehicle image transmission system, characterized in that, The system is used to implement the low-altitude patrol unmanned aerial vehicle image transmission method in any one of claims 1-4, and the system comprises: A preprocessing module is configured to collect low-altitude patrol unmanned aerial vehicle images and perform preprocessing; An initial target subject determination module is configured to obtain multiple cluster classes by clustering algorithm through the preprocessed images, determine a subject cluster class from the multiple cluster classes, determine adjacent pixel points according to the subject cluster class connected domain, traverse and analyze the adjacent pixel points, and obtain an initial target subject; A final target subject determination module is configured to establish a minimum circumscribed rectangle for the initial target subject, determine a final target subject according to the size of the circumscribed rectangle, and mark the final target subject; A transmission module is configured to perform image compression on the image of the final target subject region and transmit the image.

Images