Low-altitude inspection unmanned aerial vehicle image transmission method and system
By using clustering algorithms and quantization techniques in drone image transmission, the target subject of the image is determined and retained, the incomplete segmentation problem caused by the K-means algorithm is solved, the image compression and transmission efficiency is improved, and more efficient File transfer.
Patent Information
- Application Number
- CN202410901933.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-05
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-07-05
AI Technical Summary
In the image transmission of drone, the image segmentation caused by the K-means algorithm is incomplete, resulting in inaccurate identification subjects, affecting image compression and transmission efficiency.
By acquiring and preprocessing drone images, determining subject clusters using clustering algorithms, analyzing adjacent pixel points, establishing the smallest external rectangle, determining the final target subject, and performing different degrees of quantization and entropy encoding according to its area, performing images Compress and transfer.
The complete target subject recognition and retention of drone images is achieved, image compression efficiency is improved, file size is reduced, and transmission bandwidth and storage space are saved.
Smart Images

Figure CN119011834A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a method and system for transmitting images of a low-altitude inspection unmanned aerial vehicle. Background Art
[0002] Drones play an important role in the power tower inspection industry. Traditional power tower inspections usually require manual climbing of the tower, which is time-consuming, labor-intensive and involves safety risks. Drone inspections can quickly and accurately obtain the status information of power towers through aerial photography, thermal imaging and other technologies, and achieve comprehensive monitoring and evaluation of power towers and supporting facilities, thereby improving inspection efficiency, reducing costs and reducing personnel safety risks.
[0003] However, when the drone transmits images to the terminal, in order to improve compression efficiency, save transmission bandwidth, and save storage space, the foreground and background of the image must be separated before compression. When using the K-means clustering algorithm (K-means algorithm) to segment the image, due to color, lighting or some technical reasons, the recognition of the subject may be incomplete. Summary of the invention
[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a low-altitude inspection drone image transmission method and system, and the technical solutions adopted are as follows:
[0005] A method for transmitting images of a low-altitude inspection drone, the method comprising:
[0006] Collect low-altitude inspection drone images and perform pre-processing;
[0007] Using a clustering algorithm to obtain a plurality of clusters from the preprocessed image, and determining a main cluster from the plurality of clusters;
[0008] Determine adjacent pixel points according to the connected domain of the subject cluster, traverse and analyze the adjacent pixel points, and obtain the initial target subject;
[0009] Establishing a minimum bounding rectangle for the initial target subject, determining the final target subject according to the size of the bounding rectangle, and marking it;
[0010] The image is compressed and transmitted according to the final target subject area.
[0011] Furthermore, the collecting of low-altitude inspection drone images and preprocessing further includes:
[0012] Collect low-altitude inspection drone images;
[0013] The images are subjected to grayscale processing in sequence.
[0014] Furthermore, the step of obtaining a plurality of clusters from the preprocessed image by a clustering algorithm and determining a main cluster from the plurality of clusters further comprises:
[0015] The grayscale processed image is clustered using the K-means algorithm to obtain multiple clusters;
[0016] Comparing the grayscale values of pixel regions of different clusters in the image with the grayscale values of their neighbors;
[0017] Clusters with grayscale values that are significantly different from those of their neighbors and / or with a large number of neighbors are extracted and segmented as the main clusters.
[0018] Furthermore, the determining of adjacent pixels based on the connected domain of the subject cluster, traversing and analyzing the adjacent pixels, and obtaining the initial target subject further includes:
[0019] Determine adjacent pixels based on the connected domain of the main cluster, and for each adjacent pixel, calculate the first possibility that the pixel is the main pixel;
[0020] When the first possibility is greater than a first preset threshold, the pixel point is identified as a subject pixel point;
[0021] An initial target subject is determined according to the subject pixel points and the subject cluster connected domain.
[0022] Furthermore, the determining of adjacent pixels based on the main cluster connected domain further includes:
[0023] Determine other cluster pixel points around the main cluster connected domain; some of these pixel points are embedded in the main cluster connected domain, and some pixel points cut off the main cluster connected domain, resulting in local area disconnection.
[0024] Furthermore, the step of establishing a minimum bounding rectangle for the initial target subject and determining the final target subject according to the size of the bounding rectangle further includes:
[0025] Establishing a minimum bounding rectangle for the initial target subject, and setting the subject cluster connected domain within the area represented by the bounding rectangle with the largest area as the final target subject;
[0026] Calculate the second possibility that other regions except the connected domain of the subject cluster are the final target subject;
[0027] If the second possibility is greater than a second preset threshold, the other areas are divided into the final target body.
[0028] Furthermore, compressing and transmitting the image according to the final target subject area further includes:
[0029] The entire image is divided into a preset number of small blocks, and each small block is independently discrete cosine transformed to transform the image from the spatial domain to the frequency domain;
[0030] Quantize different regions of the image to different degrees according to the calculated quantization matrix factors;
[0031] The quantized data is entropy encoded using Huffman coding, and the encoding result is packaged into a file in a preset format for transmission.
[0032] Furthermore, the quantizing different regions of the image according to the calculated quantization matrix factors to different degrees further includes:
[0033] A quantization matrix factor is calculated based on the second likelihood calculated for each region;
[0034] The region is quantized according to the quantization matrix factors.
[0035] Furthermore, the method further comprises:
[0036] After the image is transmitted to the receiving end, the file header information is read to obtain the parameters and quantization table used for encoding;
[0037] Decoding the data stream using the Huffman table provided by the file header to obtain quantized coefficients, then using the same quantization table to reversely restore the quantized coefficients to original coefficients, and performing a preset operation on each small block to obtain an original image data block;
[0038] Convert the color space and reassemble all the small patches into a complete image.
[0039] A low-altitude inspection drone image transmission system, the system comprising:
[0040] The preprocessing module is used to collect and preprocess the images of low-altitude inspection drones;
[0041] The initial target subject determination module is used to obtain multiple clusters from the preprocessed image through a clustering algorithm, and determine the main cluster from the multiple clusters; determine adjacent pixel points based on the connected domain of the main cluster, traverse and analyze the adjacent pixel points, and obtain the initial target subject;
[0042] A final target subject determination module, used to establish a minimum bounding rectangle for the initial target subject, determine the final target subject according to the size of the bounding rectangle, and mark it;
[0043] The transmission module is used to compress and transmit the image of the final target main area.
[0044] The present invention has the following beneficial effects: the present invention collects low-altitude inspection drone images and performs preprocessing; the preprocessed images are subjected to a clustering algorithm to obtain multiple clusters, and a main cluster is determined from the multiple clusters; adjacent pixels are determined according to the main cluster connected domain, and adjacent pixels are traversed and analyzed to obtain the initial target subject; a minimum bounding rectangle is established for the initial target subject, and the final target subject is determined according to the size of the bounding rectangle, and is marked; the image is compressed and transmitted according to the final target subject area. The present invention obtains the target subject of a complete drone inspection image by using image processing technology, and compresses the image foreground and background to different degrees by using image compression technology. On the basis of K-means image segmentation, according to the change of grayscale value of the tower main body and its neighborhood, and some specific features of other different clusters of pixels on the image, the tower main body is retained, and the interfering pixels are removed to obtain a relatively complete tower main body, providing technical support for subsequent image compression and transmission; the higher the compression degree of the final target main area of the present invention, the smaller the compression degree of other areas, thereby reducing the file size, improving the compression efficiency, saving the transmission bandwidth, and saving the storage space. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0046] Figure 1 A flowchart of a method for transmitting images of a low-altitude inspection drone provided by an embodiment of the present invention;
[0047] Figure 2 An image after grayscale processing provided by an embodiment of the present invention;
[0048] Figure 3 The image after the final target subject is marked according to an embodiment of the present invention;
[0049] Figure 4 A schematic diagram of a low-altitude inspection UAV image transmission system provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0050] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the low-altitude inspection drone image transmission method and system proposed by the present invention, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0051] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0052] The specific scheme of the low-altitude inspection UAV image transmission method provided by the present invention is described in detail below with reference to the accompanying drawings.
[0053] See also Figure 1 , which shows a low-altitude inspection drone image transmission method provided by an embodiment of the present invention, the method comprising:
[0054] Step S110: Collect low-altitude inspection drone images and perform pre-processing.
[0055] This embodiment takes the application of drones in the power tower inspection industry as an example, that is, low-altitude inspection drones are mainly used to collect power tower images. Considering that high-voltage power towers are generally more prominent in inspection images, it is manifested in the grayscale image as follows: the grayscale value of the main body of the power tower is generally higher than the background area, and the grayscale value difference with the background area is large; in the RGB color mode (red / green / blue, red, green and blue color mode) image, there are areas where the clusters after local clustering are more complex, resulting in unclear details and lines. Such areas cannot be judged as foreground or background by a single color. For such areas, this embodiment can use the main cluster pixel points as a guide, traverse all the pixel points in its neighborhood, and comprehensively consider the grayscale value, overall direction, shape, distance and connectivity of these pixel points to obtain a relatively complete tower body.
[0056] In an optional implementation, step S110 further includes: acquiring low-altitude inspection drone images; and performing grayscale processing on the images in sequence. Figure 2 The grayscale processed image provided by an embodiment of the present invention is as follows. Taking the target subject as an iron tower as an example, the obtained high-voltage tower inspection image is a color image. After the image is grayscale processed, the obtained grayscale image is as follows: Figure 2 shown.
[0057] Through observation, it can be concluded that the main body of the high-voltage tower has the following characteristics: in the grayscale image, the high-voltage tower is more prominent, and the grayscale value difference with its neighborhood is more obvious; in the RGB image, the high-voltage tower as a whole presents the color of the main cluster, but in some areas of the tower there is mixed interference of multiple clusters, and some areas of the tower arm are missing; some areas of the high-voltage tower are uniformly long strips, extending from the inside to the outside.
[0058] Therefore, in this step, the image is segmented by K-means clustering, and the grayscale values of pixel areas of different clusters in the image are compared with the differences in grayscale values of their neighbors. The areas with large grayscale value differences between the main area and its neighbors, large number of neighbors, and large neighborhood grayscale value differences are extracted and segmented; since there are many adjacent pixels of different clusters around the pixels of the connected domain of the clusters that meet the conditions, some of these adjacent pixels cause the required main area to be incomplete, these adjacent pixels are traversed and analyzed, and the values, overall trends, shapes, distances, and connectivity characteristics of their neighborhood pixels are comprehensively considered to obtain a relatively complete tower; then, the minimum circumscribed rectangle is established for all suspected target towers (the connected domain formed by the main cluster pixels), and the real main tower is determined according to the size of the rectangle.
[0059] Step S120: obtaining a plurality of clusters through a clustering algorithm on the preprocessed image, and determining a main cluster from the plurality of clusters.
[0060] In an optional embodiment, step S120 further includes: obtaining multiple clusters through a K-means clustering algorithm on the grayscale processed image; comparing the grayscale values of pixel areas in different clusters in the image and the differences with the grayscale values of their neighbors; extracting clusters with large differences in grayscale values from those of their neighbors and / or with a large number of neighbors, and segmenting them out as main clusters.
[0061] In this step, by comparing the grayscale values of pixel areas of different clusters in the K-means image and the difference with the grayscale values of their neighbors, as well as the number of neighbors, the main pixel clusters are obtained, and then the complete main body of the tower is obtained by analyzing the characteristic details of the tower image.
[0062] Since there are many adjacent pixels of different clusters around the pixels of the connected domain of the clusters that meet the conditions, some of these adjacent pixels make the required main area incomplete. We traverse and analyze these adjacent pixels, and comprehensively consider the values, overall trends, shapes, distances and connectivity characteristics of their neighborhood pixels to obtain a relatively complete tower. Then, we establish the minimum circumscribed rectangle for all suspected target towers (the connected domain formed by the main cluster pixels), and determine the real main tower based on the size of the rectangle.
[0063] Specifically, the image is first segmented by K-means to obtain multiple clusters. To determine the optimal K value, the elbow method is used on the inspection image. The specific steps are: first determine the value range of K in the clustering algorithm, set K to start from 2, and add one each time. For each K value, calculate the sum of squares of clustering errors (SSE); plot the K value and the corresponding SSE value; wherein the horizontal axis is the K value and the vertical axis is the SSE value. Select the K value represented by the inflection point in the curve. This point indicates that under this K value, the contribution of increasing the K value to reducing the SSE gradually decreases, and the clustering effect gradually becomes stable; the K value represented by the inflection point is defined as the optimal K value. This embodiment shows Figure 2 In the example, when K=4, the optimal number of clusters can be obtained.
[0064] By analyzing the RGB color segmentation map with K=4, we can see that: the iron tower is generally more prominent in the image, so the grayscale value of the connected domain of the iron tower pixel point is quite different from the grayscale value of its adjacent area; the iron tower is built of multiple slender metal materials, so reflected in the image, the iron tower is composed of multiple intersecting straight lines, resulting in multiple complete neighborhoods being segmented, which increases the number of its adjacent areas; and the iron tower is the main body of the image, so its adjacent areas will be more complex. It can be obtained that if the grayscale value difference between the current cluster and all its adjacent areas is greater, the grayscale value discreteness of the adjacent areas is greater, and the number of adjacent areas is greater, the possibility that the cluster is the main cluster is greater. A cluster determines a module, and there are multiple areas in the same module, so the average value of the possibility of all areas in the module being the main area should be calculated. The higher the average value, the greater the possibility that the cluster represented by this module is the main cluster.
[0065] Therefore, the image is divided into multiple modules based on clustering. In each module, the unconnected parts of the pixels are divided into multiple regions. Let the gray value of the current region be p i , the grayscale values of all its adjacent areas are {p1, p2...p h}, j is the jth adjacent region of the current region, n i is the number of adjacent regions of the current region, n is the average number of adjacent regions of all regions, e is the e-th region represented by the current cluster, z is the number of regions in the current cluster, so the average value G1 of the possibility that all regions of the current module are the main region is calculated as follows (1):
[0066]
[0067] In the formula, p i is the gray value of region i, p j is the gray value of the jth adjacent area of area i, n i is the number of adjacent regions of region i, and n is the average number of adjacent regions of all regions. The larger the value is, the more adjacent regions there are to region i. The greater the possibility that a neighboring region is divided into multiple regions by i, resulting in a surge in the number of adjacent regions. The higher the possibility that region i is an iron tower. The larger the value is, the greater the difference between the grayscale value of region i and its adjacent regions is. The more prominent region i is in the complete image, the higher the possibility that region i is an iron tower. is the sum of the possibilities of all regions in the current module being the main region, so The larger the value, the larger the average value G1. The module with the largest G1 value belongs to, and the cluster it represents is the main cluster.
[0068] Since there are many adjacent pixels of different clusters around the pixels of the main cluster connected domain, some of these adjacent pixels make the required area incomplete. By traversing and analyzing these pixels and comprehensively considering the characteristics of their neighborhood pixel values, overall direction, shape, distance and connectivity, a relatively complete tower is obtained.
[0069] Step S130: Determine adjacent pixel points based on the connected domain of the subject cluster, traverse and analyze the adjacent pixel points, and obtain the initial target subject.
[0070] In an optional embodiment, step S130 further includes: determining adjacent pixel points based on the connected domain of the subject cluster, and for each adjacent pixel point, calculating the first possibility that the pixel point is the subject pixel point; when the first possibility is greater than a first preset threshold, the pixel point is identified as the subject pixel point; and determining the initial target subject based on the subject pixel point and the connected domain of the subject cluster.
[0071] From the RGB image, we can see that there are many other cluster pixels around the main cluster connected domain. Some of the pixels are embedded in the main cluster connected domain, and some of the pixels cut off the main cluster connected domain, resulting in local areas not being connected, affecting the judgment of the complete high-voltage tower body. Traverse these pixels. Through observation, it is found that the closer the gray value of the point is to the gray value of the main cluster, the greater the difference with the gray value of the background cluster in the current area. The current area here is an 8*8 rectangular box established with the current pixel as the center; the background cluster in the current area refers to: the area with the largest number of pixels of the same cluster in the current area except the main cluster; the greater the possibility that the point is the main pixel of the tower; from the perspective of distance, the closer these pixels are to the nearest main cluster center point, the greater the possibility that the pixel is the main body of the tower; from other perspectives, if the number of adjacent pixels of the pixel is the main pixel, the greater the possibility that the pixel is the main body of the tower.
[0072] Set the gray value of the current pixel to p o , the average gray value of the main cluster is The average gray value of the background cluster in the area where the current pixel is located is The distance between the current pixel and the center point of the nearest subject cluster is L o , the distance from the center point of the background cluster is L, so the possibility that the current pixel o is the main pixel is K o The calculation formula is as follows (2):
[0073]
[0074] In the formula, p o is the gray value of pixel o, is the average gray value of the main cluster, is the average gray value of the background cluster in the area where the pixel o is located, L o is the distance between the pixel point o and the center point of the nearest connected domain of the main cluster, and L is the distance between the pixel point o and the center point of the connected domain of the background cluster. The larger the value is, the closer the pixel point o is to the center point of the connected domain of the main cluster than to the center point of the connected domain of the background cluster, and the closer it is to the main cluster. This means that considering the attribution of the pixel point in terms of distance, the greater the possibility that the pixel point o is the main pixel point. The larger the value of is, the closer the gray value of pixel o is to the average gray value of the main cluster, rather than the gray value of the background cluster. The closer the gray value is to the gray value of the main cluster, the greater the possibility that pixel o is the main pixel.
[0075] K o The result is mapped to [0, 1] to obtain the first possibility K'1. The first preset threshold can be set according to an empirical value. In this embodiment, it is set to 0.7. When K'1>0.7, the pixel point is identified as the main pixel point.
[0076] After the subject pixel points are determined, the initial target subject is obtained, and now the final target subject needs to be determined from the initial target subject.
[0077] Step S140: establishing a minimum bounding rectangle for the initial target body, determining the final target body according to the size of the bounding rectangle, and marking it.
[0078] In an optional embodiment, step S140 further includes: establishing a minimum bounding rectangle for the initial target subject, setting the subject cluster connected domain within the area represented by the bounding rectangle with the largest area as the final target subject; calculating the second possibility that other areas except the subject cluster connected domain are the final target subject; if the second possibility is greater than a second preset threshold, dividing other areas into the final target subject.
[0079] By observing the image, we can find that there are hanging point problems around the tower body, which are caused by the hanging point problem of K-means clustering and the unclear details of the local image due to the dark light. As a result, the local main clusters in the image are truncated by the background clusters, resulting in the inability to connect as a whole. The hanging point problem means that when there are one or more abnormal points in the data set, it is difficult for the K-means algorithm to correctly assign them to a cluster. These abnormal points are called "hanging points". However, analysis of the image shows that these truncated areas are mostly composed of multiple thin and long rectangles of uniform thickness extending outward from the tower body. In addition, due to the stable lighting, the image is taken by the inspection drone. When the shooting angle is facing the tower, the reflected light on the metal surface of the tower is relatively The metal looks bright and evenly bright, with no obvious brightness difference. Taking the current position as the starting point (0°), when the drone deflects to one side in the direction of 90°, as the angle increases, the area extending from the main body of the tower gradually darkens due to the reflection effect of the metal on the light, and when the offset is closer to 90°, the brightness of the area farther away from the drone becomes more obvious (the distance is based on the distance when the drone deflects to 90°). Reflected on the grayscale map, the change of the grayscale value of the tower extension area from the starting point to the end point is: as the angle increases, the grayscale value of the tower extension area gradually decreases, and the farther away from the drone, the greater the degree of grayscale value decreases; therefore, it is the background cluster. The minimum bounding rectangle is established for all the connected domains of the main clusters in the cluster area, and the connected domain of the main cluster in the area represented by the largest bounding rectangle is set as the tower body. The possibility of the remaining areas being the initial target body is calculated, and the bounding rectangles of all the remaining areas are extended to the main area in the direction of the rectangle length until they are connected to the tower body (if the rectangular frame is extended to the edge of the image and is not connected to the tower body, the area is determined to be an unconnected area and can be excluded). In the new rectangular frame, there are main cluster pixels, background cluster pixels (background cluster pixels are the part with the largest number of pixels in the current area except the main cluster pixels) and other pixels that are not the same as the main cluster and the background of the current area. Other cluster pixels (other cluster pixels are all pixels except the subject cluster pixels and background cluster pixels). If the rectangular frame contains a suspected target subject, the sum of the subject cluster pixels and other cluster pixels is greater than the background cluster pixels. Therefore, when the number of subject cluster pixels and other cluster pixels in the rectangular frame is greater than the number of background cluster pixels, the possibility that the rectangular frame is the subject area is greater. By calculating the width of the circumscribed rectangle, the closer the width is to the average width of all circumscribed rectangles, the possibility that the rectangular frame is the subject area is greater. The greater the value of the first possibility of the non-subject cluster pixels in the rectangular frame, the greater the possibility that the rectangular frame is the initial target subject.
[0080] Let the width of the current rectangle be w1, and the average width of all the bounding rectangles be w mIn the current rectangular frame, the probability that the pixel point of the non-main cluster is the main cluster of the tower is K2, and the maximum probability that the pixel point is the main pixel point of the tower is K max , the number of subject cluster pixels in the current rectangular frame is v1, the number of background pixels is v2, and the number of other pixels is v3. Then the probability D1 of the current area being the initial target subject is as follows (3):
[0081]
[0082] Where v1 is the number of pixels of the main cluster in the current rectangular frame, v2 is the number of background pixels, v3 is the number of other pixels, w1 is the width of the current rectangular frame, and w m is the average width of all circumscribed rectangles, K2 is the possibility that the pixel point of the non-main cluster is the main cluster of the tower, and K max is the maximum value of the possibility that the pixel is the main pixel of the tower; Sigmoid is the normalization function; 0.01+|w1-w m |The 0.01 is a preset coefficient to prevent the divisor from being 0. The smaller the coefficient, the better. This embodiment only takes 0.01 as an example.
[0083] The larger the value of (v1+v3)-v2 is, the more the number of subject cluster pixels and other pixels is than the number of background pixels. The more likely the current area is affected, resulting in a break between the image and the subject. The more likely the current area is the original target subject. m The smaller the value of | is, the closer the width of the current rectangle is to the average width of all the circumscribed rectangles, which means that the width of the current area is closer to the width of the initial target body, and the greater the possibility that the current area is the initial target body; The larger the value of , the closer the possibility that the non-main cluster pixels in the rectangular frame are the main pixels of the tower are to the maximum value of the possibility, indicating that the more pixels in the area should belong to the main cluster, the greater the possibility that the current area is the initial target main body.
[0084] After obtaining the initial target subject, the possibility of the initial target subject being the final target subject is further determined by changing the grayscale value of the initial target subject caused by different angles of drone inspection and shooting.
[0085] Since the final target body is a relatively smooth metal, its brightness depends more on the reflection of light. The initial target body is divided into multiple small blocks of equal size (the length of the circumscribed rectangle of the initial target body is divided into multiple segments, the width remains unchanged, and the circumscribed rectangle is divided into small blocks of equal size). As the drone deflects from 0o to 90o, the grayscale value of the same small block in the drone inspection image at different angles has a gradual decrease, and the grayscale value change of the small block farther away from the drone is more obvious. Therefore, within the initial target body, the grayscale value of the same small block gradually decreases with the increase of the angle, and the grayscale value change of the small block farther away from the drone is more obvious than the grayscale value change of the small block closer to the drone, indicating that the current initial target body is more likely to be the final target body.
[0086] Suppose that the grayscale values of the small blocks m that are getting farther and farther away from the drone in the initial target body A at different angles are {p m1 、p m2 …… mk}, the grayscale values of the small block n that is getting closer and closer to the drone at different angles are {p n1 、p n2 …… nk}, the probability that region A is the initial target subject is D A Therefore, the possibility that the initial target subject A is the final target subject I A It is the following formula (4):
[0087]
[0088] In the formula, p ma is the gray value of the small block m of the initial target subject A at the current angle, p m(a+1) is the gray value of the small block m of the original target subject A at the next angle, p na is the gray value of the small block n of the initial target subject A at the current angle; p n(a+1) The gray value of the small block n of the original target subject A at the next angle, D A is the possibility that the initial target subject A is the initial target subject, is the product of the gray value changes of the small block m at different angles, is the product of the gray value changes of small block n at different angles. When the value is larger, it means that the gray value of the small block m changes with the increase of the shooting angle, and the degree of change is large. When the value is greater than 0, it means that the gray value of the small block m has a gradual process in which the gray value gradually decreases with the change of the angle. The smaller the value, the gray value of the small block n changes with the increase of the shooting angle, but the degree of change is small. When the value is greater than 0, it means that the gray value of the small block n also has a gradual process in which the gray value gradually decreases with the change of the angle. The larger the value of , the greater the possibility that area A is the final target. A The larger it is, the more likely it is that area A is the original target subject. Combining the two, when area A is the original target subject due to different shooting angles, resulting in obvious grayscale value changes in small blocks in the area, then the possibility that area A is the real subject area is greater.
[0089] Will I A The result is mapped to [0,1], and the second possibility I' is obtained A , the second preset threshold is 0.7, then when I' A When the value is greater than 0.7, the initial target subject is identified as the final target subject. The pixels in the rectangular frame that are identified as the subject cluster are automatically changed to the subject cluster.
[0090] At this point, the complete main body of the suspected iron tower has been identified.
[0091] Finally, the minimum circumscribed rectangle is established for all suspected target towers (the connected domain formed by the main cluster pixel points), and the target main tower is determined according to the size of the rectangle.
[0092] Depend on Figure 2 It can be seen that there is more than one high-voltage tower in the image, and there are two smaller towers on the left side of the image. However, due to the distance of the drone shooting, the target tower is much larger than the other towers. Therefore, the minimum circumscribed rectangle can be established for all suspected target towers, and the area within the rectangular box can be calculated. The connected domain of the main cluster in the rectangular box with the largest area is defined as the target tower. Figure 3 The image after the final target subject is marked according to an embodiment of the present invention, such as Figure 3 As shown, the connected domain of the subject cluster within the rectangular frame with the largest area is defined as the target tower, that is, the final target subject.
[0093] Step S150: compressing and transmitting the image according to the final target subject area.
[0094] In an optional embodiment, step S150 further includes: dividing the entire image into a preset number of small blocks, and performing an independent discrete cosine transform on each small block to convert the image from the spatial domain to the frequency domain; quantizing different regions of the image to different degrees according to the calculated quantization matrix factors; entropy encoding the quantized data using Huffman coding, and packaging the encoding results into a file in a preset format for transmission.
[0095] In an optional implementation, step S150 further includes: obtaining a quantization matrix factor according to the second possibility calculated for each region; and quantizing the region according to the quantization matrix factor.
[0096] In an optional embodiment, the method further includes: after transmitting the image to the receiving end, reading the file header information to obtain the parameters and quantization table used for encoding; using the Huffman table provided in the file header to decode the data stream to obtain quantized coefficients, and then using the same quantization table to reversely restore the quantized coefficients to the original coefficients, and performing preset operations on each small block to obtain the original image data block; converting the color space and recombining all the small blocks into a complete image.
[0097] In this embodiment, the image compression can adopt the JPEG (Joint Photographic Experts Group, JPEG) image compression algorithm. Specifically, the JPEG image compression algorithm is used according to the final target subject of the mark. First, the RGB color space is converted to the YCbCr color space, and the chrominance components Cb and Cr are sub-sampled. Then the whole image is divided into multiple 8*8 small blocks, and each small block is independently discrete cosine transformed to transform the image from the spatial domain to the frequency domain. After that, different regions of the image are quantized to different degrees. In quantization, the quantization matrix factor determines the degree of retention of the discrete cosine transform (DCT) coefficients, which also means that the larger the quantization matrix factor, the lower the corresponding DCT coefficient retention degree, and the more blurred the image details. The minimum value of the quantization matrix factor is 1. In this embodiment, the quantization matrix factor is set to θ, and the relationship between the possibility D1 of the current region being the initial target subject and the quantization matrix factor is as follows (5):
[0098] θ=arccotD1+1; (5)
[0099] Among them, arccot is the inverse cotangent function; the greater the possibility D1 that the current area is the initial target body, the greater the possibility that the area is the main body area, the smaller the quantization matrix factor of the small block where the area is located, the higher the degree of DCT coefficient retention, the lower the degree of compression of the area, and the more complete the main details are retained; when D1 is smaller, the smaller the possibility that the area is the main body area, the larger the quantization matrix factor of the small block where the area is located, the lower the degree of DCT coefficient retention, the higher the degree of compression of the area, reducing the file size, improving compression efficiency, saving transmission bandwidth, and saving storage space.
[0100] The quantized data is entropy encoded using Huffman coding. After all the steps are completed, the results are encapsulated into a JPEG format file for transmission. After transmission to the receiving end, the JPEG file header information is first read to obtain the parameters and quantization table used for encoding. The Huffman table provided by the JPEG file header is used to decode the data stream to obtain the quantized DCT coefficients. The quantization coefficients are then reversely restored to the original DCT coefficients using the same quantization table. IDCT is performed on each 8x8 block to obtain the original image data block, which is then converted from the YCbCr color space back to the RGB color space. Finally, all 8x8 blocks are reassembled into a complete image.
[0101] The method of this embodiment is adopted to obtain the target subject of the complete drone inspection image using image processing technology, and the image foreground and background are compressed to different degrees through image compression technology. On the basis of K-means image segmentation, according to the changes in the grayscale values of the tower body and its neighborhood, as well as some specific characteristics of other different clusters of pixels on the image, the tower body is retained and the interfering pixels are removed to obtain a relatively complete tower body, providing technical support for subsequent image compression and transmission; the higher the degree of compression of the final target body area, the smaller the degree of compression of other areas, thereby reducing the file size, improving compression efficiency, saving transmission bandwidth, and saving storage space.
[0102] The specific scheme of the low-altitude inspection UAV image transmission method provided by the present invention is described in detail below with reference to the accompanying drawings.
[0103] See also Figure 4 , which shows a low-altitude inspection drone image transmission system provided by an embodiment of the present invention, the system includes: a preprocessing module 410, an initial target subject determination module 420, a final target subject determination module 430, a transmission module 440 and a decompression module 450.
[0104] The preprocessing module 410 is used to collect low-altitude inspection drone images and perform preprocessing.
[0105] The initial target subject determination module 420 is used to obtain multiple clusters from the preprocessed image through a clustering algorithm, determine the main cluster from the multiple clusters; determine adjacent pixels based on the connected domain of the main cluster, traverse and analyze the adjacent pixels to obtain the initial target subject.
[0106] The final target subject determination module 430 is used to establish a minimum bounding rectangle for the initial target subject, determine the final target subject according to the size of the bounding rectangle, and mark it.
[0107] The transmission module 440 is used to compress and transmit the image of the final target main body area.
[0108] In an optional implementation, the preprocessing module 410 is further used to: collect low-altitude inspection drone images; and perform grayscale processing on the images in sequence.
[0109] In an optional embodiment, the initial target subject determination module 420 is further used to: obtain multiple clusters through the K-means clustering algorithm of the grayscale processed image; compare the grayscale values of pixel areas in different clusters in the image and the differences with the grayscale values of their neighbors; extract clusters with large differences in grayscale values from those of their neighbors and / or a large number of neighbors, and segment them out as main clusters.
[0110] In an optional embodiment, the initial target subject determination module 420 is further used to: determine adjacent pixel points based on the subject cluster connected domain, and for each adjacent pixel point, calculate the first possibility that the pixel point is the subject pixel point; when the first possibility is greater than a first preset threshold, the pixel point is identified as the subject pixel point; and determine the initial target subject based on the subject pixel points and the subject cluster connected domain.
[0111] In an optional embodiment, the initial target subject determination module 420 is further used to: determine other cluster pixel points around the main cluster connected domain; wherein some of these pixel points are embedded in the main cluster connected domain, and some pixel points cut off the main cluster connected domain, resulting in local area disconnection.
[0112] In an optional embodiment, the final target subject determination module 430 is further used to: establish a minimum enclosing rectangle for the initial target subject, and set the subject cluster connected domain within the area represented by the enclosing rectangle with the largest area as the final target subject; calculate the second possibility that other areas except the subject cluster connected domain are the final target subject; if the second possibility is greater than a second preset threshold, divide other areas into the final target subject.
[0113] In an optional embodiment, the transmission module 440 is further used to: divide the entire image into a preset number of small blocks, and perform an independent discrete cosine transform on each small block to convert the image from the spatial domain to the frequency domain; perform different degrees of quantization on different areas of the image according to the calculated quantization matrix factors; perform entropy encoding on the quantized data using Huffman coding, and encapsulate the encoding result into a file in a preset format for transmission.
[0114] In an optional implementation, the transmission module 440 is further configured to: obtain a quantization matrix factor according to the second possibility calculated for each region; and quantize the region according to the quantization matrix factor.
[0115] In an optional embodiment, the method further includes a decompression module 450, which is used to read the file header information after transmitting the image to the receiving end, obtain the parameters and quantization table used for encoding; use the Huffman table provided in the file header to decode the data stream to obtain the quantized coefficients, and then use the same quantization table to reversely restore the quantized coefficients to the original coefficients, and perform preset operations on each small block to obtain the original image data block; convert the color space and recombine all the small blocks into a complete image.
[0116] The system of this embodiment uses image processing technology to obtain the target subject of the complete drone inspection image, and compresses the image foreground and background to different degrees through image compression technology. On the basis of K-means image segmentation, according to the changes in the grayscale values of the tower body and its neighborhood, as well as some specific features of other different clusters of pixels on the image, the tower body is retained and the interfering pixels are removed to obtain a relatively complete tower body, providing technical support for subsequent image compression and transmission. The higher the degree of compression of the final target body area, the less compression of other areas, thereby reducing the file size, improving compression efficiency, saving transmission bandwidth, and saving storage space.
[0117] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0118] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
Claims
1. A method for transmitting images from a low-altitude inspection drone. It is characterized in that The method comprises: Collect low-altitude inspection drone images and perform pre-processing; Using a clustering algorithm to obtain a plurality of clusters from the preprocessed image, and determining a main cluster from the plurality of clusters; Determine adjacent pixel points according to the connected domain of the subject cluster, traverse and analyze the adjacent pixel points, and obtain the initial target subject; Establishing a minimum bounding rectangle for the initial target subject, determining the final target subject according to the size of the bounding rectangle, and marking it; The image is compressed and transmitted according to the final target subject area.
2. According to the low-altitude inspection drone image transmission method described in claim 1, It is characterized in that The collecting and preprocessing of low-altitude inspection drone images further includes: Collect low-altitude inspection drone images; The images are subjected to grayscale processing in sequence.
3. According to the low-altitude inspection drone image transmission method described in claim 2, It is characterized in that The step of obtaining a plurality of clusters from the preprocessed image by a clustering algorithm and determining a main cluster from the plurality of clusters further comprises: obtaining a plurality of clusters from the grayscale processed image by a K-means clustering algorithm; Comparing the grayscale values of pixel regions of different clusters in the image with the grayscale values of their neighbors; Clusters with grayscale values that are significantly different from those of their neighbors and / or with a large number of neighbors are extracted and segmented as the main clusters.
4. According to the low-altitude inspection drone image transmission method of claim 3, It is characterized in that The step of determining adjacent pixels based on the connected domain of the subject cluster, traversing and analyzing the adjacent pixels, and obtaining the initial target subject further includes: Determine adjacent pixels based on the connected domain of the main cluster, and for each adjacent pixel, calculate the first possibility that the pixel is the main pixel; When the first possibility is greater than a first preset threshold, the pixel point is identified as a subject pixel point; An initial target subject is determined according to the subject pixel points and the subject cluster connected domain.
5. According to the low-altitude inspection drone image transmission method of claim 4, It is characterized in that The step of determining adjacent pixels based on the main cluster connected domain further comprises: Determine other cluster pixel points around the main cluster connected domain; some of these pixel points are embedded in the main cluster connected domain, and some pixel points cut off the main cluster connected domain, resulting in local area disconnection.
6. A low-altitude inspection drone image transmission method according to claim 5, It is characterized in that The step of establishing a minimum bounding rectangle for the initial target subject and determining the final target subject according to the size of the bounding rectangle further comprises: Establishing a minimum bounding rectangle for the initial target subject, and setting the subject cluster connected domain within the area represented by the bounding rectangle with the largest area as the final target subject; Calculate the second possibility that other regions except the connected domain of the subject cluster are the final target subject; If the second possibility is greater than a second preset threshold, the other areas are divided into the final target body.
7. According to the low-altitude inspection drone image transmission method of claim 1, It is characterized in that The compressing and transmitting the image according to the final target main area further comprises: The entire image is divided into a preset number of small blocks, and each small block is independently discrete cosine transformed to transform the image from the spatial domain to the frequency domain; Quantize different regions of the image to different degrees according to the calculated quantization matrix factors; The quantized data is entropy encoded using Huffman coding, and the encoding result is packaged into a file in a preset format for transmission.
8. A low-altitude inspection drone image transmission method according to claim 7, It is characterized in that The step of quantizing different regions of the image according to the calculated quantization matrix factors to different degrees further includes: A quantization matrix factor is calculated based on the second likelihood calculated for each region; The region is quantized according to the quantization matrix factors.
9. A low-altitude inspection drone image transmission method according to claim 8, It is characterized in that The method further comprises: After the image is transmitted to the receiving end, the file header information is read to obtain the parameters and quantization table used for encoding; Decoding the data stream using the Huffman table provided by the file header to obtain quantized coefficients, then using the same quantization table to reversely restore the quantized coefficients to original coefficients, and performing a preset operation on each small block to obtain an original image data block; Convert the color space and reassemble all the small patches into a complete image.
10. A low-altitude inspection drone image transmission system, It is characterized in that The system comprises: The preprocessing module is used to collect and preprocess the images of low-altitude inspection drones; The initial target subject determination module is used to obtain multiple clusters from the preprocessed image through a clustering algorithm, and determine the main cluster from the multiple clusters; determine adjacent pixel points based on the connected domain of the main cluster, traverse and analyze the adjacent pixel points, and obtain the initial target subject; A final target subject determination module, used to establish a minimum bounding rectangle for the initial target subject, determine the final target subject according to the size of the bounding rectangle, and mark it; The transmission module is used to compress and transmit the image of the final target main area.
Citation Information
Patent Citations
Video compression method based on target detection algorithm
CN116456090A
Ship remote monitoring video efficient transmission method based on image feature matching
CN117692649A
Cited By
Power inspection method and system based on unmanned aerial vehicle
CN119580137A
A power inspection method and system based on drone
CN119580137B
Unmanned aerial vehicle inspection image transmission method and system based on asymmetric coding and decoding image compression technology
CN122027797A
Unmanned aerial vehicle inspection image transmission method and system based on asymmetric codec image compression technology
CN122027797B
