Methods and computing systems for detection
By using a Gaussian bimodal mixture model and the Otsu thresholding method, the problems of low efficiency and insufficient accuracy of existing sky segmentation methods are solved, achieving fast and accurate sky and horizon detection, which is suitable for navigation of autonomous vehicles.
Patent Information
- Application Number
- CN202110764457.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-07-07
- Filing Date
- 2021-07-06
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2041-07-06
AI Technical Summary
Existing sky segmentation methods are inefficient and inaccurate in determining the location of the sky and horizon, especially when the image contains only the sky or the ground, traditional methods struggle to distinguish them effectively.
A bimodal mixture model based on Gaussian distribution is adopted. The distribution type of the image is determined by edge detection and histogram analysis. The Otsu thresholding method is used to divide the image into sky and ground regions. Holes are removed by morphological processing, and the image is filtered to determine the horizon.
It improves the speed and accuracy of sky and ground area segmentation, enabling real-time sky and horizon detection, which is suitable for navigation of autonomous vehicles.
Smart Images

Figure CN113971789B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to sky segmentation. More specifically, this disclosure relates to performing sky segmentation on image or video frames to help a moving vehicle determine the position of the sky and / or horizon relative to the vehicle for navigation purposes. Background Technology
[0002] Many types of autonomous vehicles (such as cars, airplanes, and ships) need to determine which direction is up and which is down. Some vehicles use Global Positioning System (GPS) data and / or Inertial Navigation System (INS) data to determine the position of the sky and / or horizon relative to the vehicle. However, visual imagery can also be used to determine the position of the sky and / or horizon relative to the vehicle. For example, a sky segmentation process can be applied to image or video frames to determine the position of the sky and / or horizon relative to the vehicle. Sky segmentation uses machine learning models to associate pixels in an image or video frame with the sky.
[0003] Sky segmentation methods typically fall into one of three categories. One category uses color or intensity information to divide an image into two regions (e.g., ground and sky). Another category uses gradient-based methods to detect the horizon as the longest horizontal edge in the image. Yet another category uses deep learning semantic segmentation to segment the sky region. Summary of the Invention
[0004] A method for segmenting an image into sky and ground regions is disclosed. The method includes receiving or identifying a first image. The method further includes determining a distribution based at least in part on the intensity of each pixel in the first image. The method further includes determining that the distribution is bimodal. The method further includes, in response to determining that the distribution is bimodal, dividing the first image to generate a second image. The second image includes a plurality of first pixels and a plurality of second pixels. The method further includes determining that a horizon is defined between the plurality of first pixels and the plurality of second pixels.
[0005] A computing system is also disclosed. The computing system includes one or more processors and a memory system. The memory system includes one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations. The operations include identifying a first image. The operations also include identifying a plurality of edges in the first image. The operations further include determining a distribution based at least in part on the intensity of each pixel in the first image when the number of edges is greater than a predetermined threshold. The operations also include determining a bimodal mixture whose distribution approximates a Gaussian distribution. The operations further include, in response to determining the bimodal mixture whose distribution approximates a Gaussian distribution, dividing the first image to generate a second image. The second image includes a plurality of first pixels corresponding to a ground region and a plurality of second pixels corresponding to a sky region. The operations also include determining that a horizon is defined between the first pixels and the second pixels.
[0006] In another example, the operation includes identifying a first image. The image is captured by a camera on an aircraft in flight. The operation also includes identifying multiple edges in the first image. The operation further includes determining a histogram based at least in part on the intensity of each pixel in the first image when the number of edges exceeds a predetermined threshold. The operation also includes determining that the histogram approximates a bimodal mixture of Gaussian distributions more closely than a unimodal Gaussian distribution. Determining that the histogram approximates a bimodal mixture of Gaussian distributions more closely includes determining a first error between the histogram and the bimodal mixture of the Gaussian distribution corresponding to the histogram. The determination that the histogram approximates a bimodal mixture of Gaussian distributions more closely is based at least in part on the first error. In response to determining that the histogram approximates a bimodal mixture of Gaussian distributions more closely, the operation also includes dividing the first image to produce a second image. The second image includes multiple first pixels corresponding to ground regions and multiple second pixels corresponding to sky regions. The first pixels have a first color, and the second pixels have a different second color. The operation also includes filtering the second image to produce a third image. The operation also includes determining that a horizon is defined between the first and second pixels in the third image. The operation also includes combining at least a portion of the first and third images to produce a fourth image. The fourth image also includes a horizon. Attached Figure Description
[0007] Various aspects of this teaching are illustrated in conjunction with the accompanying drawings, which are incorporated in and form part of this specification, and together with the description, serve to explain the principles of this teaching.
[0008] Figure 1 A schematic diagram of an aircraft in flight, based on an example, is shown.
[0009] Figure 2 A flowchart illustrating a method for segmenting an image into sky and ground regions, based on an example, is shown.
[0010] Figure 3 The image shown is a first image captured by a camera on an airplane in flight, based on an example.
[0011] Figure 4 The first image, showing multiple identified edges, is presented according to an example.
[0012] Figure 5 The distribution (e.g., histogram) based at least in part on the first image is shown according to the example.
[0013] Figure 6 A second (e.g., segmented) image, at least in part based on the first image, is shown according to an example.
[0014] Figure 7 A second image based on the example is shown, in which a subset of the first (e.g., black) pixels is converted into second (e.g., white) pixels.
[0015] Figure 8 A second image based on the example is shown, in which a subset of the second (e.g., white) pixels is converted into the first (e.g., black) pixels.
[0016] Figure 9 The example shows a third (e.g., filtered) image representing the second image after the pixels have been transformed.
[0017] Figure 10 A fourth (e.g., combined) image, representing at least a portion of a third image overlaid on a first image, is shown, according to an example.
[0018] It should be noted that some details in the accompanying drawings have been simplified and are drawn for ease of understanding rather than to maintain strict structural accuracy, detail, and proportion. Detailed Implementation
[0019] This teaching will now be referred to in detail, examples of which are shown in the accompanying drawings. Throughout the drawings, the same reference numerals are used to designate the same elements. In the following description, reference is made to the accompanying drawings, which form a part of it, and specific examples of practicing this teaching are shown therein by way of illustration. Therefore, the following description is merely exemplary.
[0020] The system and method disclosed in this paper can be used to visually detect and locate sky regions within an image. This solution can be used in air-to-air visual inspection systems for detection and avoidance tasks. If both sky and ground are present in an image, the most distinct regions are likely the sky region and the ground region. Such images can be obtained from aircraft in the air. This has proven to be true in most cases because aircraft are high above the ground, and the ground typically loses most of its fine details, which contributes to higher intra-class dissimilarity. If only the sky is present in the image, then few strong edges will exist. Sometimes certain types of clouds can contribute to strong edges, but generally, the sky does not have strong edges. If only the ground is present in the image, the intensity representation (e.g., histogram) can often be unimodal. Therefore, the problem may include determining whether the image representation is unimodal or multimodal, and then, if it is multimodal, determining the optimal intensity threshold for dividing the image into a first (e.g., dark) region and a second (e.g., bright) region.
[0021] The systems and methods disclosed in this paper offer increased speed and accuracy superior to conventional systems and methods. For example, one method for deep semantic segmentation does not run in real time. Therefore, a significant time investment is required to create the dataset needed for ground truth analysis, as well as to actually train the algorithm. In contrast, the systems and methods disclosed in this paper do not require these time investments. Furthermore, the systems and methods disclosed in this paper lead to an intuitive definition of multimodal discrete functions.
[0022] The method may include using a Canny edge detector to determine how many strong edges are present in an image. If few edges are found, the image is determined to be entirely sky, and the function returns. The histogram of the grayscale image is then determined. The method may then include determining whether the histogram is unimodal or multimodal (e.g., bimodal). Since a histogram can represent a discrete, discontinuous function, defining the pattern as a local maximum may not be applicable. The method may then include using the Levenberg-Marquadt algorithm to approximate histograms with unimodal and bimodal Gaussian distributions. The method may then use the error between the actual histogram values and the distribution to classify the histogram as bimodal or unimodal. If the histogram is determined to be bimodal, then Otsu's thresholding method can be used to optimally divide the image into sky and ground regions, and this binary output is returned. Otherwise, the image is determined to be entirely ground, and the function returns. If the histogram is determined to be bimodal, holes in the sky and ground regions are removed morphologically.
[0023] Using Otsu thresholding to segment images containing both sky and ground is not previously used when images contain only sky or only ground. In these cases, Otsu's method still segments the image into dark and light regions, but these regions do not correspond to sky and ground. When the image histogram is discrete and unsmooth, the traditional definition of a mode as a local maxima may fail to capture the overall trend of the data. Approximating the distribution with unimodal and bimodal Gaussian functions is fast and still captures the trend of the data. The method also sets a bound on the feasible amplitude of the Gaussian function. Otherwise, a bimodal function could simply set one of the modal regions to have an amplitude of 0 and effectively become a unimodal function. The feasible region has been optimized to best capture the trend of the data.
[0024] Figure 1 A schematic diagram of an aircraft 100 in flight, according to an example, is shown. Aircraft 100 can be or includes an airplane, helicopter, unmanned aerial vehicle (e.g., drone), spacecraft, etc. Aircraft 100 may include a camera 110. Camera 110 may be coupled to aircraft 100 and / or positioned within aircraft 100. Camera 110 may be configured to capture one or more images. Camera 110 may also, or alternatively, be configured to capture video. In one example, camera 110 may be configured to capture a continuous stream of images (i.e., video) over time, and the images may be still frames obtained from the video.
[0025] Images and / or video can be transmitted to a computing system 120 on aircraft 100. In another example, computing system 120 may be located on the ground (e.g., in a control station) and communicate with an onboard computing system. Computing system 120 may include one or more processors and a memory system. The memory system may include one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations. Figure 2 Examples of the operation are provided in the document.
[0026] Figure 2 A flowchart of a method 200 for segmenting an image into sky and ground regions, according to an example, is shown. The illustrative sequence of method 200 is described below. One or more steps of method 200 can be performed, repeated, or omitted in a different order. As shown below, one or more steps of method 200 can be performed by computing system 120.
[0027] Method 200 may include receiving or recognizing a first image, as at 202. Figure 3An example of a first image 300 is shown. The first image 300 may be captured by a camera 110 and transmitted to and received / recognized by a computing system 120. The camera 110 may be in an upright position and pointed forward, such that the first image 300 is in front of the aircraft 100. Therefore, the first image 300 may include at least a portion of the flight path of the aircraft 100. The first image 300 may include a ground area 310, a sky area 320, or both. In this particular example, the first image 300 may also include occlusions (e.g., from nearby objects on the ground) 330. The first image 300 may be in color and may be converted to grayscale and / or binary (e.g., black and white).
[0028] Method 200 may also include detecting multiple edges (three identified: 410, 420, 430) in the first image 300 to produce an edge-detected first image 400, such as at 204. Figure 4 A first image 400 including edges 410, 420, and 430 is shown. As used herein, an "edge" refers to a point or line (e.g., one or more pixels) in an image where the brightness changes abruptly (e.g., greater than a predetermined threshold). Although the first image 400 for edge detection includes more than three edges, for simplicity, ... Figure 4 Only three edges are identified. This step can be performed by computing system 120. More specifically, computing system 120 can use Canny edge detection (e.g., with low hysteresis parameters) or Laplacian edge detection to detect edges 410, 420, 430. Detection can be based at least in part on a sensitivity threshold of the intensity of pixels in images 300 and / or 400. For example, the sensitivity threshold can be increased (e.g., by a user) to detect more edges in images 300 and / or 400, or decreased to detect fewer edges in images 300 and / or 400. Image 400 can be binary, and pixels representing edges 410, 420, 430 can have values of zero (e.g., white), and do not represent pixels representing edges 410, 420, 430 can have values of one (e.g., black), or vice versa.
[0029] Method 200 may further include determining whether the number of edges 410, 420, 430 in image 400 is greater than a predetermined amount / threshold, such as at 206. This step may be performed by computing system 120. When the number of edges 410, 420, 430 is less than the predetermined amount / threshold, it can be determined that image 300 and / or image 400 does not include ground region 310 and includes the entire sky region 320. Then, method 200 may loop back to step 202. When the number of edges 410, 420, 430 is greater than the predetermined amount / threshold, it can be determined that image 300 and / or image 400 includes ground region 310. For example, it can be determined that image 300 and / or image 400 includes a combination of ground region 310 and sky region 320. Then, method 200 may be performed as described below.
[0030] Method 200 may further include determining distribution 500, at least in part, based on images 300 and / or 400, as at 208. This step may be performed by computation system 120. More specifically, in response to the number of edges 410, 420, 430 being greater than a predetermined amount / threshold, computation system 120 may determine distribution 500, at least in part, based on the intensity of one or more (e.g., each) pixels in images 300 and / or 400. This is in Figure 5 As shown in the diagram. Distribution 500 can be or includes a histogram. The first image 300 can be stored in an 8-bit format, meaning there are 256 intensity possibilities for each pixel. Therefore, the X-axis of distribution 500 can range from 0 to 255, and the Y-axis can range from 0 to 1.
[0031] Method 200 may also include determining whether distribution 500 is unimodal or bimodal, as at 210. This step can be performed by computational system 120. This step may include comparing distribution 500 with a corresponding unimodal Gaussian distribution 510. This also... Figure 5 This is illustrated in the diagram. A first error can then be determined between distribution 500 and the unimodal Gaussian distribution 510. In this particular example, the first error is 4.8. This step may also include comparing distribution 500 with a bimodal mixture 520 corresponding to the Gaussian distribution of distribution 500. This is also... Figure 5 As shown in the diagram. A second error can then be determined between distribution 500 and the bimodal mixture 520 of the Gaussian distribution. In this example, the second error is 2.3.
[0032] Then, whether distribution 500 is unimodal or bimodal can be determined at least in part based on the first error and / or the second error. More specifically, distribution 500 can be determined to be unimodal when the first error is less than the second error, and to be bimodal when the second error is less than the first error. In this particular example, the second error is smaller because the bimodal mixture 520 of the Gaussian distribution more closely approximates distribution 500. Therefore, distribution 500 is determined to be bimodal, meaning that the first image 300 comprises two (e.g., different) distributions.
[0033] When distribution 500 is unimodal, it can be determined that image 300 and / or image 400 comprises a single distribution (e.g., all ground areas 310 and no sky area 320). Method 200 can then loop back to step 202. When distribution 500 is bimodal, it can be determined that image 300 and / or image 400 includes sky area 320. For example, it can be determined that image 300 and / or image 400 comprises two distributions (e.g., a combination of ground area 310 and sky area 320). Method 200 can then proceed as described below.
[0034] In response to determining that distribution 500 is bimodal, method 200 may further include partitioning image 300 and / or image 400 to produce a second (e.g., partitioned) image 600, as at 212. This step may be performed by computation system 120. More specifically, computation system 120 may partition image 300 and / or image 400 (e.g., partition into two distributions) using adaptive binary intensity thresholding or Otsu's binarization method. Figure 6 The second image 600 is shown. The second image 600 may include a plurality of first (e.g., black) pixels 610 corresponding to the ground region 310 and a plurality of second (e.g., white) pixels 620 corresponding to the sky region 320.
[0035] Method 200 may further include filtering the second image 600 to produce a third (e.g., filtered) image 900, as at 214. This step may be performed by the computing system 120. This step may include converting one or more subsets (three labeled: 611-613) of first (e.g., black) pixels 610 in the second image 600 into second (e.g., white) pixels 620. This is in Figure 7As shown in the diagram. The boundaries surrounding subsets 611-613 are shaded to make them more easily identifiable. Subsets 611-613 represent first (e.g., black) pixels 610 that are at least partially (or completely) surrounded by second (e.g., white) pixels 620. As will be understood, no part of the ground region 310 should be in or surrounded by the sky region 320, therefore this step converts those parts of the ground region 310 into the sky region 320, resulting in a continuous sky region 320.
[0036] This step may also include converting one or more subsets (three labeled: 621-623) of second (e.g., white) pixels 620 in the second image 600 into first (e.g., black) pixels 610. This is in Figure 8 As shown in the diagram. Boundaries surrounding subsets 621-623 are shaded to make them more easily identifiable. Subsets 621-623 represent second (e.g., white) pixels 620 that are at least partially (or completely) surrounded by first (e.g., black) pixels 610. As will be understood, no portion of the sky region 320 should be in or surrounded by the ground region 310; therefore, this step converts those portions of the sky region 320 into the ground region 310, resulting in a continuous ground region 310. As described above, filtering the second image 600 can produce a third (e.g., filtered) image 900, which... Figure 9 As shown in the image.
[0037] Method 200 may further include identifying a horizon 910 between the first pixel 610 and the second pixel 620 in the third image 900, such as at 216. This step may be performed by the computing system 120. The horizon 910 may be a substantially horizontal line; however, the horizon 910 may have vertical variations corresponding to mountains, valleys, buildings, trees, etc.
[0038] In an implementation, method 200 may further include combining at least a portion of the first image 300 and the third image 900 to produce a fourth (e.g., combined) image 1000, as at 218. This is in Figure 10 As shown in the figure. This step can be performed by a computing system 120. This can include overlaying at least a portion of the third image 900 onto the first image 300 to produce a fourth image 1000. The overlaid portion of the third image 900 substantially corresponds to the ground region 310 in the first image 300. The fourth image 1000 may include a horizon 910. As shown, portions of the third image 900 may be transparent or opaque, allowing the first image 300 below to be seen. Figure 10In one example, a portion of the third image 900 is shaded to make it clearer. In another example, a portion of the third image 900 can be solid, making the first image 300 underneath invisible. In yet another example, this step can be omitted.
[0039] Method 200 may also include navigating (e.g., maneuvering) aircraft 100, as at 220. Navigation (e.g., maneuvering) of aircraft 100 may be based at least in part on a third image 900, a horizon 910, a fourth image 1000, or a combination thereof. For example, the pitch and / or roll of the aircraft may be determined at least in part based on the horizon 910, and aircraft 100 may be maneuvered to adjust pitch and / or roll. Navigation may be performed by computing system 120 (e.g., automatically). In another example, navigation may be performed by a user. The user may be in aircraft 100 (e.g., a pilot), or the user may be on the ground and remotely maneuvering aircraft 100.
[0040] In another example, method 200 may also, or alternatively, include detecting one or more objects, as at 222. Objects may be detected at least in part based on a third image 900, a horizon 910, a fourth image 1000, or a combination thereof. For example, the third image 900, the horizon 910, and / or the fourth image 1000 may be used as input to a path planning or object detection algorithm. Detected objects may be or include moving objects. For example, objects may be or include other aircraft in flight, and aircraft 100 may navigate in response to (e.g., avoidance) other aircraft in flight. Objects may be continuously tracked using method 200.
[0041] As used herein, the terms “inner” and “outer”; “up” and “down”; “upper” and “lower”; “upward” and “downward”; “upstream” and “downstream”; “above” and “below”; “inward” and “outward”; and other similar terms used herein refer to their relative positions to each other, rather than to indicate a specific direction or spatial orientation. The terms “couple,” “coupled,” “connect,” “connection,” “connected,” “in connection with,” and “connecting” mean “directly connected” or “connected via one or more intermediate elements or components.” Similarly, the terms “bonded” and “bonding” mean “directly bonded to” or “bonded to via one or more intermediate elements, components, or layers.”
[0042] Although the numerical ranges and parameters described in this disclosure are approximate, the values illustrated in the specific examples are reported as precisely as possible. However, any numerical value inherently contains a certain degree of error, which is necessarily caused by the standard deviation found in the corresponding test measurement. Furthermore, all ranges disclosed herein should be understood to encompass any and all subranges contained herein.
[0043] While this teaching has been illustrated with reference to one or more examples, changes and / or modifications may be made to the illustrated examples without departing from the spirit and scope of the appended technical solutions. Furthermore, while a particular feature of this teaching may be disclosed only with respect to one example among several examples, such a feature may be combined with one or more other features of other examples, which may be desirable and advantageous for any given or particular function. As used herein, the terms “a,” “an,” and “the” may refer to one or more elements or portions of elements. As used herein, the terms “first” and “second” may refer to two distinct elements or portions of elements. As used herein, the term “at least one of A and B” relative to a list of items such as A and B means A alone, B alone, or A and B. Those skilled in the art will recognize that these and other variations are possible. Furthermore, within the scope of the use of the terms “including,” “includes,” “having,” “has,” “with,” or variations thereof in the detailed description and technical solutions, such terminology is intended to be inclusive in a manner similar to the term “comprising.” Furthermore, in the discussions and technical solutions herein, the term "about" indicates that the listed values may be slightly varied, provided that the changes do not cause the process or structure to deviate from the intended purpose described herein. Finally, "exemplary" indicates that the description is used as an example and not implying that it is ideal.
[0044] It should be understood that the above-disclosed variations and other features and functions, or alternatives thereof, can be combined into many other different systems or applications. Various alternatives, modifications, changes, or improvements that are not currently foreseen or anticipated can subsequently be made by those skilled in the art, and these alternatives, modifications, changes, or improvements are also intended to be covered by the following technical solutions.
[0045] Clause 1: A method comprising identifying a first image; determining a distribution based at least in part on the intensity of each pixel in the first image; determining that the distribution is bimodal; in response to determining that the distribution is bimodal, dividing the first image to produce a second image, wherein the second image includes a plurality of first pixels and a plurality of second pixels; and determining that a horizon is defined between the plurality of first pixels and the plurality of second pixels.
[0046] Clause 2: The method according to Clause 1, wherein the first image is captured by a camera on the aircraft in flight, and wherein the camera faces forward and covers the direction of flight.
[0047] Clause 3: The method according to Clause 1 or Clause 2 further includes identifying multiple edges in the first image, wherein the distribution is determined in response to the determination that the number of edges is greater than a predetermined threshold.
[0048] Clause 4: The method described in Clause 3, wherein the distribution includes a histogram.
[0049] Clause 5: The method according to any one of Clauses 1-4, wherein determining that the distribution is bimodal includes the error between the determined distribution and the bimodal mixture of the Gaussian distribution corresponding to the distribution.
[0050] Clause 6: The method according to any one of Clauses 1-5, wherein the first image is segmented using adaptive binary intensity thresholding.
[0051] Clause 7: The method according to any one of Clauses 1-6, wherein the first pixel corresponds to the ground region and the second pixel corresponds to the sky region.
[0052] Clause 8: The method according to any one of Clauses 1-7 further includes filtering the second image to produce a third image, wherein filtering the second image includes converting a subset of the first pixels into second pixels, and wherein a horizon is determined in the third image.
[0053] Clause 9: The method described in Clause 8, wherein a subset of the first pixel is surrounded by the second pixel.
[0054] Clause 10: The method according to any one of Clauses 1-9 further includes at least in part a horizon-based navigation aircraft.
[0055] Clause 11: A computing system comprising one or more processors; and a memory system comprising one or more non-transitory computer-readable media storing instructions, which, when executed by at least one of the one or more processors, cause the computing system to perform operations including identifying a first image; identifying a plurality of edges in the first image; determining a distribution based at least in part on the intensity of each pixel in the first image when the number of edges is greater than a predetermined threshold; determining a bimodal mixture of the distribution approximating a Gaussian distribution; in response to determining the bimodal mixture of the distribution approximating a Gaussian distribution, dividing the first image to produce a second image, wherein the second image includes a plurality of first pixels corresponding to a ground region and a plurality of second pixels corresponding to a sky region; and determining that a horizon is defined between the first pixels and the second pixels.
[0056] Clause 12: The computing system according to Clause 11, wherein the first image is captured by a camera on an aircraft in flight, and wherein the camera faces forward and covers the direction of flight.
[0057] Clause 13: The computational system according to Clause 11 or Clause 12, wherein determining a bimodal mixture of distributions approximating a Gaussian distribution comprises: determining a first error between the distribution and a unimodal Gaussian distribution corresponding to the distribution; and determining a second error between the distribution and a bimodal mixture of the distribution and a Gaussian distribution corresponding to the distribution, wherein the determination of the bimodal mixture of distributions approximating a Gaussian distribution is based at least in part on the first error and the second error.
[0058] Clause 14: The computing system according to any one of Clauses 11-13 further includes filtering the second image to produce a third image, wherein filtering the second image includes: converting a subset of first pixels into second pixels, wherein the subset of first pixels is surrounded by second pixels; and converting a subset of second pixels into first pixels, wherein the subset of second pixels is surrounded by first pixels, wherein a horizon is determined in the third image.
[0059] Clause 15: A computing system pursuant to any one of Clauses 11-14, wherein operation further comprises at least in part being based on a horizon-guided aircraft.
[0060] Clause 16: A computing system comprising: one or more processors; and a memory system including one or more non-transitory computer-readable media storing instructions, which, when executed by at least one of the one or more processors, cause the computing system to perform operations including: identifying a first image, wherein the image is captured by a camera on an aircraft in flight; identifying a plurality of edges in the first image; determining a histogram based at least in part on the intensity of each pixel in the first image when the number of edges is greater than a predetermined threshold; determining that the histogram approximates a bimodal mixture of Gaussian distributions more closely than a unimodal Gaussian distribution, wherein determining that the histogram approximates a bimodal mixture of Gaussian distributions more closely includes determining the histogram and the Gaussian distribution corresponding to the histogram. The first error between the bimodal blends of the histograms, wherein determining that the bimodal blends whose histograms more closely approximate a Gaussian distribution are at least partially based on the first error; in response to determining that the bimodal blends whose histograms more closely approximate a Gaussian distribution are bimodal, dividing a first image to produce a second image, wherein the second image includes a plurality of first pixels corresponding to a ground region and a plurality of second pixels corresponding to a sky region, wherein the first pixels have a first color and wherein the second pixels have a different second color; filtering the second image to produce a third image; determining that a horizon is defined between the first pixels and the second pixels in the third image; and combining at least a portion of the first image and the third image to produce a fourth image, wherein the fourth image also includes a horizon.
[0061] Clause 17: The computational system according to Clause 16, wherein determining that the histogram more closely approximates a bimodal mixture of Gaussian distributions further comprises determining a second error between the histogram and a unimodal Gaussian distribution corresponding to the histogram, wherein determining that the histogram more closely approximates a bimodal mixture of Gaussian distributions is based at least in part on the first error and the second error.
[0062] Clause 18: The computing system according to Clause 17, wherein filtering the second image includes converting a subset of the first pixels into second pixels, and wherein the subset of the first pixels is surrounded by the second pixels.
[0063] Clause 19: The computing system according to Clause 18, wherein filtering the second image further includes converting a subset of the second pixels into first pixels, and wherein the subset of the second pixels is surrounded by the first pixels.
[0064] Clause 20: The computing system described in Clause 19 further includes: detecting objects in a third or fourth image; and navigating the aircraft based at least in part on the trajectory of the objects.
Claims
1. A method (200) for detection, comprising: Identify the first image (300); The distribution of pixel intensity of the first image (500) is determined at least in part based on the intensity of each pixel in the first image (300). The distribution (500) is determined to be bimodal; In response to determining that the distribution is bimodal, the first image (300) is divided to produce a second image (600), wherein the second image (600) includes a plurality of first pixels (610) and a plurality of second pixels (620), wherein the first pixels (610) correspond to ground regions and the second pixels (620) correspond to sky regions; and It was determined that a horizon (910) was defined between a plurality of first pixels (610) and a plurality of second pixels (620), and The method further includes identifying a plurality of edges (410, 420, 430) in the first image (300), wherein the distribution (500) is determined in response to determining that the number of the edges (410, 420, 430) is greater than a predetermined threshold.
2. The method according to claim 1, wherein, The first image (300) is captured by a camera (110) on a flying aircraft (100), wherein the camera (110) is facing forward and covers the direction of flight.
3. The method according to claim 1, wherein, The distribution (500) includes a histogram.
4. The method according to claim 1, wherein, Determining that the distribution (500) is bimodal includes determining the error between the distribution (500) and a bimodal mixture (520) of the Gaussian distribution corresponding to the distribution (500).
5. The method according to claim 1, wherein, The first image (300) is segmented using adaptive binary intensity thresholding.
6. The method of claim 1, further comprising filtering the second image (600) to generate a third image (900), wherein, Filtering the second image (600) includes converting a subset (611-613) of the first pixels into the second pixels (620), and wherein the horizon (910) is determined in the third image (900).
7. The method according to claim 6, wherein, The subset (611-613) of the first pixel is surrounded by the second pixel (620).
8. The method according to any one of claims 1-7, further comprising navigating the aircraft (100) at least in part based on the horizon (910).
9. A computing system (120), comprising: One or more processors; as well as A memory system including one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations, the operations including: Identify the first image (300); Identify multiple edges (410, 420, 430) in the first image (300); When the number of the edges (410, 420, 430) is greater than a predetermined threshold, the distribution of pixel intensity of the first image (500) is determined at least in part based on the intensity of each pixel in the first image (300). The distribution (500) is determined to be a bimodal mixture (520) of a Gaussian distribution; In response to determining that the distribution (500) approximates a bimodal mixture (520) of the Gaussian distribution, the first image (300) is divided to produce a second image (600), wherein the second image (600) includes a plurality of first pixels (610) corresponding to the ground region and a plurality of second pixels (620) corresponding to the sky region; and It is determined that a horizon (910) is defined between the first pixel (610) and the second pixel (620).
10. The computing system according to claim 9, wherein, The first image (300) is captured by a camera (110) on a flying aircraft (100), wherein the camera (110) is facing forward and covers the direction of flight.
11. The computing system according to claim 9, wherein, Determining that the distribution (500) approximates the bimodal mixture (520) of the Gaussian distribution includes: Determine the first error between the distribution (500) and the unimodal Gaussian distribution (510) corresponding to the distribution (500); and A second error is determined between the distribution (500) and the bimodal mixture (520) of the Gaussian distribution corresponding to the distribution (500), wherein the determination of the bimodal mixture (520) of the distribution (500) approximates the Gaussian distribution is based at least in part on the first error and the second error.
12. The computing system according to claim 9, further comprising filtering the second image (600) to generate a third image (900), wherein, Filtering the second image (600) includes: A subset of the first pixels (611-613) is converted into the second pixel (620), wherein the subset of the first pixels (611-613) is surrounded by the second pixel (620); and A subset of the second pixel (621-623) is converted into the first pixel (610), wherein the subset of the second pixel (621-623) is surrounded by the first pixel (610). The horizon (910) is determined in the third image (900).
13. The computing system according to any one of claims 9-12, wherein, The operation further includes at least in part navigation of the aircraft (100) based on the horizon (910).
14. A computing system (120), comprising: One or more processors; as well as A memory system including one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations, the operations including: Identify the first image (300), wherein the first image (300) is captured by a camera (110) on an aircraft (100) in flight; Identify multiple edges (410, 420, 430) in the first image (300); When the number of the edges (410, 420, 430) is greater than a predetermined threshold, a histogram (500) representing the pixel intensity of the first image is determined at least in part based on the intensity of each pixel in the first image (300). Determining that the histogram (500) approximates a bimodal mixture of Gaussian distributions (520) more closely than a unimodal Gaussian distribution (510), wherein determining that the histogram (500) approximates a bimodal mixture of Gaussian distributions (520) more closely includes determining a first error between the histogram (500) and the bimodal mixture of Gaussian distributions (520) corresponding to the histogram (500), wherein determining that the histogram approximates a bimodal mixture of Gaussian distributions (520) more closely is based at least in part on the first error; In response to determining that the histogram (500) more closely approximates the bimodal mixture (520) of the Gaussian distribution, the first image (300) is divided to produce a second image (600), wherein the second image (600) includes a plurality of first pixels (610) corresponding to the ground region and a plurality of second pixels (620) corresponding to the sky region, wherein the first pixels (610) have a first color, and wherein the second pixels (620) have a different second color; The second image (600) is filtered to produce the third image (900); It is determined that a horizon (910) is defined between the first pixel (610) and the second pixel (620) in the third image (900); and At least a portion of the first image (300) and the third image (900) are combined to produce a fourth image (1000), wherein the fourth image (1000) further includes the horizon (910).
15. The computing system according to claim 14, wherein, Determining that the histogram (500) more closely approximates the bimodal mixture (520) of the Gaussian distribution further includes determining a second error between the histogram (500) and the unimodal Gaussian distribution (510) corresponding to the histogram (500), wherein determining that the histogram (500) more closely approximates the bimodal mixture (520) of the Gaussian distribution is at least partially based on the first error and the second error.
16. The computing system according to claim 15, wherein, Filtering the second image (600) includes converting a subset (611-613) of the first pixels into the second pixels (620), wherein the subset (611-613) of the first pixels is surrounded by the second pixels (620).
17. The computing system according to claim 16, wherein, Filtering the second image (600) further includes converting a subset (621-623) of the second pixels into the first pixel (610), wherein the subset (621-623) of the second pixels is surrounded by the first pixel (610).
18. The computing system according to claim 17, further comprising: Detect objects in the third image (900) or the fourth image (1000); and The aircraft (100) is made to navigate at least in part based on the trajectory of the object.
