Methods and computing systems for detection

Abstract

The present application relates to methods and computing systems for detection. One method includes receiving a first image. The method also includes detecting a plurality of edges in the first image. The method also includes connecting the edges. The method also includes identifying a contour in the first image based at least in part on the connected edges. The method also includes determining a convex hull of the contour. The method also includes generating a second image that includes the convex hull.

Description

Technical Field

[0001] This disclosure relates to sky segmentation. More specifically, this disclosure relates to performing sky segmentation on image or video frames to help vehicles determine the boundary between the ground and the sky (e.g., the horizon), for navigation purposes and / or to detect objects in the vehicle's field of view. Background Technology

[0002] Sky segmentation uses machine learning models to associate pixels in an image or video frame with the sky. Sky segmentation methods are typically edge-based or content-based. Edge-based sky segmentation methods do not produce a complete pixel-by-pixel sky segmentation. Instead, they end up finding the horizon for micro-aircraft attitude estimation. Content-based sky segmentation methods require large amounts of manually labeled pixel-by-pixel data and are also computationally expensive. In addition to the aforementioned drawbacks, neither method works well under constraints of limited labeled data and real-time runtime. Summary of the Invention

[0003] A method is disclosed. The method includes receiving a first image. The method further includes detecting multiple edges in the first image. The method further includes connecting the edges. The method further includes identifying contours in the first image based at least in part on the connected edges. The method further includes determining a convex hull of the contours. The method further includes generating a second image including the convex hull.

[0004] A computing system is also disclosed. The computing system includes one or more processors and a memory system. The memory system includes one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations. The operations include receiving a first image. The first image is captured by a camera on an aircraft in flight. The operations also include detecting a plurality of edges in the first image, at least partially based on a threshold. The operations also include connecting the edges. The operations also include identifying a maximum contour in the first image, at least partially based on the connected edges. The operations also include determining a convex hull of the maximum contour. The convex hull represents a portion of the ground region in the first image. The operations also include generating a second image including the convex hull. A plurality of first pixels in the second image represent the convex hull, and a plurality of second pixels in the second image do not represent the convex hull.

[0005] In another example, the operation includes receiving a first image. The first image is captured by a camera on a vehicle. The operation also includes detecting multiple edges in the first image, at least in part based on a threshold. The operation also includes connecting the edges. Edges are connected using a shape closure with a square kernel, and the shape closure uses a kernel with an area from approximately 20 pixels to approximately 50 pixels. The operation also includes identifying a maximum contour in the first image, at least in part based on the connected edges. The maximum contour includes a maximum area. The operation also includes determining a convex hull of the maximum contour. The convex hull represents a portion of the ground region in the first image. The operation also includes generating a second image including the convex hull. A plurality of first pixels with a first color in the second image represent the convex hull, and a plurality of second pixels with a second color in the second image do not represent the convex hull. The operation also includes identifying the bottommost first pixel in each column of the second image. The operation also includes converting a second pixel below the bottommost first pixel in each column into a first pixel to produce a modified second image. The first pixels in the modified second image represent the ground region, and the second pixels in the modified second image represent the sky region. The operation also includes combining at least a portion of the first image and at least a portion of the modified second image to produce a combined image. Attached Figure Description

[0006] Various aspects of this teaching are illustrated in conjunction with the accompanying drawings, which are incorporated in and form part of this specification, and together with the description, are used to explain the principles of this teaching.

[0007] Figure 1 A schematic diagram of an aircraft in flight, based on an example, is shown.

[0008] Figure 2 A flowchart illustrating a method for segmenting an image into sky and ground regions, based on an example, is shown.

[0009] Figure 3 The image shown is a first image captured by a camera on an airplane in flight, based on an example.

[0010] Figure 4 The first image, showing multiple identified edges, is presented according to an example.

[0011] Figure 5 The first image with connected edges is shown, based on the example.

[0012] Figure 6 The first image with the largest recognizable contour is shown, based on the example.

[0013] Figure 7 A first image with a recognized convex hull is shown, based on an example.

[0014] Figure 8 A second image, according to an example, includes a plurality of first pixels (e.g., black pixels) representing a convex hull.

[0015] Figure 9 A second image is shown, based on the example, with the bottommost first pixel identified in each column (e.g., the bottommost black pixel).

[0016] Figure 10 A second image based on the example is shown, in which a second pixel (e.g., a white pixel) below the bottommost first pixel (the bottommost black pixel) is converted to the first pixel (e.g., a black pixel) to produce a modified second image.

[0017] Figure 11 An image combination based on an example is shown, which includes a modified second image overlaid on a first image.

[0018] It should be noted that some details in the accompanying drawings have been simplified and are drawn to facilitate understanding rather than to maintain strict structural accuracy, detail, and proportion. Detailed Implementation

[0019] This teaching will now be referred to in detail, examples of which are shown in the accompanying drawings. In the drawings, the same reference numerals are used throughout to designate the same elements. In the following description, reference is made to the accompanying drawings, which form a part thereof, and wherein the references are shown by way of specific examples illustrating the practice of this teaching. Therefore, the following description is merely exemplary.

[0020] The systems and methods disclosed herein can identify / generate separating boundaries between ground and sky regions in approximate images. These systems and methods can also mask non-sky (e.g., ground) regions from an image to generate a sky background, which can minimize false positives regarding boundaries and / or detected objects. The systems and methods disclosed herein can operate without training data and in real-time or near real-time to generate pixel-by-pixel sky segmentation maps. Once boundaries have been detected, the systems and methods can use these boundaries to aid in object detection (e.g., other aircraft, birds, etc.) and to guide aircraft to avoid such objects. The accuracy of the detection process can be enhanced by employing the sky segmentation techniques described herein.

[0021] The system and method can quickly and accurately detect non-sky regions with non-linear boundaries within an image. Furthermore, the system and method do not require labeled data to train a machine learning (ML) model. Instead of finding a horizon and dividing the image into "above the horizon" and "below the horizon," the system and method segment non-sky regions pixel-by-pixel.

[0022] The camera can be fixed to the aircraft at a defined position relative to the aircraft, with its field of view pointing towards the area of ​​interest (e.g., forward and / or in the direction of the aircraft's travel). The sky region can be entirely contained above the non-sky region. The non-sky region can contain more of its edge portion than the sky region. The sky and non-sky regions are separated by a horizon: a moderately straight line that traverses most to all of the image's width. Examples of systems and methods can be used to divide an image into sky and non-sky regions.

[0023] Figure 1 A schematic diagram of an aircraft 100 in flight, according to an example, is shown. Aircraft 100 can be or includes an airplane, helicopter, unmanned aerial vehicle (e.g., drone), spacecraft, etc. Aircraft 100 may include a camera 110. Camera 110 may be coupled to and / or positioned within aircraft 100. Camera 110 may be configured to capture one or more images. Camera 110 may also, or alternatively, be configured to capture video. In one example, camera 110 may be configured to capture a continuous stream of images (i.e., video) over time, and the images may be still frames extracted from the video.

[0024] Images and / or video can be transmitted to a computing system 120 on aircraft 100. In another example, computing system 120 may be located on the ground (e.g., in a control station) and communicate with an onboard computing system on / in aircraft 100. Computing system 120 may include one or more processors and a memory system. The memory system may include one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause computing system 120 to perform operations. The operations will be referred to below. Figure 2 Describe it.

[0025] Figure 2 A flowchart of a method 200 for segmenting an image into sky and ground regions, based on an example, is shown. The illustrative sequence of method 200 is described below. One or more steps of method 200 can be performed in a different order, repeated, or omitted.

[0026] Method 200 may include receiving a first image, as at 202. Figure 3 An example of the first image 300 is shown. The first image 300 can be received by the computing system 120.

[0027] As described above, the first image 300 can be captured by the camera 110. The camera 110 can be in an upright position and pointed forward, such that the first image 300 includes the region of interest in the direction in which the aircraft 100 is moving. Therefore, the first image 300 can include at least a portion of the flight path of the aircraft 100. The first image 300 can include a ground area 310, a sky area 320, or both.

[0028] Method 200 may also include detecting multiple edges in the first image 300 (three are identified: 410, 420, 430), such as at 204. Figure 4 Edges 410, 420, and 430 detected / identified in the first image 300 are shown. As used herein, an "edge" refers to a point, set of points, or line (e.g., one or more pixels) in an image where the brightness changes abruptly (e.g., greater than a predetermined threshold). Although the first image 300 includes more than three edges, only three are identified for simplicity. The steps can be performed by a computing system 120. More specifically, the computing system 120 can use Canny edge detection (e.g., with low hysteresis parameters) or Laplacian edge detection methods to detect edges 410, 420, and 430 in the first image 300. In at least one example, detection can be based at least in part on a sensitivity threshold. For example, the sensitivity threshold can be increased (e.g., by a user) to detect more edges in the first image 300, or the sensitivity threshold can be decreased to detect fewer edges in the first image 300. In at least one example, the first image 300 may be binary, and indicates that the pixels at edges 410, 420, 430 may have a value of 0, but does not indicate that the pixels at edges 410, 420, 430 may have a value of 1, or vice versa.

[0029] Method 200 may also include connecting one or more of the edges 410, 420, 430 in the first image 300, such as at 206. Figure 5 The edges 410, 420, 430 (from) are shown after the edges 510, 520, 530 that are connected to create the connection. Figure 4 The steps can be performed by the computing system 120. More specifically, the computing system 120 can use shape closure to generate the connected edges 510, 520, 530. The shape closure can use a kernel having an area of ​​approximately 2 pixels to approximately 200 pixels, approximately 5 pixels to approximately 150 pixels, approximately 10 pixels to approximately 100 pixels, or approximately 20 pixels to approximately 50 pixels. The kernel can have a shape that is substantially square, rectangular, triangular, circular, oval, elliptical, etc.

[0030] Method 200 may also include identifying one or more contours in the first image 300 (three are shown: 610, 620, 630), such as at 208. Figure 6 Contours 610, 620, and 630 in the first image 300 are shown. The steps can be performed by the computing system 120. Contours 610, 620, and 630 can be identified at least partially based on connected edges. For example, each of contours 610, 620, and 630 can have a continuous perimeter defined by connected edges. In an example, the first image 300 may include multiple contours 610, 620, and 630, and the step may include identifying a maximum contour 610, which is outlined with a shading pattern for better visibility. The maximum contour 610 may be or include the contour with the most pixels (e.g., the largest area).

[0031] Method 200 may also include determining (e.g., the largest) convex hull 710 of contour 610, such as at 210. Figure 7 The convex hull 710 of contour 610 is shown. The steps can be performed by computational system 120. The convex hull 710 is outlined with a shading pattern to demonstrate and describe step 210. In practice, such shading may or may not be used. As used herein, a “convex hull” refers to (e.g., completely) the smallest convex shape that surrounds another shape (e.g., contour 610). Therefore, a line cannot be drawn from a first point inside the shape to a second point inside the shape that falls outside the shape at some points along the line. The convex hull 710 represents a portion of the ground region 310 in the first image 300. The top 720 of the convex hull 710 may represent a horizon (e.g., between the ground region 310 and the sky region 320).

[0032] Method 200 may further include generating a second image 800, which includes a convex hull 710, such as at 212. Alternatively, the first image 300 may be modified to produce a second image 900 that includes the convex hull 710. Figure 8 A second image 800 is shown, which includes a convex hull 710. The steps can be performed by a computing system 120. The second image 800 may include a plurality of first (e.g., black) pixels 810 and a plurality of second (e.g., white) pixels 820. The first pixels 810 may represent the convex hull 710, which represents at least a portion of the ground region 310. The second pixels 820 do not represent the convex hull 710. Instead, at least a portion of the second pixels 820 may represent the sky region 320. For example, a second pixel 820 above a first pixel 810 may represent the sky region 320. However, as can be seen, some second pixels 820 are below first pixels 810. Since the sky region 320 cannot be below the ground region 310, this is corrected below.

[0033] Method 200 may also include identifying the bottommost first pixel 911 in column 910 of the second image 800, such as at 214. Figure 9 As shown. The steps can be performed by the computing system 120. This step can be repeated for each column in the second image 800. This produces the bottommost layer 920 of the first pixel 810 in the second image 800. The bottommost layer 920 is outlined with a shading pattern for easier viewing.

[0034] Method 200 may further include converting a second pixel 820 in column 910 below the bottom first pixel 911 into a first pixel 810 to produce a modified second image 1000, as shown at 216. Figure 10 As shown. The steps can be performed by the computing system 120. This corrects the aforementioned problem. Now, the first pixel 810 represents the ground region 310, and the second pixel 820 represents the sky region 320. There is no longer a portion of the sky region 320 located below the ground region 310. The modified second image 1000 may be or include a sky mask. As used herein, "sky mask" refers to a binary image where "false" (e.g., 0) values ​​indicate ground pixels and "true" (e.g., 1) values ​​indicate sky pixels.

[0035] In the example, method 200 may further include combining at least a portion of the first image 300 and the modified second image 1000 to produce a combined image 1100, as shown at 218. Figure 11 As shown. The steps can be performed by a computing system 120. In at least one example, this can include overlaying at least a portion of the modified second image 1000 onto the first image 300 to produce a combined image 1100. The overlaid portion of the modified second image 1000 substantially corresponds to the ground region 310 in the first image 300. As shown, the portion of the modified second image 1000 can be transparent or opaque, allowing the underlying first image 300 to be seen. Figure 11 In this example, the modified portion of the second image 1000 is shown with a shaded line for easier viewing. In another example, the modified portion of the second image 1000 can be solid, so that the underlying first image 300 is not visible. In yet another example, this step can be omitted.

[0036] Method 200 may also include navigating (e.g., maneuvering) aircraft 100, as at 220. Navigation (e.g., maneuvering) of aircraft 100 may be based at least in part on second image 800, modified second image 1000, combined image 1100, or a combination thereof. Navigation may be performed by computing system 120 (e.g., automatically). In another example, navigation may be performed by a user. The user may be on aircraft 100 (e.g., a pilot), or the user may be on the ground and remotely maneuvering aircraft 100.

[0037] In another example, method 200 may also, or alternatively, include detecting one or more objects, as at 222. The steps may be performed by computing system 120. Objects may be detected at least in part based on second image 800, modified second image 1000, combined image 1100, or a combination thereof. For example, second image 800, modified second image 1000, and / or combined image 1100 may be used as input to path planning or object detection algorithms. When an image is segmented into sky and non-sky regions, the object detection algorithm can detect objects more accurately and / or detect objects with fewer false positives, making it possible to detect objects above the horizon (e.g., in the sky region). Detected objects may be or include moving objects. For example, objects may be or include other aircraft in flight, and aircraft 100 may navigate in response to (e.g., to avoid) other aircraft in flight.

[0038] As used herein, the terms “inner” and “outer”; “up” and “down”; “upper” and “lower”; “upward” and “downward”; “upstream” and “downstream”; “above” and “below”; “inward” and “outward”; and other similar terms used herein refer to their relative positions to each other and are not intended to indicate a particular direction or spatial orientation. The terms “couple,” “coupled,” “connect,” “connection,” “connected,” “in connection with,” and “connecting” mean “direct connection” or “connection via one or more intermediate elements or components.” Similarly, the terms “bonded” and “bonding” mean “direct bonding” or “bonding via one or more intermediate elements, components, or layers.”

[0039] Although the numerical ranges and parameters described in this disclosure are approximate, the values ​​illustrated in the specific examples are reported as precisely as possible. However, any numerical value inherently contains a certain degree of error, which is necessarily caused by the standard deviation found in the corresponding test measurements. Furthermore, all ranges disclosed herein should be understood to encompass any and all subranges included herein.

[0040] Although this teaching has been shown with respect to one or more examples, changes and / or modifications may be made to the examples shown without departing from the spirit and scope of the appended technical solutions. Furthermore, although a particular feature of this teaching may have been disclosed with respect to only one of several examples, such feature may be combined with one or more other features of other examples, which may be desirable and advantageous for any given or particular function. As used herein, the terms “a,” “an,” and “the” may refer to one or more elements or portions of elements. As used herein, the terms “first” and “second” may refer to two different elements or portions of elements. As used herein, the term “at least one of A and B” in a list of items such as A and B means A alone, B alone, or A and B. Those skilled in the art will recognize that these and other variations are possible. Furthermore, with respect to the terms “including,” “includes,” “having,” “has,” “with,” or variations thereof used in the detailed description and technical solutions, such terms are intended to have an inclusive nature in a manner similar to the term “comprising.” Furthermore, in the discussions and technical solutions herein, the term "about" indicates that the listed values ​​may be slightly modified, as long as such modification does not cause the process or structure to deviate from the intended purpose described herein. Finally, "exemplary" indicates that the description is used as an example and not implying that it is ideal.

[0041] It should be understood that variations or alternatives to the disclosed and other features and functions, or alternatives thereof, can be combined into many other different systems or applications. Various alternatives, modifications, changes, or improvements that are not currently foreseen or anticipated can then be made by those skilled in the art, and are also intended to be covered by the appended technical solutions.

[0042] Clause 1: Includes a method for receiving a first image; detecting multiple edges in the first image; connecting the edges; identifying contours in the first image based at least in part on the connected edges; determining the convex hull of the contours; and generating a second image including the convex hull.

[0043] Clause 2: The method described in Clause 1, wherein the first image is captured by a camera on an aircraft in flight.

[0044] Clause 3: The method according to Clause 1 or 2, wherein the edges are connected using shape closure, and wherein the shape closure uses a kernel having an area from approximately 2 pixels to approximately 200 pixels.

[0045] Clause 4: The method according to any one of Clauses 1 to 3, wherein the convex hull represents a portion of the ground region in the first image.

[0046] Clause 5: The method described in Clause 4, wherein a plurality of first pixels in the second image represent a convex hull.

[0047] Clause 6: The method described in Clause 5, wherein a plurality of second pixels in the second image do not represent a convex hull.

[0048] Clause 7: The method described in Clause 6, wherein a portion of the second pixel represents a sky region in the first image.

[0049] Clause 8: The method according to Clause 6 or 7 further includes identifying the bottommost first pixel in a column of the second image; and converting a second pixel in the column below the bottommost first pixel into a first pixel to produce a modified second image.

[0050] Clause 9: The method described in Clause 8 further includes maneuvering the aircraft based at least in part on a modified second image.

[0051] Clause 10: The method described under Clause 8 or 9 further includes detecting a second aircraft in flight based at least in part on a modified second image.

[0052] Clause 11: A computing system including one or more processors; and a memory system including one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations including receiving a first image, wherein the first image is captured by a camera on an aircraft in flight; detecting a plurality of edges in the first image based at least in part on a threshold; connecting the edges; identifying a maximum contour in the first image based at least in part on the connected edges; determining a convex hull of the maximum contour, wherein the convex hull represents a portion of a ground region in the first image; and generating a second image including the convex hull, wherein a plurality of first pixels in the second image represent the convex hull, and wherein a plurality of second pixels in the second image do not represent the convex hull.

[0053] Clause 12: The computing system according to Clause 11, wherein the edges are connected using shape closure, and wherein the shape closure uses a kernel having an area from approximately 10 pixels to approximately 100 pixels.

[0054] Clause 13: The computing system according to Clause 11 or 12, wherein the operation further includes identifying the bottommost first pixel in each column of the second image; and converting a second pixel in each column below the bottommost first pixel into a first pixel to produce a modified second image, wherein the first pixel in the modified second image represents a ground region, and wherein the second pixel in the modified second image represents a sky region.

[0055] Clause 14: The computing system according to Clause 13, wherein the operation further includes combining at least a portion of the first image and at least a portion of the modified second image to produce a combined image.

[0056] Clause 15: The computing system pursuant to Clause 14, wherein operation further includes sending or displaying notifications of aircraft control based at least in part on combined images.

[0057] Clause 16: A computing system including one or more processors; and a memory system including one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations including receiving a first image, wherein the first image is captured by a camera on an aircraft in flight; detecting a plurality of edges in the first image at least in part based on a threshold; connecting the edges, wherein the edges are connected using morphological closures having a square kernel, and wherein the morphological closures use a kernel having an area from about 20 pixels to about 50 pixels; identifying a maximum contour in the first image at least in part based on the connected edges, wherein the maximum contour includes the most Large area; determining the convex hull of the largest contour, wherein the convex hull represents a portion of the ground region in the first image; generating a second image including the convex hull, wherein a plurality of first pixels having a first color in the second image represent the convex hull, and wherein a plurality of second pixels having a second color in the second image do not represent the convex hull; identifying the bottommost first pixel in each column of the second image; converting a second pixel below the bottommost first pixel in each column to a first pixel to produce a modified second image, wherein the first pixels in the modified second image represent the ground region, and wherein the second pixels in the modified second image represent the sky region; and combining at least a portion of the first image and at least a portion of the modified second image to produce a combined image.

[0058] Clause 17: The computing system according to Clause 16, wherein combining at least a portion of the first image and at least a portion of the modified second image comprises overlaying a portion of the modified second image corresponding to a ground region onto the first image.

[0059] Clause 18: The computing system according to Clause 17, wherein the modified second image is at least partially transparent when overlaid on the first image.

[0060] Clause 19: The computing system according to any one of Clauses 16 to 18 further includes detecting a second aircraft in flight based at least in part on the combined images.

[0061] Clause 20: The computing system described in Clause 19 also includes sending or displaying notifications of aircraft control based at least in part on the trajectory of the second aircraft.

Claims

1. A method (200) for detection, comprising: Receive the first image (300), wherein, The first image was captured by a camera on an aircraft in flight; Detect multiple edges (410, 420, 430) in the first image (300); Connect the edges (410, 420, 430); The maximum contour (610) in the first image (300) is identified at least in part based on the connected edges (410, 420, 430). Determine the convex hull (710) of the maximum profile (610); and A second image (800) including the convex hull (710) is generated.

2. The method according to claim 1, wherein, Generating the second image (800) includes modifying the first image (300) to generate the second image (800).

3. The method according to claim 1, wherein, The convex hull (710) represents a portion of the ground region (310) in the first image (300).

4. The method according to claim 3, wherein, The plurality of first pixels (810) in the second image (800) represent the convex hull (710).

5. The method according to claim 4, wherein, The plurality of second pixels (820) in the second image (800) do not represent the convex hull (710).

6. The method according to claim 5, wherein, A portion of the second pixel (820) represents the sky region (320) in the first image (300).

7. The method according to claim 5, further comprising: Identify the bottom first pixel (911) in column (910) of the second image (800); and The second pixel (820) below the bottom first pixel (911) in the column (910) is converted into the first pixel (810) to produce a modified second image (1000).

8. The method of claim 7, further comprising navigating the aircraft (100) based at least in part on the modified second image (1000).

9. The method of claim 7, further comprising detecting objects in space based at least in part on the modified second image (1000).

10. A computing system (120), comprising: One or more processors; as well as A memory system comprising one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations including: Receive a first image (300), wherein the first image (300) is captured by a camera (110) on an aircraft (100) in flight; Multiple edges (410, 420, 430) in the first image (300) are detected at least in part based on thresholds. Connect the edges (410, 420, 430); The maximum contour in the first image (300) is identified at least in part based on the connected edges (410, 420, 430); Determine the convex hull (710) of the maximum contour, wherein the convex hull (710) represents a portion of the ground region (310) in the first image (300); and A second image (800) including the convex hull (710) is generated, wherein a plurality of first pixels (810) in the second image (800) represent the convex hull (710), and wherein a plurality of second pixels (820) in the second image (800) do not represent the convex hull (710).

11. The computing system according to claim 10, wherein, The edges (410, 420, 430) use shape closure connections, and the shape closures use kernels with an area ranging from 10 pixels to 100 pixels.

12. The computing system according to claim 10, wherein, The operation also includes: Identify the bottom first pixel (911) in each column (910) of the second image (800); and The second pixel (820) below the bottom first pixel (911) in each column (910) is converted into a first pixel (810) to produce a modified second image (1000), wherein the first pixel (810) in the modified second image (1000) represents the ground area (310), and wherein the second pixel (820) in the modified second image (1000) represents the sky area (320).

13. The computing system according to claim 12, wherein, The operation also includes combining at least a portion of the first image (300) and at least a portion of the modified second image (1000) to produce a combined image (1100).

14. The computing system according to claim 13, wherein, The operation also includes navigating the aircraft (100) based at least in part on the combined images (1100).

15. A computing system (120), comprising: One or more processors; as well as A memory system comprising one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations including: Receive a first image (300), wherein the first image (300) is captured by a camera (110) on a vehicle; Multiple edges (410, 420, 430) in the first image (300) are detected at least in part based on thresholds. The edges (410, 420, 430) are connected using a shape closure with a square core, and the shape closure uses a core with an area of ​​20 to 50 pixels. The maximum contour (610) in the first image (300) is identified at least in part based on the connected edges (410, 420, 430), wherein the maximum contour (610) includes the maximum area; Determine the convex hull (710) of the maximum contour (610), wherein the convex hull (710) represents a portion of the ground region (310) in the first image (300); Generate a second image (800) including the convex hull (710), wherein a plurality of first pixels (810) having a first color in the second image (800) represent the convex hull (710), and wherein a plurality of second pixels (820) having a second color in the second image (800) do not represent the convex hull (710). Identify the bottom first pixel (911) in each column (910) of the second image (800); The second pixel (820) below the bottommost first pixel (911) in each column (910) is converted to the first pixel (810) to produce a modified second image (1000), wherein the first pixel (810) in the modified second image (1000) represents the ground region (310), and wherein the second pixel (820) in the modified second image (1000) represents the sky region (320); and At least a portion of the first image (300) and at least a portion of the modified second image (1000) are combined to produce a combined image (1100).

16. The computing system according to claim 15, wherein, Combining at least a portion of the first image (300) and at least a portion of the modified second image (1000) includes overlaying a portion of the modified second image (1000) corresponding to the ground region (310) onto the first image (300).

17. The computing system according to claim 16, wherein, The modified second image (1000) is at least partially transparent when overlaid on the first image (300).

18. The computing system of claim 15, further comprising detecting objects at least in part based on the combined image (1100).

19. The computing system of claim 18, further comprising enabling the vehicle to navigate at least in part based on the trajectory of the object.

Images