Method and device for processing view image of moving object

Abstract

A method and apparatus for processing a view image of a moving object are provided. The method includes: generating a first view image of a first field of view observed from a moving object and a second view image of a second field of view observed from the moving object; determining a first occlusion area generated in the first view image by an obstacle in the first field of view, and determining a second occlusion area generated in the second view image by an obstacle in the second field of view; determining a first temporary boundary of an obstacle in the first field of view relative to a first overlap region between the first field of view and the second field of view based on the first occlusion region; determining a second temporary boundary of the obstacle in the second field of view relative to the first overlap region based on the second occlusion region; and generating a top view image of the moving object based on the first temporary boundary and the second temporary boundary.

Description

[0001] Cross - reference to related applications

[0002] This application claims priority to Korean Patent Application No. 10 - 2023 - 0188695, filed with the Korean Intellectual Property Office on December 21, 2023, the entire disclosure of which is incorporated herein by reference for all purposes. Technical field

[0003] The following description relates to a method and apparatus for processing view images of a moving object. Background art

[0004] Currently, many vehicles are equipped with a surround view monitoring (SVM) system for parking assistance. The SVM system can provide a top - view image by synthesizing images from fisheye cameras installed in various directions. A driver can identify objects that are invisible to the naked eye in the area through the SVM system. In addition, the SVM system can be implemented in application scenarios such as parking. Summary of the invention

[0005] The present invention content is provided to introduce, in a simplified form, a series of concepts further described below in the detailed description. The present invention content is not intended to identify the key features or essential features of the claimed subject matter, nor is it intended to assist in determining the scope of the claimed subject matter.

[0006] In general, a method for processing view images of a moving object includes: generating a first view image of a first field of view observed from the moving object, and generating a second view image of a second field of view observed from the moving object; determining a first occlusion region generated in the first view image based on an obstacle in the first field of view, and determining a second occlusion region generated in the second view image based on an obstacle in the second field of view; determining a first temporary boundary of the obstacle in the first field of view with respect to a first overlapping region between the first field of view and the second field of view based on the first occlusion region; determining a second temporary boundary of the obstacle in the second field of view with respect to the first overlapping region based on the second occlusion region; and generating a top - view image of the moving object based on the first temporary boundary and the second temporary boundary.

[0007] Determining the first occlusion region may include: detecting obstacle candidates located in the first view image by performing semantic segmentation based on the first view image; and determining, as the first occlusion region, a region corresponding to the obstacle in the first field of view among the detected obstacle candidates.

[0008] Determining the second occlusion area may include: detecting obstacle candidates located in the second view image by performing semantic segmentation based on the second view image; and determining, as the second occlusion area, an area corresponding to an obstacle in the second field of view among the detected obstacle candidates.

[0009] The obstacle candidates may include a vehicle, a lane, a road, or a combination thereof, and the obstacle in the first field of view may be a vehicle.

[0010] The method may include generating a first local top view image and a second local top view image by respectively warping the first view image and the second view image.

[0011] Determining the first temporary boundary may include: setting a boundary candidate with respect to the first overlapping area in the first local top view image; determining local areas of the first overlapping area by dividing the first overlapping area according to the boundary candidate; comparing a first corresponding occlusion area of the first occlusion area in the first local top view image with the local areas; and determining, based on the result of the comparison, one of the boundary candidates as the first temporary boundary.

[0012] Determining the second temporary boundary may include: setting a boundary candidate with respect to the first overlapping area in the second local top view image; determining local areas of the first overlapping area by dividing the first overlapping area according to the boundary candidate; comparing a second corresponding occlusion area of the second occlusion area in the second local top view image with the local areas; and determining, based on the result of the comparison, one of the boundary candidates as the second temporary boundary.

[0013] Comparing the first corresponding occlusion area with the local areas may include: comparing the first corresponding occlusion area with the local areas based on at least one of the area occupied by the first corresponding occlusion area in each local area and the distance between the first corresponding occlusion area shown in each local area and the representative position of the moving object relative to the first occlusion area.

[0014] Generating the top view image may include: comparing the visibility of the first overlapping area of the first view image with the visibility of the first overlapping area of the second view image, wherein the visibility of the first overlapping area of the first view image is identified by the first temporary boundary and the visibility of the first overlapping area of the second view image is identified by the second temporary boundary; and generating the top view image by selectively using one of the first view image and the second view image with respect to the first overlapping area based on the final boundary determined according to the result of the comparison.

[0015] The method may include: generating a third view image of a third field of view observed from a moving object; determining a third occlusion region generated in the third view image based on an obstacle in the third field of view; determining a third temporary boundary of the second overlapping region between the first field of view and the third field of view based on the first occlusion region; and determining a fourth temporary boundary of the second overlapping region based on the third occlusion region.

[0016] Generating a top view image may include: comparing the visibility of the second overlapping region of the first view image with the visibility of the second overlapping region of the third view image, wherein the visibility of the second overlapping region of the first view image is identified by the third temporary boundary, and the visibility of the second overlapping region of the third view image is identified by the fourth temporary boundary; and generating a top view image by selectively using one of the first view image and the third view image with respect to the second overlapping region based on the final boundary determined according to the result of the comparison.

[0017] The first field of view may be one of a front field of view, a rear field of view, a left field of view, and a right field of view of the moving object, and the second field of view may be another field of view adjacent to the first field of view among the front field of view, the rear field of view, the left field of view, and the right field of view of the moving object.

[0018] In general, an electronic device includes: one or more processors; and a memory configured to store instructions, wherein, in response to the instructions being executed by the one or more processors, the electronic device performs the following operations: generating a first view image of a first field of view observed from a moving object and a second view image of a second field of view observed from the moving object; determining a first occlusion region generated in the first view image based on an obstacle in the first field of view, and determining a second occlusion region generated in the second view image based on an obstacle in the second field of view; determining a first temporary boundary of the obstacle in the first field of view with respect to a first overlapping region between the first field of view and the second field of view based on the first occlusion region; determining a second temporary boundary of the obstacle in the second field of view with respect to the first overlapping region based on the second occlusion region; and generating a top view image of the moving object based on the first temporary boundary and the second temporary boundary.

[0019] In response to the instructions being executed by the one or more processors, the electronic device may perform the following operations to determine the first occlusion region: detecting obstacle candidates located in the first view image by performing semantic segmentation based on the first view image; and determining, as the first occlusion region, a region corresponding to an obstacle in the first field of view among the detected obstacle candidates.

[0020] When executed by one or more processors in response to an instruction, the electronic device can perform the following operations to determine a second occluded region: detecting obstacle candidates located in the second view image by performing semantic segmentation based on the second view image; and determining, as the second occluded region, a region corresponding to an obstacle in the second field of view among the detected obstacle candidates

[0021] When executed by one or more processors in response to an instruction, the electronic device can generate a first partial top view image and a second partial top view image by warping the first view image and the second view image, respectively

[0022] When executed by one or more processors in response to an instruction, the electronic device can perform the following operations to determine a first temporary boundary: setting a boundary candidate with respect to the first overlapping region in the first partial top view image; determining local regions of the first overlapping region by dividing the first overlapping region by the boundary candidate; comparing a first corresponding occluded region of a first occluded region of the first partial top view image with the local regions; and determining, as the first temporary boundary, one of the boundary candidates based on the result of the comparison

[0023] When executed by one or more processors in response to an instruction, the electronic device can perform the following operations to determine a second temporary boundary: setting a boundary candidate with respect to the first overlapping region in the second partial top view image; determining local regions of the first overlapping region by dividing the first overlapping region by the boundary candidate; comparing a second corresponding occluded region of the second occluded region in the second partial top view image with the local regions; and determining, as the second temporary boundary, one of the boundary candidates based on the result of the comparison

[0024] When executed by one or more processors in response to an instruction, the electronic device can perform the following operations to compare the first corresponding occluded region with the local regions: comparing the first corresponding occluded region with the local regions based on at least one of the area occupied by the first corresponding occluded region in each of the local regions and the distance between the first corresponding occluded region shown in each of the local regions and the representative position of the moving object with respect to the first occluded region

[0025] When executed by one or more processors in response to an instruction, the electronic device can perform the following operations to generate a top view image: comparing the visibility of the first overlapping region of the first view image with the visibility of the first overlapping region of the second view image, where the visibility of the first overlapping region of the first view image is identified by the first temporary boundary and the visibility of the first overlapping region of the second view image is identified by the second temporary boundary; and generating a top view image by selectively using one of the first view image and the second view image with respect to the first overlapping region based on a final boundary determined according to the result of the comparison

[0026] In general, a moving object includes: a first camera configured to generate a first view image of a first field of view observed from the moving object; a second camera configured to generate a second view image of a second field of view observed from the moving object; and one or more processors configured to: determine a first occlusion region generated in the first view image based on an obstacle in the first field of view and a second occlusion region generated in the second view image based on an obstacle in the second field of view; determine a first provisional boundary of the obstacle in the first field of view relative to a first overlapping region between the first field of view and the second field of view based on the first occlusion region; determine a second provisional boundary of the obstacle in the second field of view relative to the first overlapping region based on the second occlusion region; and generate a top view image of the moving object based on the first provisional boundary and the second provisional boundary.

[0027] One or more processors may be configured to: detect obstacle candidates located in the first view image by performing semantic segmentation on the first view image; and determine a region corresponding to the obstacle in the first field of view among the detected obstacle candidates as the first occlusion region.

[0028] One or more processors may be configured to: generate a first local top view image and a second local top view image by respectively warping the first view image and the second view image; set boundary candidates relative to the first overlapping region in the first local top view image; determine local regions of the first overlapping region by dividing the first overlapping region into the boundary candidates; compare a first corresponding occlusion region of the first occlusion region of the first local top view image with the local regions; and determine one of the boundary candidates as the first provisional boundary based on the result of the comparison.

[0029] Other features and aspects will become apparent from the following detailed description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Examples of various view images of a moving object according to one or more embodiments are shown.

[0031] Figure 2 Examples of occlusion regions generated by obstacles in view images according to one or more embodiments are shown.

[0032] Figure 3 Examples of angles of boundaries between adjacent fields of view according to one or more embodiments are shown.

[0033] Figure 4 Examples of a local top view image of a view image and a semantic segmentation result of the local top view image according to one or more embodiments are shown.

[0034] Figure 5A Shows an example of a boundary candidate for an overlapping region according to one or more embodiments.

[0035] Figure 5B Shows an example process of using a boundary to determine a local top - view image of an overlapping region according to one or more embodiments.

[0036] Figure 6 Shows an example process of synthesizing a local top - view image according to one or more embodiments.

[0037] Figure 7 Shows an example method of processing a view image of a moving object according to one or more embodiments.

[0038] Figure 8 Shows an example configuration of an electronic device according to one or more embodiments.

[0039] Figure 9 Shows an example configuration of a moving object according to one or more embodiments.

[0040] Throughout the drawings and the detailed description, unless otherwise described or specified, the same reference numerals can be understood to refer to the same or similar elements, features, and structures. The drawings may not be drawn to scale, and for clarity, illustration, and convenience, the relative dimensions, proportions, and depictions of elements in the drawings may be enlarged. Detailed Description

[0041] The following detailed description is provided to assist the reader in obtaining a comprehensive understanding of the methods, apparatuses, and / or systems described herein. However, after understanding the disclosure of this application, various changes, modifications, and equivalents of the methods, apparatuses, and / or systems described herein will be apparent. For example, the sequences within the operations described herein and / or the sequences of operations are merely examples and are not limited to those set forth herein, but rather will be clear that these can be changed after understanding the disclosure of this application, except that the sequences within the operations and / or the sequences of operations must occur in a certain order. As another example, the sequences of operations and / or the sequences within the operations can be executed in parallel, except that at least a portion of the sequences of operations and / or at least a portion of the sequences within the operations must occur in sequence (e.g., a specific order). Additionally, descriptions of features that are known after understanding the disclosure of this application may be omitted for greater clarity and conciseness.

[0042] Although terms such as "first", "second", and "third", or A, B, (a), (b) may be used herein to describe various members, components, regions, layers, or sections, these members, components, regions, layers, or sections are not limited by these terms. For example, each of these terms is not used to define the essence, order, or sequence of the corresponding member, component, region, layer, or section, but is only used to distinguish the corresponding member, component, region, layer, or section from other members, components, regions, layers, or sections. Thus, without departing from the teachings of the examples, the first member, component, region, layer, or section mentioned in the examples described herein may also be referred to as the second member, component, region, layer, or section.

[0043] Throughout the specification, when a component or element is described as being "on", "connected to", "coupled to", or "joined to" another component, element, or layer, the component or element may be directly "on" the other component, element, or layer, directly "connected to", directly "coupled to", or directly "joined to" the other component, element, or layer (e.g., in contact with the other component, element, or layer), or there may reasonably be one or more other components, elements, or layers therebetween. When a component or element is described as being "directly on" another component, element, or layer, "directly connected to", "directly coupled to", or "directly joined to" another component, element, or layer, there may be no other components, elements, or layers therebetween. Similarly, for example, "between" and "directly between", as well as "adjacent to" and "immediately adjacent to" may be interpreted as described above.

[0044] The singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It should also be understood that the terms "comprising / including" and / or "having / containing" when used herein specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0045] The terms used herein are for the purpose of describing various examples only and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, the articles "a," "an," and "the" are also intended to include the plural forms. As a non-limiting example, the terms "comprising" or "comprises," "including" or "includes," and "having" or "has" specify the presence of the stated features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof, or alternatively the presence of alternative stated features, quantities, operations, components, elements, and / or combinations thereof. Additionally, while one embodiment may state that the terms "comprising" or "comprises," "including" or "includes," and "having" or "has" indicate the presence of the stated features, quantities, operations, components, elements, and / or combinations thereof, there may be other embodiments in which one or more of the stated features, quantities, operations, components, elements, and / or combinations thereof are absent.

[0046] As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more thereof. Phrases such as "at least one of A, B, and C," "at least one of A, B, or C," etc. are intended to have a disjunctive meaning, and these phrases "at least one of A, B, and C," "at least one of A, B, or C," etc. also include examples in which one or more of each item of A, B, and / or C are present (e.g., any combination of one or more of each item of A, B, and C), unless the corresponding description and embodiments require such a list (e.g., "at least one of A, B, and C") to be interpreted as having a conjunctive meaning.

[0047] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. On the contrary, the examples described herein are provided only to illustrate some of the many possible ways of implementing the methods, apparatuses, and / or systems described herein, which will be apparent upon understanding the disclosure of the present application. In this document, the use of the term "may" with respect to an example or embodiment (e.g., with respect to what an example or embodiment may include or implement) means that there is at least one example or embodiment in which such a feature is included or implemented, and not all examples are limited thereto. In this document, the use of the terms "example" or "embodiment" has the same meaning (e.g., the phrase "in one example" has the same meaning as "in one embodiment," and "one or more examples" has the same meaning as "in one or more embodiments").

[0048] Figure 1 Examples of various view images of a moving object are shown.

[0049] Reference Figure 1, the first to fourth cameras 111, 112, 113, and 114 can be mounted on the moving object 100. The moving object 100 can be a movable device controlled by a driver on the moving object 100, or a movable device controlled by a driver outside the moving object 100. For example, the moving object 100 can be a vehicle, a robot, a drone, etc. (by way of example only).

[0050] The first to fourth cameras 111, 112, 113, and 114 can be respectively set to capture views in different directions. The first camera 111 can capture a first view image of a first field of view 121 observed from the moving object 100, the second camera 112 can capture a second view image of a second field of view 122 observed from the moving object 100, the third camera 113 can capture a third view image of a third field of view 123 observed from the moving object 100, and the fourth camera 114 can capture a fourth view image of a fourth field of view 124 observed from the moving object 100. The first to fourth fields of view 121, 122, 123, and 124 can respectively correspond to the viewing angles or capture ranges of the first to fourth cameras 111, 112, 113, and 114.

[0051] In an example, the first to fourth cameras 111, 112, 113, and 114 can be wide-angle cameras. In an example, the first to fourth cameras 111, 112, 113, and 114 can have a viewing angle of 90 degrees (°) or greater. In an example, the range of the viewing angles of the first to fourth cameras 111, 112, 113, and 114 can be from 180° to 220°. In an example, the first to fourth cameras 111, 112, 113, and 114 can be fish-eye cameras.

[0052] Due to the wide viewing angles of the first to fourth cameras 111, 112, 113, 114, first overlapping regions to fourth overlapping regions 131, 132, 133, 134 can be formed between adjacent fields of view in the first to fourth fields of view 121, 122, 123, 124. For example, a first overlapping region 131 can be formed between the first field of view 121 and the second view 122, a second overlapping region 132 can be formed between the first field of view 121 and the third field of view 123, a third overlapping region 133 can be formed between the second view 122 and the fourth field of view 124, and a fourth overlapping region 134 can be formed between the third field of view 123 and the fourth field of view 124.

[0053] The view images of the first to fourth fields of view 121, 122, 123, and 124 can be used to generate a top view image. For example, a partial top view image can be generated by performing image distortion on the view images of the first to fourth fields of view 121, 122, 123, and 124, or a top view image can be generated by synthesizing (e.g., merging) the partial top view images. The partial top view image can refer to an image that can form a part of the top view image. The top view image can correspond to an aerial view image.

[0054] When generating the top view image, multiple view images available in each of the first to fourth overlapping regions 131, 132, 133, and 134 can be generated. For example, a first view image of the first field of view 121 and a second view image of the second field of view 122 can be used in the first overlapping region 131. Depending on the occlusion state of each view image, at least one of the multiple available view images can be selectively used. Since the view images are selectively used by considering the occlusion state, blind spots can be minimized and the driver's visibility can be ensured.

[0055] Although Figure 1 an example of using four cameras (i.e., the first to fourth cameras 111, 112, 113, and 114) is shown, the example is not limited thereto. Various numbers of cameras (e.g., two cameras, three cameras, or five or more cameras) can be used to generate the top view image, and image synthesis based on the occlusion state can be used to generate the top view image.

[0056] Figure 2 An example of an occlusion region generated in the view image based on the presence of an obstacle is shown.

[0057] Reference Figure 2 , a first occlusion region 241 can be generated in the second view image of the second field of view 222 of the second camera 212 based on the presence of the obstacle 240. In the example, when the moving object 200 is a vehicle, the obstacle 240 can correspond to an adjacent vehicle disposed adjacent to the moving object 200 as an example. In the example, when the moving object 200 is parked, a first occlusion region 241 of the second field of view 222 can be generated based on an adjacent vehicle parked adjacent to the moving object 200. In the first view image of the first field of view 221 of the first camera 211, the first occlusion region 241 may not exist, or may appear less obvious compared to the second view image. In this example, using the first view image of the first field of view 221 to generate the top view image may be more beneficial for ensuring visibility than using the second view image of the second field of view 222 to generate the top view image.

[0058] When generating the corresponding region of the first overlapping region 231 of the top - view image of the moving object 200, the first view image can be used instead of the second view image, and the blind spot can be minimized. In the example, the first field of view 221 can be one of the front field of view, rear field of view, left field of view, and right field of view of the moving object 200, and the second field of view 222 can be another field of view adjacent to the first field of view 221 among the front field of view, rear field of view, left field of view, and right field of view of the moving object 200. In addition to the second field of view 222, the obstacle 240 can be one or more obstacles 240 that can exist in other fields of view. A view image with relatively less occlusion can be used to generate the top - view image of the overlapping region with respect to other fields of view.

[0059] Figure 3 An example of the angle showing the boundary between adjacent fields of view is shown.

[0060] Reference Figure 3 , boundaries 321, 322, 323, and 324 can be defined for the corresponding first overlapping region to fourth overlapping region 311, 312, 313, and 314. The boundaries 321, 322, 323, and 324 can be determined based on the occlusion region with respect to each of the first overlapping region to fourth overlapping region 311, 312, 313, and 314. The visibility of each of the first overlapping region to fourth overlapping region 311, 312, 313, and 314 of the view images of the first field of view to fourth field of view 301, 302, 303, and 304 can be identified by the boundaries 321, 322, 323, and 324. The boundaries 321, 322, 323, and 324 can be specified by angles θ1 to θ4.

[0061] The boundaries 321, 322, 323, and 324 can be selected from the temporary boundaries. The temporary boundaries of the view images of the first field of view to fourth field of view 301, 302, 303, and 304 can be determined based on the occlusion region of each of the view images of the first field of view to fourth field of view 301, 323, 303, and 304. The temporary boundaries can be specified by angles θ 1(x) to θ 4(x) to specify. x can represent the first field of view to fourth field of view 301, 302, 303, and 304. The boundaries 321, 322, 323, and 324 of each of the first overlapping region to fourth overlapping region 311, 312, 313, and 314 can be selected from the temporary boundaries of each of the first overlapping region to fourth overlapping region 311, 312, 313, 314. There can be two competing temporary boundaries for each of the first overlapping region to fourth overlapping region 311, 312, 313, 314, and one of the two temporary boundaries can be selected as the boundaries 321, 322, 323, 324 of each of the first overlapping region to fourth overlapping region 311, 312, 313, 314. The selected boundary can be called the final boundary.

[0062] In the example, the temporary boundaries of the first overlapping region 311 of the first view image with respect to the first field of view 301 specified by the angle θ 1(1) and the temporary boundaries of the second overlapping region 312 specified by the angle θ 2(1) can be determined, and the temporary boundaries of the first overlapping region 311 of the second view image with respect to the second field of view 302 specified by the angle θ 1(2) and the temporary boundaries of the third overlapping region 313 specified by the angle θ 3(2) can be determined. One of the temporary boundaries of the first overlapping region 311 specified by the angle θ 1(1) and the temporary boundaries of the first overlapping region 311 specified by the angle θ 1(2) can be determined as the boundary 321 of the first overlapping region 311. The boundary 321 can be referred to as the final boundary of the first overlapping region 311.

[0063] The angle θ 1(x) to θ 4(x) can represent visibility. The larger the angle θ 1(x) to θ 4(x) , the greater the visibility. For example, for the first overlapping region 311, in an example where the angle of the first temporary boundary (e.g., the angle θ 1(1) ) is greater than the angle of the second temporary boundary (e.g., the angle θ 1(2) ), this can indicate that the first view image at the angle θ 1(1) has higher visibility than the second view image at the angle θ 1(2) . In this example, instead of the second view image, the first view image can be used to generate the corresponding region of the first overlapping region 311 of the top view image. As described above, the view image can be selected based on the comparison between the temporary boundaries based on the angle θ 1(x) to θ 4(x) . However, since this is based on the definition of the angle θ 1(x) to θ 4(x) , when the definition of the angle θ 1(x) to θ 4(x) changes, the influence of the angle θ 1(x) to θ 4(x) on visibility and the selection algorithm can change.

[0064] Figure 4 Shows an example of a partial top view image of the captured view image and the semantic segmentation result of this partial top view image.

[0065] Reference Figure 4, local top - view images 411, 412, 413, and 414 can be generated based on the initially captured view images of the first field of view (e.g., front field of view), the second field of view (e.g., left field of view), the third field of view (e.g., right field of view), and the fourth field of view (e.g., rear field of view). The local top - view images 411, 412, 413, and 414 can be generated by respectively warping the initially captured view images of the first field of view (e.g., front field of view), the second field of view (e.g., left field of view), the third field of view (e.g., right field of view), and the fourth field of view (e.g., rear field of view). For example, the mapping information of the view image for the local area in the top - view image can be predetermined, and based on this mapping information, the local top - view images 411, 412, 413, and 414 are generated by warping the initially captured view images. The final top - view image can be generated by synthesizing the local top - view images 411, 412, 413, and 414. The view images of the first to fourth fields of view can correspond to examples of various view images, and the examples are not limited thereto.

[0066] Figure 4 Examples of generating the corresponding local top - view images 411, 412, 413, and 414 of the view images before generating the final top - view image are shown. However, the examples are not limited thereto. The final top - view image can be directly generated based on the initially captured view images without performing conversions on the local top - view images 411, 412, 413, and 414 of the initially captured view images. To determine the boundaries of the overlapping regions of the initially captured view images, the occlusion regions generated by the obstacles in the first to fourth fields of view in the initially captured view images can be determined.

[0067] Obstacles can be detected based on semantic segmentation. For example, a neural network can be used as an example to perform semantic segmentation. However, the examples are not limited thereto. By performing semantic segmentation based on the view image, the obstacle candidates shown in the view image can be detected. Among the obstacle candidates, the regions corresponding to the obstacles in the first to fourth fields of view can be determined as the occlusion regions. For example, the first occlusion region generated by the obstacle in the first field of view in the first view image and the second occlusion region generated by the obstacle in the second field of view in the second view image can be determined. In the example, the obstacle candidates shown in the first view image can be detected by performing semantic segmentation based on the first view image, and the region corresponding to the obstacle in the first field of view among the obstacle candidates can be determined as the first occlusion region.

[0068] At Figure 4In the example, corresponding semantic segmentation results 421, 422, 423, and 424 can be generated based on semantic segmentation of local top - view images 411, 412, 413, and 414. The semantic segmentation results 421, 422, 423, and 424 can include obstacle candidates. As a non - restrictive example, the obstacle candidates can include vehicles, lanes, roads, or combinations thereof. Obstacles can be detected from the obstacle candidates. For example, when the moving object is a vehicle, the obstacles in each field of view can be vehicles. When the moving object is a vehicle, adjacent vehicles may be the greatest obstacle to visibility. The obstacles in each field of view can be the same or different from each other.

[0069] Figure 5A An example of a boundary candidate for the overlapping region is shown.

[0070] Reference Figure 5A , the boundary candidate 551 can be used to determine the boundary of the overlapping region 550. For example, the boundary candidate 551 can be used to determine a temporary boundary of the overlapping region 550, and the boundary of the overlapping region 550 can be selected from the temporary boundary. The boundary candidate 551 can include a line passing through the representative position 501 of the moving object 500 in the overlapping region 550. For example, the representative position 501 can exist between the camera positions (e.g., the mid - point). For example, when the moving object 500 is represented as a quadrilateral, the representative position 501 can correspond to the corners of the quadrilateral.

[0071] The boundary candidate 551 can evenly divide the maximum angle formed by the boundary candidates (e.g., the first boundary candidate and the last boundary candidate) at the ends of the boundary candidate 551. The overlapping region 550 can be divided by the boundary candidate 551. The regions divided by the boundary candidate 551 can be called local regions of the overlapping region 550. For example, when the number of boundary candidates 551 is k + 1, k local regions can be formed.

[0072] Figure 5B An example process of using the boundary to determine the local top - view image with respect to the overlapping region is shown.

[0073] Reference Figure 5B, the first corresponding occlusion region 512 of the first local top - view image 510 can be determined based on the first occlusion region generated in the first view image based on the obstacles in the first field of view. The first temporary boundary 513 of the obstacles in the first field of view with respect to the first overlapping region 511 between the first field of view and the second field of view can be determined based on the first corresponding occlusion region 512. In an example, a boundary candidate with respect to the first overlapping region 511 can be set in the first local top - view image 510, the local regions of the first overlapping region 511 can be determined by dividing the first overlapping region 511 according to the boundary candidate, the first corresponding occlusion region 512 of the first local top - view image 510 can be compared with the local regions, and one of the boundary candidates can be determined as the first temporary boundary 513 based on the comparison result.

[0074] The first corresponding occlusion region 512 can be compared with the local regions based on at least one of the area occupied by the first corresponding occlusion region 512 in each local region and the distance between the first corresponding occlusion region 512 shown in each local region and the representative position 514 of the moving object 500 with respect to the first overlapping region 511. For example, when the area occupied by the first corresponding occlusion region 512 is greater than or equal to a predetermined percentage (e.g., 80%), and the distance between the first corresponding occlusion region 512 shown in each local region and the representative position 514 is less than or equal to a predetermined distance (e.g., 20 centimeters (cm)), the representative boundary candidate that covers the corresponding local region among the boundary candidates can be determined as the first temporary boundary 513. Among the boundary candidates, the boundary candidate with the smallest angle θ 1(1) can be determined as the representative boundary candidate.

[0075] Similar to determining the first temporary boundary 513, the second temporary boundary 523 of the second local top - view image 520 can be determined. More specifically, the second corresponding occlusion region 522 of the second local top - view image 520 can be determined based on the second occlusion region generated in the second view image by the obstacles in the second field of view. The second temporary boundary 523 of the obstacles in the second field of view with respect to the first overlapping region 521 between the first field of view and the second field of view can be determined based on the second corresponding occlusion region 522. For example, a boundary candidate with respect to the first overlapping region 521 can be set in the second local top - view image 520, the local regions of the first overlapping region 521 can be determined by dividing the first overlapping region 521 according to the boundary candidate, the second corresponding occlusion region 522 of the second local top - view image 520 can be compared with the local regions, and one of the boundary candidates can be determined as the second temporary boundary 523 based on the comparison result.

[0076] The second corresponding occluded region 522 can be compared with the local region based on at least one of the area occupied by the second corresponding occluded region 522 in each local region and the distance between the second corresponding occluded region 522 shown in each local region and the representative position 524 of the moving object 500 relative to the first overlapping region 521. For example, when the area occupied by the second corresponding occluded region 522 is greater than or equal to a predetermined percentage (e.g., 80%), and the distance between the second corresponding occluded region 522 shown in each local region and the representative position 524 is less than or equal to a predetermined distance (e.g., 20 cm), the representative boundary candidate in the boundary candidates that covers the corresponding local region can be determined as the second temporary boundary 523. Among the boundary candidates, the boundary candidate with the minimum angle θ 1(2) can be determined as the representative boundary candidate.

[0077] The first overlapping region 531 of the top view image 530 of the moving object 500 can be formed based on the first temporary boundary 513 and the second temporary boundary 523. The visibility of the first overlapping region 511 of the first local top view image 510 identified by the first temporary boundary 513 can be compared with the visibility of the first overlapping region 521 of the second local top view image 520 identified by the second temporary boundary 523. Based on the final boundary determined by the comparison result, the first overlapping region 531 of the top view image 530 can be formed by selectively using one of the first local top view image 510 and the second local top view image 520 with respect to the first overlapping region 531 of the top view image 530. The local top view image with higher visibility with respect to the first overlapping region 531 among the first local top view image 510 and the second local top view image 520 can be selected. One of the first temporary boundary 513 and the second temporary boundary 523 can be selected as the boundary of the first overlapping region 531 based on the visibility with respect to the first overlapping region 531.

[0078] Other overlapping regions of the top view image 530 can be formed by a process similar to the selective use process of the first partial top view image 510 and the second partial top view image 520 with respect to the first overlapping region 531. For example, when a third view image of a third field of view observed from the moving object 500 is received, a third occlusion region generated in the third view image based on the obstacles in the third field of view can be determined, a third temporary boundary with respect to the first overlapping region 521 between the first field of view and the third field of view can be determined based on the first occlusion region, and a fourth temporary boundary with respect to the first overlapping region 521 can be determined based on the third occlusion region. The visibility of the first overlapping region 521 of the first view image identified by the third temporary boundary can be compared with the visibility of the first overlapping region 521 of the third view image identified by the fourth temporary boundary, and based on the final boundary determined according to the comparison result, the first overlapping region 521 of the top view image 530 can be formed by selectively using one of the first view image and the third view image with respect to the first overlapping region 521.

[0079] Figure 6 An example process of synthesizing a partial top view image according to one or more embodiments is shown.

[0080] Reference Figure 6 , the top view image 650 of the moving object 600 can be generated by synthesizing (e.g., merging) the first partial top view image to the fourth partial top view images 610, 620, 630, and 640. A first occlusion region generated in the first partial top view image 610 based on the obstacles in the first field of view, a second occlusion region generated in the second partial top view image 620 based on the obstacles in the second field of view, a third occlusion region generated in the third partial top view image 630 based on the obstacles in the third field of view, and a fourth occlusion region generated in the fourth partial top view image 640 based on the obstacles in the fourth field of view can be determined. The obstacles in each field of view can be the same or different from each other.

[0081] A first temporary boundary of the obstacles in the first field of view with respect to the first overlapping region 651 of the top view image 650 can be determined based on the first occlusion region, and a second temporary boundary of the obstacles in the second field of view with respect to the first overlapping region 651 can be determined based on the second occlusion region. Based on the first temporary boundary and the second temporary boundary, the first overlapping region 651 of the top view image 650 can be formed by selectively using the local region 611 of the first partial top view image 610 with respect to the first overlapping region 651 and selectively using the local region 621 of the second partial top view image 620 with respect to the first overlapping region 651.

[0082] The third temporary boundary of the obstacle in the first field of view relative to the second overlapping region 652 of the top - view image 650 can be determined based on the first occlusion region, and the fourth temporary boundary of the obstacle in the third field of view relative to the second overlapping region 652 can be determined based on the third occlusion region. The second overlapping region 652 can be formed based on the third temporary boundary and the fourth temporary boundary by selectively using the local region 613 of the first local top - view image 610 relative to the second overlapping region 652 and selectively using the local region 631 of the third local top - view image 630 relative to the second overlapping region 652.

[0083] The third overlapping region 653 of the top - view image 650 can be formed by selectively using the local region 623 of the second local top - view image 620 relative to the third overlapping region 653 and selectively using the local region 641 of the fourth local top - view image 640 relative to the third overlapping region 653, and the fourth overlapping region 654 of the top - view image 650 can be formed by selectively using the local region 633 of the third local top - view image 630 relative to the fourth overlapping region 654 and selectively using the local region 643 of the fourth local top - view image 640 relative to the fourth overlapping region 654. The remaining local images 612, 622, 632, and 642 of the first local image to the fourth local image 610 to 640 can be applied to the remaining regions of the top - view image 650. The remaining regions can be referred to as non - overlapping regions, dedicated regions, or exclusive regions to distinguish the remaining regions from the first overlapping region to the fourth overlapping region 651 to 654.

[0084] Figure 7 An example method of processing view images of a moving object according to one or more embodiments is shown. Figure 7 The operations in can be performed in the order and manner shown. However, without departing from the spirit and scope of the present disclosure, the order of some operations can be changed, or some operations can be omitted. Additionally, Figure 7 some of the operations shown can be performed in parallel or simultaneously. In the example, Figures 1 to 6 the description of can also be applicable to Figure 7 and is incorporated herein by reference. Therefore, for the sake of brevity, the above description may not be repeated here.

[0085] Refer to Figure 7, in operation 710, the electronic device may receive a first view image of a first field of view observed from a moving object and a second view image of a second field of view observed from the moving object. In operation 720, the electronic device may determine a first occluded area generated in the first view image based on an obstacle in the first field of view and a second occluded area generated in the second view image based on an obstacle in the second field of view. In operation 730, the electronic device may determine a first temporary boundary of the obstacle in the first field of view with respect to a first overlapping area between the first field of view and the second field of view based on the first occluded area. In operation 740, the electronic device may determine a second temporary boundary of the obstacle in the second field of view with respect to the first overlapping area based on the second occluded area. In operation 750, the electronic device may generate a top view image of the moving object based on the first temporary boundary and the second temporary boundary.

[0086] Operation 720 may include: detecting candidate obstacles shown in the first view image by performing semantic segmentation based on the first view image, and determining an area corresponding to the obstacle in the first field of view among the candidate obstacles as the first occluded area.

[0087] Candidate obstacles may include, by way of example only, vehicles, lanes, roads, or combinations thereof, and the obstacle in the first field of view may be a vehicle.

[0088] The electronic device may generate a first partial top view image and a second partial top view image by warping the first view image and the second view image, respectively.

[0089] Operation 730 may include: setting a boundary candidate with respect to the first overlapping area in the first partial top view image; determining a local area of the first overlapping area by dividing the first overlapping area into the boundary candidate; comparing a first corresponding occluded area of the first occluded area in the first partial top view image with the local area; and determining one of the boundary candidates as the first temporary boundary based on a final boundary determined by the comparison result.

[0090] Comparing the first corresponding occluded area with the local area may include: comparing the first corresponding occluded area with the local area based on at least one of an area occupied by the first corresponding occluded area in each local area and a distance between the first corresponding occluded area shown in each local area and a representative position of the moving object with respect to the first occluded area.

[0091] Operation 750 may include: comparing the visibility of a first overlapping region of a first view image identified by a first temporary boundary with the visibility of a first overlapping region of a second view image identified by a second temporary boundary; and generating a top view image by selectively using one of the first view image and the second view image with respect to the first overlapping region based on a final boundary determined according to the comparison result.

[0092] The electronic device may receive a third view image of a third field of view observed from a moving object, determine a third occlusion region generated in the third view image based on an obstacle in the third field of view, determine a third temporary boundary with respect to a second overlapping region between the first field of view and the third field of view based on the first occlusion region, and determine a fourth temporary boundary with respect to the second overlapping region based on the third occlusion region.

[0093] Operation 750 may include: comparing the visibility of a second overlapping region of a first view image identified by a third temporary boundary with the visibility of a second overlapping region of a third view image identified by a fourth temporary boundary; and generating a top view image by selectively using one of the first view image and the third view image with respect to the second overlapping region based on a final boundary determined according to the comparison result.

[0094] The first field of view may be one of a front field of view, a rear field of view, a left field of view, and a right field of view of the moving object, and the second field of view may be another field of view adjacent to the first field of view among the front field of view, the rear field of view, the left field of view, and the right field of view of the moving object.

[0095] Figure 8 An example of the configuration of the electronic device is shown. Refer to Figure 8 , the electronic device 800 may include at least one processor 810 and a memory 820. Although Figure 8 not shown, the electronic device 800 may further include other devices, such as, by way of example, a storage device, an input device, an output device, and a network device.

[0096] The memory 820 may be connected to at least one processor 810 and may store instructions executable by the at least one processor 810, data to be computed by the at least one processor 810, data processed by the at least one processor 810, or a combination thereof. The memory 820 may include a non-transitory computer-readable storage medium (e.g., high-speed random access memory (RAM)) and / or a non-volatile computer-readable storage medium (e.g., a disk storage device, a flash device, or other non-volatile solid-state storage device).

[0097] The at least one processor 810 may execute instructions to perform reference Figures 1 to 7 and Figure 9The described operations. For example, when these instructions are executed by at least one processor 810, cause the electronic device 800 to perform the following operations: receive a first view image of a first field of view observed from a moving object and a second view image of a second field of view observed from the moving object; determine a first occlusion region generated by an obstacle in the first field of view in the first view image and a second occlusion region generated by an obstacle in the second field of view in the second view image; determine a first temporary boundary of the obstacle in the first field of view with respect to a first overlapping region between the first field of view and the second field of view based on the first occlusion region; determine a second temporary boundary of the obstacle in the second field of view with respect to the first overlapping region based on the second occlusion region; and generate a top view image of the moving object based on the first temporary boundary and the second temporary boundary.

[0098] Figure 9 An example of the configuration of a moving object (e.g., a vehicle) according to one or more embodiments is shown.

[0099] Reference Figure 9 , the moving object 900 may include a camera group 910 and at least one processor 920. The moving object 900 may also include other devices, such as, by way of example, a memory 930, a storage device 940, an input device 950, an output device 960, a network device 970, a control system 980, and a drive system 990. For example, the moving object 900 may include Figure 8 the electronic device 800.

[0100] The camera group 910 may include multiple cameras that generate view images of various fields of view observed from the moving object 900. For example, the camera group 910 may include: a first camera that generates a first view image of a first field of view observed from the moving object 900; a second camera that generates a second view image of a second field of view observed from the moving object 900; a third camera that generates a third view image of a third field of view observed from the moving object 900; and a fourth camera that generates a fourth view image of a fourth field of view observed from the moving object 900. However, the examples are not limited thereto, and various numbers of cameras (e.g., two cameras, three cameras, or five or more cameras) may be used to generate a top view image, and image synthesis based on the occlusion state may be used to generate a top view image.

[0101] At least one processor 920 may execute instructions to perform the above reference Figures 1 to 8The described operations. For example, at least one processor 920 may determine a first occlusion area generated in the first view image based on an obstacle in the first field of view, and a second occlusion area generated in the second view image based on an obstacle in the second field of view, determine a first temporary boundary of the obstacle in the first field of view relative to a first overlapping area between the first field of view and the second field of view based on the first occlusion area, determine a second temporary boundary of the obstacle in the second field of view relative to the first overlapping area based on the second occlusion area, and generate a top view image of the moving object 900 based on the first temporary boundary and the second temporary boundary.

[0102] The output device 960 may output the top view image. For example, the output device may include a display device that displays the top view image. In an example, the moving object 900 may be a vehicle having a display device, and the top view image may be provided through the display device of the vehicle. The driver may control the vehicle by referring to the top view image displayed on the display device. When the moving object 900 is a robot or a drone, the top view image itself or information for generating the top view image may be provided to a separate control device provided separately from the moving object 900 through the network device 970, and the control device may provide the top view image to the driver through the display device of the control device.

[0103] The control system 980 may control the moving object 900 and / or the drive system 990 based on the top view image. For example, the control system may control the speed and / or steering of the moving object 900 according to the object recognition result based on the top view image and / or the operation of the driver.

[0104] The cameras 111, 112, 113, 114, 211, and 212, the processor 810, the memory 820, the camera group 910, the processor 920, the memory 930, the storage device 940, the input device 950, the output device 960, the network device 970, the control system 980, the drive system 990, and other processors, memories, and communication interfaces described herein (including those described herein with respect to Figures 1 to 9The counterparts and other descriptions) are implemented by or on behalf of hardware components. As described above, or in addition to the above description, examples of hardware components that can be used to perform the operations described in the present application, where appropriate, include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in the present application. In other examples, one or more hardware components for performing the operations described in the present application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer can be implemented by one or more processing elements (e.g., logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond and execute instructions in a defined manner to achieve a desired result). In an example, a processor or computer includes one or more memories that store instructions or software executed by the processor or computer, or is connected to such one or more memories. The hardware components implemented by the processor or computer can execute instructions or software, e.g., an operating system (OS) and one or more software applications running on the OS, to perform the operations described in the present application. The hardware components can also access, manipulate, process, create, and store data in response to the execution of the instructions or software. For the sake of brevity, the singular terms "processor" or "computer" may be used in the description of the examples described in the present application, but in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. Thus, although some references may refer to a single processor or computer, these references are also intended to refer to multiple processors or computers. For example, a single hardware component or two or more hardware components can be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components can be implemented by one or more processors, or a processor and a controller, and one or more other hardware components can be implemented by one or more other processors or another processor and another controller. One or more processors or a processor and a controller can implement a single hardware component, or two or more hardware components. As described above, or in addition to the above description, example hardware components can have any one or more of different processing configurations, examples of which include single processors, independent processors, parallel processors, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing.

[0105] Figures 1 to 9 shown in and with respect to Figures 1 to 9The methods for performing the operations described in this application are performed by computing hardware, such as by one or more processors or computers, which are implemented as described above to implement instructions (e.g., computer or processor / processing device readable instructions) or software to perform the operations performed by these methods in this application. For example, a single operation or two or more operations can be performed by a single processor, or two or more processors, or a processor and a controller. One or more operations can be performed by one or more processors or a processor and a controller, and one or more other operations can be performed by one or more other processors or another processor and another controller. One or more processors or a processor and a controller can perform a single operation or two or more operations. As a non-limiting example, a reference to a processor or one or more processors configured to perform two or more operations refers to a processor or two or more processors configured to jointly perform all of the two or more operations, and a configuration in which two or more processors each perform any corresponding operations of the two or more operations (e.g., the corresponding one or more processors are configured to perform each of the two or more operations, or any corresponding combination of one or more processors is configured to perform any corresponding combination of the two or more operations).

[0106] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement the hardware components and perform the methods described above can be written as a computer program, code segment, instruction, or any combination thereof, for individually or jointly instructing or configuring one or more processors or computers to operate as a machine or special-purpose computer to perform the operations performed by the above-described hardware components and methods. In one example, the instructions or software include machine code directly executable by one or more processors or computers, such as machine code generated by a compiler. In another example, the instructions or software include higher-level code executable by one or more processors or computers using an interpreter. The instructions or software can be written in any programming language based on the block diagrams and flowcharts shown in the figures, and the corresponding descriptions herein, which disclose algorithms for performing the operations performed by the hardware components and the methods described above.

[0107] Instructions or software for controlling computing hardware (e.g., one or more processors or computers) to implement the hardware components and execute the methods described above, along with any associated data, data files, and data structures, can be recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media and thus are not signals per se. As described above, or in addition to the above description, examples of non-transitory computer-readable storage media include one or more of any of the following: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-RLTH, BD-RE, Blu-ray or optical disc storage devices, hard disk drives (HDD), solid state drives (SSD), flash memory, card-type memories (e.g., multimedia cards or micro-cards (e.g., Secure Digital (SD) or Extreme Digital (XD))), magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid state disks, and any other device configured to perform the following operations: storing instructions or software and any associated data, data files, and data structures in a non-transitory manner and providing the instructions or software and any associated data, data files, and data structures to one or more processors or computers such that the one or more processors or computers can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system such that the one or more processors or computers store, access, and execute the instructions and software and any associated data, data files, and data structures in a distributed manner.

[0108] Although the present disclosure includes specific examples, it will be apparent after understanding the disclosure of this application that various changes in form and detail can be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered only as descriptive and not for the purpose of limitation. The description of a feature or aspect in each example is considered applicable to similar features or aspects in other examples. Appropriate results can be achieved if the described techniques are performed in a different order and / or if the components in the described systems, architectures, devices, or circuits are combined in a different manner and / or replaced or supplemented by other components or their equivalents.

[0109] Accordingly, in addition to the above and as disclosed in all of the accompanying drawings, the scope of the present disclosure also includes the claims and their equivalents, that is, all variations within the scope of the claims and their equivalents should be construed as being included within the present disclosure.

Claims

1. A method for processing a view image of a moving object, comprising: generating a first view image of a first field of view observed from the moving object, and generating a second view image of a second field of view observed from the moving object; Determine a first occlusion region generated in the first view image based on an obstacle in the first field of view, and determine a second occlusion region generated in the second view image based on an obstacle in the second field of view; determining, based on the first occlusion area, a first temporary boundary of an obstacle in the first field of view relative to a first overlapping area between the first field of view and the second field of view; Determine a second temporary boundary of the obstacle in the second field of view relative to the first overlapping area based on the second occlusion area; as well as A top view image of the moving object is generated based on the first temporary boundary and the second temporary boundary.

2. The method according to claim 1, wherein: Determining the first occlusion area includes: detecting obstacle candidates located in the first view image by performing semantic segmentation based on the first view image; and determining an area among the detected obstacle candidates corresponding to the obstacle in the first field of view as the first occlusion area, Wherein, determining the second occlusion area includes: detecting obstacle candidates located in the second view image by performing semantic segmentation based on the second view image; and An area among the detected obstacle candidates corresponding to the obstacle in the second field of view is determined as the second occlusion area.

3. The method according to claim 2, wherein: The obstacle candidates include vehicles, lanes, roads, or a combination thereof, and The obstacle in the first field of view is a vehicle.

4. The method according to claim 1, further comprising: A first partial top-view image and a second partial top-view image are generated by warping the first view image and the second view image, respectively.

5. The method according to claim 4, wherein: Determining the first temporary boundary includes: setting a boundary candidate relative to the first overlapping area in the first local top-view image; Determine a local area of ​​the first overlapping area by dividing the first overlapping area according to the boundary candidates; comparing a first corresponding occlusion region of the first occlusion region in the first local top-view image with the local region; and determining one of the boundary candidates as the first temporary boundary based on a result of the comparison, Wherein, determining the second temporary boundary includes: setting a boundary candidate relative to the first overlapping area in the second local top-view image; Determine a local area of ​​the first overlapping area by dividing the first overlapping area according to the boundary candidates; comparing a second corresponding occlusion area of ​​the second occlusion area in the second local top-view image with the local area; and One of the boundary candidates is determined as the second provisional boundary based on a result of the comparison.

6. The method according to claim 5, wherein: Comparing the first corresponding occlusion area with the local area includes comparing the first corresponding occlusion area with the local area based on at least one of an area occupied by the first corresponding occlusion area in each of the local areas and a distance between the first corresponding occlusion area shown in each of the local areas and a representative position of the moving object relative to the first occlusion area.

7. The method according to claim 1, wherein: Generating the top view image comprises: comparing visibility of the first overlapping area of ​​the first view image with visibility of the first overlapping area of ​​the second view image, wherein visibility of the first overlapping area of ​​the first view image is identified by the first temporary boundary and visibility of the first overlapping area of ​​the second view image is identified by the second temporary boundary; and The top view image is generated by selectively using one of the first view image and the second view image with respect to the first overlapping area, based on a final boundary determined according to a result of the comparison.

8. The method according to claim 1, further comprising: generating a third view image of a third field of view observed from the moving object; determining a third occlusion region generated in the third view image based on an obstacle in the third field of view; determining, based on the first occlusion area, a third temporary boundary relative to a second overlapping area between the first field of view and the third field of view; as well as A fourth provisional boundary relative to the second overlapping area is determined based on the third occlusion area.

9. The method according to claim 8, wherein: Generating the top view image comprises: comparing visibility of the second overlapping area of ​​the first view image with visibility of the second overlapping area of ​​the third view image, wherein visibility of the second overlapping area of ​​the first view image is identified by the third temporary boundary and visibility of the second overlapping area of ​​the third view image is identified by the fourth temporary boundary; and The top view image is generated by selectively using one of the first view image and the third view image with respect to the second overlapping area, based on a final boundary determined according to a result of the comparison.

10. The method of claim 1, wherein: The first field of view is one of a front field of view, a rear field of view, a left field of view, and a right field of view of the moving object, and The second field of view is another field of view adjacent to the first field of view among the front field of view, the rear field of view, the left field of view, and the right field of view of the moving object.

11. A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform the method of claim 1.

12. An electronic device comprising: one or more processors; as well as a memory configured to store instructions; Wherein, in response to the instruction being executed by the one or more processors, the electronic device performs the following operations: generating a first view image of a first field of view observed from the moving object and a second view image of a second field of view observed from the moving object; Determine a first occlusion region generated in the first view image based on an obstacle in the first field of view, and determine a second occlusion region generated in the second view image based on an obstacle in the second field of view; determining, based on the first occlusion area, a first temporary boundary of an obstacle in the first field of view relative to a first overlapping area between the first field of view and the second field of view; Determining a second temporary boundary of an obstacle in the second field of view relative to the first overlapping area based on the second occlusion area; and A top view image of the moving object is generated based on the first temporary boundary and the second temporary boundary.

13. The electronic device according to claim 12, wherein: In response to the instruction being executed by the one or more processors, the electronic device is caused to perform the following operations to determine the first occlusion area: detecting obstacle candidates located in the first view image by performing semantic segmentation based on the first view image; as well as determining an area among the detected obstacle candidates corresponding to the obstacle in the first field of view as the first occlusion area, Wherein, in response to the instruction being executed by the one or more processors, the electronic device performs the following operations to determine the second occlusion area: detecting obstacle candidates located in the second view image by performing semantic segmentation based on the second view image; as well as An area among the detected obstacle candidates corresponding to the obstacle in the second field of view is determined as the second occlusion area.

14. The electronic device according to claim 12, wherein: In response to the instructions being executed by the one or more processors, the electronic device is caused to generate a first partial top-view image and a second partial top-view image by respectively warping the first view image and the second view image.

15. The electronic device according to claim 14, wherein: In response to the instructions being executed by the one or more processors, the electronic device is caused to perform the following operations to determine the first temporary boundary: setting a boundary candidate relative to the first overlapping area in the first local top-view image; determining a local area of ​​the first overlapping area by dividing the first overlapping area into the boundary candidates; comparing a first corresponding occluded area of ​​the first occluded area of ​​the first local top-view image with the local area; as well as determining one of the boundary candidates as the first temporary boundary based on a result of the comparison, Wherein, in response to the instruction being executed by the one or more processors, the electronic device performs the following operations to determine the second temporary boundary: setting a boundary candidate relative to the first overlapping area in the second local top-view image; Determine a local area of ​​the first overlapping area by dividing the first overlapping area according to the boundary candidates; comparing a second corresponding occlusion area of ​​the second occlusion area in the second local top-view image with the local area; and One of the boundary candidates is determined as the second provisional boundary based on a result of the comparison.

16. The electronic device according to claim 15, wherein: In response to the instruction being executed by the one or more processors, the electronic device performs the following operations to compare the first corresponding occlusion area with the local area: based on at least one of an area occupied by the first corresponding occlusion area in each of the local areas, and a distance between the first corresponding occlusion area shown in each of the local areas and a representative position of the moving object relative to the first occlusion area, the first corresponding occlusion area is compared with the local area.

17. The electronic device according to claim 12, wherein: In response to the instructions being executed by the one or more processors, the electronic device is caused to perform the following operations to generate the top view image: comparing visibility of the first overlapping area of ​​the first view image with visibility of the first overlapping area of ​​the second view image, wherein visibility of the first overlapping area of ​​the first view image is identified by the first temporary boundary and visibility of the first overlapping area of ​​the second view image is identified by the second temporary boundary; and The top view image is generated by selectively using one of the first view image and the second view image with respect to the first overlapping area, based on a final boundary determined according to a result of the comparison.

18. A mobile object, comprising: a first camera configured to generate a first view image of a first field of view observed from the moving object; a second camera configured to generate a second view image of a second field of view observed from the moving object; as well as One or more processors configured to: Determine a first occlusion region generated in the first view image based on an obstacle in the first field of view and a second occlusion region generated in the second view image based on an obstacle in the second field of view; determining, based on the first occlusion area, a first temporary boundary of an obstacle in the first field of view relative to a first overlapping area between the first field of view and the second field of view; Determine a second temporary boundary of the obstacle in the second field of view relative to the first overlapping area based on the second occlusion area; as well as A top view image of the moving object is generated based on the first temporary boundary and the second temporary boundary.

19. The mobile object according to claim 18, wherein: The one or more processors are configured to: detecting obstacle candidates located in the first view image by performing semantic segmentation based on the first view image; as well as An area among the detected obstacle candidates corresponding to the obstacle in the first field of view is determined as the first occlusion area.

20. The mobile object according to claim 18, wherein: The one or more processors are configured to: generating a first partial top-view image and a second partial top-view image by warping the first view image and the second view image, respectively; setting a boundary candidate relative to the first overlapping area in the first local top-view image; determining a local area of ​​the first overlapping area by dividing the first overlapping area into the boundary candidates; comparing a first corresponding occluded area of ​​the first occluded area of ​​the first local top-view image with the local area; as well as One of the boundary candidates is determined as the first provisional boundary based on a result of the comparison.

Images