Depth information determination method and device, aircraft and computer readable storage medium

Abstract

The invention discloses a depth information determination method and device, an aircraft and a computer readable storage medium, and the method comprises the steps: obtaining N images obtained when N shooting devices of the aircraft shoot a target scene at a first moment, the target scene comprises a target object, and N is greater than or equal to 3; performing stereo matching on the at least two image groups to obtain depth information of the target object; one image group comprises two images in the M images, and the M images are subsets of the N images; the at least two image groups at least comprise a first image group, an image area of the target object in the first image group has repeated textures in a first direction, an included angle between the first direction and a base line of the first binocular shooting device is greater than a first angle, and the first binocular shooting device comprises two shooting devices for shooting the first image group. Through the mode, the accuracy of depth information measurement can be improved, so that the obstacle avoidance performance of the aircraft can be improved.

Description

Technical Field

[0001] The present application relates to the field of aircraft technology, and in particular to a depth information determination method, device, aircraft, and computer-readable storage medium. Background Art

[0002] With the rapid development of drone technology, drones are increasingly being used in logistics, aerial photography, agricultural plant protection, and other fields. However, drones carry the risk of colliding with obstacles during flight, especially at low altitudes. A collision could not only cause the drone itself to crash, resulting in economic losses, but also potentially cause secondary damage to people or property on the ground, leading to serious safety accidents. To mitigate the risk of collision, drones are commonly equipped with obstacle avoidance features. Effective obstacle avoidance requires three-dimensional (3D) environmental awareness. This capability allows drones to obtain depth information about obstacles and, in turn, their relative position, enabling safe flight paths.

[0003] The accuracy of depth information directly impacts the reliability of obstacle avoidance systems. If depth measurement errors occur, drones may fail to avoid obstacles in a timely manner or make incorrect avoidance decisions, increasing the risk of collision. Therefore, accurately measuring obstacle depth information is a pressing technical challenge. Summary of the Invention

[0004] The present application discloses a depth information determination method, device, aircraft and computer-readable storage medium, which determine the depth information of a target object based on at least the stereo matching results of a first image group, thereby improving the accuracy of depth information measurement and thus improving the obstacle avoidance performance of the aircraft.

[0005] In a first aspect, an embodiment of the present application provides a depth information determination method that can be applied to an aircraft, or a device matched with an aircraft (such as a chip or processor placed inside the aircraft), the method comprising: obtaining N images of a target scene obtained by N shooting devices of the aircraft at a first moment; wherein, one shooting device captures one image, the target scene includes a target object, and N is an integer greater than or equal to 3; stereo matching is performed on at least two image groups to obtain depth information of the target object; wherein, one image group includes two images out of M images, the M images are a subset of the N images, and the M images are captured by M shooting devices out of the N shooting devices; the at least two image groups include at least a first image group, the image area of ​​the target object in the first image group has a repeated texture in a first direction, the angle between the first direction and the baseline of the first binocular shooting device is greater than the first angle, and the first binocular shooting device includes two shooting devices out of the M shooting devices that capture the first image group.

[0006] In this technical solution, the angle between the first direction and the baseline of the first binocular camera device is relatively large, that is, the first direction can be avoided from being parallel or approximately parallel to the baseline of the first binocular camera device. Furthermore, the first direction can be avoided from being coincident or approximately coincident with the epipolar line of the first image group. In this way, even if there is repeated texture in the image area of ​​the target object in the first direction, during the stereo matching process of the first image group, the angle between the direction searched along the epipolar line and the first direction is relatively large. Therefore, stereo matching ambiguity can be avoided during the stereo matching process of the first image group, which is beneficial to improving the accuracy of the depth information of the target object.

[0007] Taking the target object as a linear obstacle as an example, the first direction is the extension direction of the linear obstacle. The aircraft uses at least a pair of binocular cameras (i.e., the first binocular camera) with a large angle between the baseline and the linear obstacle in the image. This is conducive to avoiding stereo matching ambiguity during the stereo matching process, thereby improving the measurement accuracy of the depth information of the linear obstacle.

[0008] In conjunction with the first aspect, in one implementation, L pairs of binocular camera devices consisting of M camera devices meet a first condition, and a pair of binocular camera devices consists of two camera devices; wherein the first condition includes: the baselines of the L pairs of binocular camera devices are not parallel, and / or the baselines of the L pairs of binocular camera devices do not belong to the same plane.

[0009] In this technical solution, at least one of the L pairs of binocular cameras can have a baseline that is not parallel to the first direction. In this way, even if at least one of the L pairs of binocular cameras has a baseline that is parallel to the first direction, causing stereo matching ambiguity, the binocular camera with a baseline that is not parallel to the first direction can eliminate the stereo matching ambiguity, thereby improving the accuracy of the depth information of the target object.

[0010] In combination with the first aspect, in one implementation, the shooting angles of the M shooting devices match the flight direction of the aircraft at the first moment.

[0011] In this technical solution, the camera angle corresponding to a particular flight direction allows for better observation of obstacles in that direction. By selecting M cameras that match the aircraft's flight direction at a first moment, and using the images captured by these M cameras (i.e., M images) to determine the depth information of the target object, the M images provide a clearer image of the target object, thereby improving the accuracy of the target object's depth information.

[0012] In combination with the first aspect, in one implementation, the at least two image groups also include a second image group; if a stereo matching ambiguity is detected during stereo matching of the second image group, the depth information of the target object is determined based on the matching result obtained by stereo matching of the first image group.

[0013] In combination with the first aspect, in one implementation, the two shooting devices that capture the second image group are located on the left side and the right side of the nose of the aircraft, respectively.

[0014] In combination with the first aspect, in one implementation, the angle between the projections of the two optical axes of the second binocular camera device on the first plane is a second angle, wherein the first plane is a plane formed by the roll axis and pitch axis of the aircraft, and the second angle is less than 180 degrees and greater than or equal to 90 degrees; the angle between the two optical axes of the second binocular camera device and the positive direction of the yaw axis of the aircraft is a third angle, and the third angle is greater than 45 degrees and less than or equal to 90 degrees.

[0015] This technical solution allows the second binocular camera (i.e., the one that captures the second set of images) to cover both the forward and upward fields of view of the aircraft, thereby facilitating more timely detection of obstacles in the forward direction. Furthermore, if the aircraft's camera is a fisheye camera, the fisheye camera's actual effective field of view may not reach its theoretical field of view, resulting in a limited upper field of view covered by the second binocular camera (i.e., the front binocular camera). By tilting the second binocular camera upward, this helps ensure the integrity of the multi-view coverage above the aircraft.

[0016] In combination with the first aspect, in one implementation, one of the first binocular camera devices is located on the upper side of a nose, the lower side of a nose, or the lower side of a fuselage of the aircraft.

[0017] In combination with the first aspect, in one implementation, an angle between an optical axis of one of the first binocular camera devices and the yaw axis of the aircraft is a fourth angle, and the fourth angle is greater than 0 degrees and less than or equal to 45 degrees.

[0018] In this technical solution, when the aircraft's shooting device is a fisheye camera, the actual effective field of view angle of the fisheye camera may not reach the theoretical field of view angle, which will result in limited forward field of view covered by the shooting devices above and below the aircraft. By tilting the upper shooting device and / or the lower shooting device forward, it is helpful to ensure the integrity of the multi-eye coverage in front of the aircraft.

[0019] In combination with the first aspect, in one implementation, the method further includes: replacing the first instruction with a second instruction, wherein the first instruction is used to instruct the aircraft to fly backward, and the second instruction is used to instruct the aircraft to turn around and fly forward.

[0020] In this technical solution, the forward tilt of the upper and / or lower camera devices results in an area behind the aircraft that cannot be covered by at least three camera devices. The aircraft may not be able to accurately measure the depth information of target objects (such as linear obstacles) in this area. If it flies directly in sequence, it may not be able to circumvent the target object. By replacing the backward flight instruction with a U-turn and then forward flight, the aircraft's forward field of view is covered by at least three camera devices, so the depth information of target objects in this area can be accurately obtained, thereby improving flight safety.

[0021] In combination with the first aspect, in one implementation, stereo matching is performed on at least two image groups to obtain depth information of the target object, including: when the aircraft meets the second condition, stereo matching is performed on at least two image groups to obtain depth information of the target object; wherein the second condition includes one or more of the following: it is detected that the target scene includes the target object, and there are repeated textures on the surface of the target object; the aircraft enters a matching ambiguity prevention mode; the aircraft is performing a first mission, the scene for performing the first mission includes the target object, and there are repeated textures on the surface of the target object.

[0022] In a second aspect, an embodiment of the present application provides a depth information determination device, which includes a unit for implementing the method described in the first aspect.

[0023] In a third aspect, an embodiment of the present application provides an aircraft, comprising a processor and N shooting devices; the processor is used to execute the method described in the first aspect.

[0024] In an optional embodiment, the aircraft may further include a memory; the memory is used to store computer programs or instructions; and the processor is specifically used to call the computer program or instructions from the memory to execute the method described in the first aspect.

[0025] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program or computer instructions, and when the computer program or computer instructions are executed, the aircraft executes the method described in the first aspect.

[0026] In a fifth aspect, an embodiment of the present application provides a computer program product comprising a computer program or instructions, which, when executed on an aircraft, enables the aircraft to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a schematic diagram of an application scenario applicable to the embodiments of the present application;

[0028] Figure 2 This embodiment of the present application provides a Figure 1 A top view of the aircraft 10 and the target scene is shown;

[0029] Figure 3 This embodiment of the present application provides a Figure 1 A bottom view of the aircraft 10 is shown;

[0030] Figure 4 This is a flow chart of a method for determining depth information provided by an embodiment of the present application;

[0031] Figure 5 2 is a schematic diagram of projections of optical axes of a camera a and a camera e on a first plane provided in an embodiment of the present application;

[0032] Figure 6 is a schematic diagram of an optical axis of a shooting device a provided in an embodiment of the present application;

[0033] Figure 7 is a schematic diagram of an optical axis of a shooting device b provided in an embodiment of the present application;

[0034] Figure 8 is a structural diagram of a depth information determination device provided in an embodiment of the present application;

[0035] Figure 9 It is a structural schematic diagram of an aircraft provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] It should be understood that the terms "first", "second", etc. involved in the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. "At least one" in the embodiments of the present application refers to one or more, and "a plurality" refers to two or more. "And / or" in the embodiments of the present application describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. Among them, A and B can be singular or plural. The character " / " can indicate that the previous and subsequent associated objects are in an "or" relationship. In addition, the symbol " / " can also represent a division sign, that is, performing a division operation.

[0037] In the embodiments of the present application, "at least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or multiple items. For example, at least one of a, b, or c can represent the following seven situations: a, b, c, a and b, a and c, b and c, a, b, and c. Each of a, b, and c can be an element or a set containing one or more elements.

[0038] See Figure 1 , Figure 1 This is a schematic diagram of an application scenario applicable to the embodiments of the present application. The application scenario may include, but is not limited to, an aircraft and at least one target object, wherein the aircraft includes multiple shooting devices. Figure 1 The number and form of the equipment, devices, and objects shown are for illustrative purposes only and do not constitute a limitation on the embodiments of the present application. Actual applications may include two or more aircraft and more objects. Figure 1 The application scenario shown is an example including an aircraft 10 and a target object 11. Figure 1 As shown, the right side of the nose of the aircraft 10 includes a camera a, and the upper side of the nose of the aircraft 10 includes a camera b. Optionally, the lower side of the nose of the aircraft 10 includes a camera c. Optionally, the right side of the tail of the aircraft 10 includes a camera d.

[0039] It should be noted that Figure 1 The right side view of the aircraft 10 and the target scene is shown, which only illustrates the camera that can be observed on the right side. In other possible implementations, the left side of the nose of the aircraft 10 may also include a camera ( Figure 1 Not shown), the lower side of the nose of the aircraft 10 may include other photographic devices ( Figure 1 not shown).

[0040] In one possible implementation, Figure 1 The aerial vehicle 10 and the overhead view of the target scene can be shown as Figure 2 As shown. Figure 2 As shown, the left side of the nose of the aircraft 10 may further include a photographing device e. Optionally, the left side of the tail of the aircraft 10 may further include a photographing device f.

[0041] In one possible implementation, Figure 1 The bottom view of the aircraft 10 can be shown as Figure 3 As shown. Figure 3 As shown, the lower side of the fuselage of the aircraft 10 may include a photographing device c. Optionally, the lower side of the fuselage of the aircraft 10 may also include a photographing device g.

[0042] Aircraft 10 may include, but is not limited to, unmanned aerial vehicles and manned aerial vehicles. This embodiment of the present application does not limit the form of aircraft 10. For example, aircraft 101 may include, but is not limited to, fixed-wing aircraft, rotorcraft, model aircraft, airship, hot air balloon, and robot. Rotorcraft may include, but is not limited to, helicopters and multi-rotor aircraft.

[0043] The movement of an aircraft in three-dimensional space can be carried out along three mutually perpendicular axes, such as Figure 1 As shown in FIG, the three axes are roll axis (Roll), pitch axis (Pitch), and yaw axis (Yaw).

[0044] The roll axis is an axis that runs across the front and rear of the aircraft. The nose of the aircraft points in the positive direction of the roll axis, while the tail of the aircraft points in the negative direction. Rotation about the roll axis is called roll, and the aircraft can tilt left or right by rotating about it.

[0045] The pitch axis is an axis running left to right along the aircraft. Rotation around the pitch axis is called pitch. Rotating the aircraft around the pitch axis causes the nose of the aircraft to rise or fall, similar to how an airplane tilts its head up or down.

[0046] The yaw axis is perpendicular to the aircraft's fuselage and is typically perpendicular to the ground. The positive yaw direction is upward, while the negative yaw direction is downward. Rotation about the yaw axis is called yaw. Rotating an aircraft about the yaw axis can cause the nose of the aircraft to turn left or right, similar to how an airplane changes direction in the horizontal plane.

[0047] The shooting devices in the embodiments of the present application (such as shooting devices a-shooting devices g) have the function of capturing images and / or videos. The embodiments of the present application do not limit the number, installation location, and form of the shooting devices. For example, the form of the shooting device may include but is not limited to: fisheye camera, wide-angle camera, pinhole camera, etc. Optionally, the field of view (FOV) of the shooting device in the embodiments of the present application may be greater than or equal to 180 degrees. The field of view refers to the maximum angular range that a lens or sensor can capture, and the field of view determines the width of the scene that the shooting device "sees".

[0048] The scene the aircraft is facing refers to the actual environment or scene that can be captured by the aircraft's camera. The scene can be natural scenery, urban buildings, farmland, forests, rivers, and any other environment that the aircraft sees during flight.

[0049] A target scene refers to a scene containing a target object. Target object 11 may include, but is not limited to, movable objects and immovable objects. This embodiment of the application does not limit the form of target object 11. For example, the form of target object 11 may include, but is not limited to, cable lines, buildings, roads, bridges, mountains, forests, rivers, plants, and animals.

[0050] The surface of the target object 11 has a repetitive texture. In other words, the surface of the target object 11 has highly similar images or periodic patterns. For example, the target object 11 may include but is not limited to: wires, rolling doors, blinds, and escalator steps.

[0051] It can be understood that the application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art will know that with the evolution of technology and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0052] The depth information determination method proposed in the embodiment of the present application is described below. The depth information determination method can be executed by an aircraft, or by a device matched with the aircraft (such as a chip or processor placed inside the aircraft). The embodiment of the present application takes the depth information determination method executed by an aircraft as an example for explanation. The aircraft can be Figure 1-Figure 3 Aircraft 10 in.

[0053] See Figure 4 , Figure 4 This is a flow chart of a method for determining depth information provided in an embodiment of the present application. Figure 4 As shown, the depth information determination method may include but is not limited to the following steps.

[0054] S401: Acquire N images of a target scene captured by N photographing devices of an aircraft at a first moment; wherein one photographing device captures one image, and the target scene includes a target object.

[0055] N is an integer greater than or equal to 3, so that the target object can be observed and photographed by at least N photographing devices of the aircraft.

[0056] The N cameras may include all or part of the aircraft's cameras. For example, taking N=3 as an example, the N cameras may include Figure 2 The photographing devices a, b and e shown, or N photographing devices may include Figure 1 The photographing device a, photographing device b and photographing device c are shown. For another example, taking N=4 as an example, the N photographing devices may include Figure 1 The photographing device a, photographing device b, photographing device c and Figure 2 The shooting device e shown.

[0057] In one possible implementation, all or some of the aircraft's camera devices may capture images at the same time. It is understood that different camera devices on the aircraft may have different viewing angles, and therefore, the elements in images captured by different camera devices on the aircraft at the same time may differ. This embodiment of the present application uses an example in which N camera devices on the aircraft can capture images at the same time (i.e., a first time point) and all capture a target object in a target scene.

[0058] S402: Perform stereo matching on at least two image groups to obtain depth information of a target object; wherein one image group includes two images from M images, where the M images are a subset of N images, and the M images are captured by M cameras from N cameras; the at least two image groups include at least a first image group, where an image region of the target object in the first image group has a repetitive texture in a first direction, an angle between the first direction and a baseline of a first binocular camera is greater than a first angle, and the first binocular camera includes two cameras from the M cameras that captured the first image group.

[0059] In an embodiment of the present application, each of the N cameras of the aircraft can capture an image of a target object. The aircraft can determine the depth information of the target object based on images captured by M of the N cameras (i.e., M images). The M cameras are a subset of the N cameras, i.e., the M cameras include some or all of the N cameras. Similarly, the M images are a subset of the N images, i.e., the M images include some or all of the N images.

[0060] M images can be organized into at least two image groups (hereinafter referred to as L image groups, where L is an integer greater than or equal to 2). Each image group includes two of the M images. Similarly, M cameras can be organized into at least two pairs of binocular cameras (hereinafter referred to as L pairs of binocular cameras). A pair of binocular cameras consists of two of the M cameras. M is greater than or equal to 3, less than or equal to N, and is an integer.

[0061] The aircraft can determine the depth information of the target object based on the stereo matching results of the L image groups. In other words, the depth information of the target object can be determined based on images of the target object captured by at least three cameras of the aircraft. This helps reduce the probability of detecting stereo matching ambiguities during stereo matching of images containing the target object, thereby improving the accuracy of the depth information.

[0062] Stereo matching is a computer vision technology. Stereo matching is the process of finding matching points in images (i.e., points on the target object) by comparing two images. These corresponding points represent the same physical point in three-dimensional space (i.e., the same point on the target object) in the images from the two perspectives. Take an image group in L image groups as an example, where the two images are image a and image b, and both image a and image b include images of the target object. Assuming that the reference image required for stereo matching is image a, the stereo matching process is as follows: specify a point of the target object on the reference image (i.e., image a), search for the matching point corresponding to the point on image b along the epipolar line, and obtain the disparity based on the horizontal coordinates of the two points. Furthermore, based on the disparity and the parameters of the shooting device that captured images a and b, the depth of the target object is calculated. For example, depth = (focal length * baseline length) / disparity.

[0063] In stereo matching, an epipolar line refers to the search path for matching points in another image. In binocular stereo vision, for a point in a reference image, its corresponding matching point in the other view must lie on a specific straight line in that view, called the epipolar line. The position of the epipolar line can be determined by the geometric relationship between the cameras (intrinsic and extrinsic parameters) and the baseline of the two cameras. The baseline of the two cameras is the line connecting the optical centers of the two cameras.

[0064] In the embodiment of the present application, the two images in each of the L image groups may be epipolar corrected.

[0065] The internal parameters of the shooting device are independent of the external environment of the shooting device, and define the relationship between the coordinate system of the shooting device and the image coordinate system. The internal parameters may include but are not limited to the following parameters: the focal length of the lens, the coordinates of the principal point, and the distortion parameters. Among them, the principal point coordinates refer to the coordinates of the intersection of the origin of the image coordinate system (usually at the center of the image) and the optical axis of the shooting device on the image plane. Distortion parameters include radial distortion and tangential distortion, etc. The external parameters of the shooting device describe the relative position and direction between the coordinate system of the shooting device and a global reference coordinate system (usually the world coordinate system). The external parameters may include but are not limited to the following parameters: rotation (R) and translation (t). Among them, rotation (R) is used to describe the rotation of the coordinate system of the shooting device relative to the world coordinate system. Translation (t) is used to describe the position of the origin of the coordinate system of the shooting device in the world coordinate system.

[0066] Disparity is the pixel-level difference between the positions of corresponding points in two images of a 3D scene. Disparity can be measured in pixels. The formula for converting pixel disparity to real-distance disparity is: dphysical = dpixels * pixel pitch. Here, dpixels represents pixel disparity, or the disparity value in pixels; dphysical represents real-distance disparity, or the disparity value in physical distance; and pixel pitch is the physical size of each pixel on the camera sensor.

[0067] During the stereo matching process, the following situation may occur: the same pixel in the reference image has multiple matching points in the other image. This phenomenon is called stereo matching ambiguity.

[0068] The embodiment of the present application does not limit the order in which stereo matching of different image groups in the L image groups is performed. For example, stereo matching of different image groups can be performed sequentially in sequence, or can be performed simultaneously and in parallel.

[0069] Taking the example of L image groups including a first image group and a second image group, the aircraft can first perform stereo matching on the first image group, and after obtaining the stereo matching results for the first image group, perform stereo matching on the second image group. Alternatively, the aircraft can first perform stereo matching on the second image group, and after obtaining the stereo matching results for the second image group, perform stereo matching on the first image group. Alternatively, the aircraft can perform stereo matching on the first image group and the second image group in parallel.

[0070] In an embodiment of the present application, the L image groups include at least a first image group, the L pairs of binocular camera devices include at least a first binocular camera device, and the first image group is captured by the first binocular camera device. In other words, one image in the first image group is captured by one camera in the first binocular camera device, and another image in the first image group is captured by another camera in the first binocular camera device.

[0071] The image region of the target object in the first image group has a repetitive texture in a first direction, and the angle between the first direction and the baseline of the first binocular camera device is greater than a first angle. The first angle can be 0 degrees or a larger angle, such as 10 degrees, 15 degrees, etc.

[0072] The target object's image area refers to the area enclosed by the target object's outline or edge within each image in the first image group. For example, if the target object is a linear obstacle, the target object's image area refers to the area comprised of the two-dimensional pixels representing the linear obstacle in the image after the three-dimensional linear obstacle is captured.

[0073] In the embodiments of the present application, the linear obstacle may be parallel to the baseline of the first binocular camera in actual three-dimensional space, or may be non-parallel but coplanar. However, from the perspective of the first binocular camera, the linear obstacle appears parallel to the baseline of the first binocular camera in two-dimensional space. In other words, after the linear obstacle is captured, the area composed of two-dimensional pixels representing the linear obstacle in the image is also parallel to the baseline of the first binocular camera.

[0074] By implementing the embodiments of the present application, the angle between the first direction and the baseline of the first binocular camera device can be made larger. That is, the first direction can be prevented from being parallel or approximately parallel to the baseline of the first binocular camera device. Furthermore, the first direction can be prevented from coinciding or approximately coinciding with the epipolar lines of the first image group. In this way, even if there is repeated texture in the image area of ​​the target object in the first direction, during the stereo matching process of the first image group, the angle between the direction searched along the epipolar lines and the first direction is large. Therefore, stereo matching ambiguity can be avoided during the stereo matching process of the first image group, thereby facilitating improved accuracy of depth information of the target object.

[0075] Taking the target object as a linear obstacle as an example, the first direction is the extension direction of the linear obstacle. By implementing the embodiments of the present application, at least two pairs of binocular camera devices of the aircraft can be used to capture the linear obstacle, and stereo matching is performed based on the captured images. Among them, the aircraft uses at least a pair of binocular camera devices (i.e., the first binocular camera device) with a larger angle between the baseline and the linear obstacle in the image. This is conducive to avoiding stereo matching ambiguity during the stereo matching process, thereby improving the measurement accuracy of the depth information of the linear obstacle.

[0076] In one possible implementation, the L pairs of binocular cameras meet a first condition, which includes: the baselines of the L pairs of binocular cameras are not parallel, and / or the baselines of the L pairs of binocular cameras do not belong to the same plane. In this way, at least one pair of the L pairs of binocular cameras can have a baseline that is not parallel to the first direction. In this way, even if at least one pair of the L pairs of binocular cameras has a baseline that is parallel to the first direction, causing stereo matching ambiguity, the stereo matching ambiguity can be eliminated by using a binocular camera whose baseline is not parallel to the first direction, thereby improving the accuracy of depth information of the target object.

[0077] For example, M=3, the M shooting devices include Figure 2 Taking the camera a, camera b and camera e as an example, M cameras can form three pairs of binocular camera devices, namely, binocular camera device 1 composed of camera a and camera e, binocular camera device 2 composed of camera a and camera b, and binocular camera device 3 composed of camera e and camera b. Figure 2 It can be seen that the baselines of these three pairs of binocular camera devices are not parallel.

[0078] For example, if M=4, the M shooting devices include Figure 2 The photographing device a, photographing device b, photographing device e and Figure 1 Taking the camera c shown in the figure as an example, M cameras can form 6 pairs of binocular camera devices, namely binocular camera device 1 composed of camera a and camera e, binocular camera device 2 composed of camera a and camera b, binocular camera device 3 composed of camera e and camera b, binocular camera device 4 composed of camera a and camera c, binocular camera device 5 composed of camera b and camera c, and binocular camera device 6 composed of camera e and camera c. Figure 1 、 Figure 2 It can be seen that the baselines of the three pairs of binocular camera devices can form four planes, that is, the baselines of the six pairs of binocular camera devices do not belong to the same plane.

[0079] like Figure 2As shown, the target object is a horizontal wire. If stereo matching is performed based on images captured by binocular camera 1, which consists of camera a and camera e, stereo matching ambiguity will occur because the baseline of binocular camera 1 is parallel to the horizontal wire. In this case, stereo matching is performed using images captured by binocular camera 2, which consists of camera e and camera a. Since the baseline of binocular camera 2 is not parallel to the horizontal wire, the stereo matching ambiguity can be eliminated, and the depth information of the wire can be accurately determined. Similarly, stereo matching is performed using images captured by binocular camera 3, binocular camera 4, binocular camera 5, or binocular camera 6. Since the baselines of binocular camera 3, binocular camera 4, binocular camera 5, and binocular camera 6 are all not parallel to the horizontal wire, the stereo matching ambiguity can also be eliminated.

[0080] In one possible implementation, the viewing angles of the M cameras match the aircraft's flight direction at a first moment. The correspondence between each flight direction of the aircraft and the camera can be stored in the aircraft. The viewing angle of a camera corresponding to a particular flight direction can better observe obstacles in that flight direction. By selecting M cameras that match the aircraft's flight direction at a first moment and using images captured by these M cameras (i.e., M images) to determine the depth information of the target object, the M images can more clearly record the image information of the target object, thereby improving the accuracy of the target object's depth information.

[0081] For example, Figure 1-Figure 3 Taking the camera in the figure as an example, the correspondence between each flight direction of the aircraft and the camera can be shown in Table 1. Taking the forward direction in Table 1 as an example, when the aircraft is flying in the forward direction, the camera angles of camera a, camera b, and camera e (or camera a, camera b, camera e, and camera c) can better observe obstacles in the forward direction.

[0082] Table 1

[0083]

[0084]

[0085] In one possible implementation, the correspondence between each flight direction of the aircraft and the camera device can be set by the aircraft by default or by the user, or by the user operating the aircraft controller to set it for the aircraft. The embodiment of the present application does not limit the setting method of this correspondence. The controller has a communication connection with the aircraft, and the controller can control the flight of the aircraft through the communication connection. The form of the controller may include but is not limited to: a remote control, a mobile phone, a tablet computer (pad), a desktop computer, a laptop computer, an all-in-one computer, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a handheld device, and a wearable device.

[0086] In one possible implementation, the L image groups may include not only the first image group but also a second image group. If a stereo matching ambiguity is detected during stereo matching of the second image group, the depth information of the target object may be determined by the matching result obtained by stereo matching of the first image group.

[0087] Optionally, during the process of stereo matching of L image groups, the aircraft can first perform stereo matching on one image group (such as the second image group). After detecting stereo matching ambiguity, the aircraft then performs stereo matching with another image group (such as the first image group). No stereo matching ambiguity is detected during the stereo matching of the first image group. The depth information of the target object is determined based on the matching result obtained by stereo matching of the first image group, which is conducive to improving the accuracy of the depth information.

[0088] It can be understood that if no stereo matching ambiguity is detected during the stereo matching of the second image group, the depth information of the target object can be determined by the matching result obtained by stereo matching of the second image group, or the depth information of the target object can be determined by the matching result obtained by stereo matching of the first image group.

[0089] Optionally, if no stereo matching ambiguity is detected during stereo matching of the second image group, the aircraft can cancel stereo matching of the first image group. This can not only determine the depth information of the target object based on the matching results obtained by stereo matching of the second image group, but also help save power consumption of the aircraft.

[0090] In a possible implementation, the binocular camera that captures the second image group may be referred to as a second binocular camera. The two cameras included in the second binocular camera may be located on the left side and the right side of the nose of the aircraft, respectively. Figure 2 The second binocular camera device can be called a front binocular camera device.

[0091] Optionally, when the flight direction of the aircraft at the first moment is the forward direction, stereo matching can be performed on the second image group captured by the front binocular camera devices located on the left and right sides of the aircraft's nose respectively. This is beneficial for first performing stereo matching through the image group under the optimal viewing angle, which is beneficial for more timely detection of obstacles in the forward direction and further determination of enhanced depth information of obstacles (such as target objects).

[0092] In one possible implementation, the angle between the projections of the two optical axes of the two cameras of the second binocular camera device on the first plane is a second angle, wherein the first plane is a plane formed by the roll axis and pitch axis of the aircraft, and the second angle is less than 180 degrees and greater than or equal to 90 degrees.

[0093] For information about the aircraft's roll, pitch, and yaw axes, see Figure 1 The two cameras of the second binocular camera are Figure 2 Taking the photographing device a and the photographing device e in FIG as an example, the schematic diagram of the projection of the optical axes of the photographing device a and the photographing device e on the first plane can be as follows: Figure 5 As shown, Figure 5 This is a bird's-eye view of the aircraft. In this way, the second binocular camera can cover the aircraft's front field of view as much as possible, which is conducive to more timely detection of obstacles in the forward direction.

[0094] A camera has at least one optical axis, which is the central axis of symmetry of the lens optical system. The optical axis is perpendicular to the center of the imaging sensor and passes through the optical centers of all lenses. If a camera has multiple optical axes, the optical axis in the embodiments of this application may refer to the principal optical axis.

[0095] Optional, Figure 2 The angle between the projections of the optical axes of the camera d and the camera f on the first plane may also be a second angle, such as Figure 5 Optional, Figure 2 The angle between the projections of the optical axes of the camera a and the camera d on the first plane may also be a second angle, such as Figure 5 Optional, Figure 2 The angle between the projections of the optical axes of the photographing device e and the photographing device f on the first plane may also be a second angle, such as Figure 5 In this way, the aircraft's surrounding field of view can be covered as much as possible, which helps to detect surrounding obstacles more promptly.

[0096] In a possible implementation, the angles between the two optical axes of the second binocular camera device and the positive direction of the yaw axis of the aircraft are both a third angle, and the third angle is greater than 45 degrees and less than or equal to 90 degrees.

[0097] One of the second binocular camera devices is used as Figure 1 Taking the shooting device a in the figure as an example, the schematic diagram of the optical axis of the shooting device a can be as follows Figure 6 The setting position of the shooting device a can be understood as being tilted upward at a certain angle relative to the first plane, as shown in FIG. Figure 6 As shown, the upward tilt angle of the camera a relative to the first plane is the difference between 90 degrees and the third angle. Optionally, the third angle can be 70 degrees, that is, the upward tilt angle of the camera a relative to the first plane can be 20 degrees. Figure 2 The setting method of the shooting device e) in the figure can refer to the setting method of the shooting device a, and will not be repeated here. In this way, the second binocular shooting device can cover the front field of view and the upper field of view of the aircraft, which is conducive to more timely detection of obstacles in the forward direction. In addition, in the case where the shooting device of the aircraft is a fisheye camera, since the actual effective field of view of the fisheye camera may not reach the theoretical field of view, this will result in the upper field of view covered by the second binocular shooting device (i.e., the front binocular shooting device) being limited. By tilting the second binocular shooting device upward, it is helpful to ensure the integrity of the multi-eye coverage above the aircraft. Among them, the actual effective field of view of the fisheye camera may not reach the theoretical field of view due to reasons such as large edge distortion of the fisheye field of view angle and assembly errors.

[0098] In one possible implementation, a Time of Flight (ToF) sensor could be located on the bottom of the aircraft. This sensor can detect target objects within a certain distance (such as linear obstacles at any angle). This can solve the following problem: When the second binocular camera is tilted upward, there is an area below the aircraft that is not covered by the three cameras. The depth information of targets appearing in this area may not be accurately measured by the second binocular camera, but the depth information of the target objects can be accurately measured by the ToF sensor. A ToF sensor is a depth perception technology that calculates distance by measuring the round-trip time of light pulses.

[0099] Optional, Figure 2 The angle between the optical axis of the camera d and the positive direction of the yaw axis of the aircraft is also the third angle, such as Figure 6 Optional, Figure 2 The angle between the optical axis of the camera f and the positive direction of the yaw axis of the aircraft is also the third angle. In this way, the field of view around and above the aircraft can be covered as much as possible, which is conducive to more timely detection of obstacles.

[0100] In one possible implementation, one of the first binocular camera devices can be located above the nose, below the nose, or below the fuselage of the aircraft. This approach prevents the baseline of the first binocular camera device from being parallel to linear obstacles, thereby helping to avoid stereo matching ambiguity during stereo matching of the first image group captured by the first binocular camera device.

[0101] For example, one of the first binocular camera devices may be Figure 1 The shooting device b and shooting device c shown in FIG. It should be noted that, Figure 1 The position of the camera c shown on the lower side of the aircraft fuselage is only a schematic position and does not constitute a limitation to the embodiments of the present application. In other possible implementations, the camera c can also be located on the lower side of the aircraft nose.

[0102] In a possible implementation, an angle between an optical axis of one of the first binocular camera devices and the yaw axis of the aircraft is a fourth angle, and the fourth angle is greater than 0 degrees and less than or equal to 45 degrees.

[0103] One of the first binocular camera devices is used as Figure 1 Taking the shooting device b in the example, the schematic diagram of the optical axis of the shooting device b can be as follows Figure 7 The setting position of the camera b can be understood as being tilted forward at a certain angle relative to the second plane, which is a plane formed by the yaw axis and pitch axis of the aircraft.

[0104] like Figure 7 As shown, the tilt angle of the camera b relative to the second plane is a fourth angle. Optionally, the fourth angle can be 10 degrees. By tilting the camera b forward and tilting the front binocular camera (i.e., the camera a and the camera e) upward, it can be ensured that the front field of view and the upper field of view of the aircraft are covered by at least three cameras. In this way, even if a target object (such as a linear obstacle) appears in the front field of view and the upper field of view, the stereo matching ambiguity can be resolved, which is conducive to accurately calculating the depth information of the target object. In addition, in the case where the camera of the aircraft is a fisheye camera, since the actual effective field of view angle of the fisheye camera may not reach the theoretical field of view angle, this will result in the front field of view covered by the camera above and below the aircraft (such as the camera b, the camera c) is limited. By tilting the upper camera and / or the lower camera forward, it is beneficial to ensure the integrity of the multi-eye coverage in front of the aircraft.

[0105] Optional, Figure 1 The angle between the optical axis of the camera c and the yaw axis of the aircraft is also the fourth angle, such as Figure 7As shown, the position of camera c can be understood as being tilted forward at a certain angle relative to the second plane. The tilt angle of camera c relative to the second plane is a fourth angle. By using the front binocular cameras (i.e., camera a and camera e) and tilting camera c forward, the forward field of view of the aircraft can be ensured to be covered by at least three cameras. This allows stereo matching to resolve ambiguities even if a target object (such as a linear obstacle) appears in the forward field of view, thereby facilitating accurate calculation of the target object's depth information.

[0106] In a possible implementation, a shooting device (such as Figure 1 The camera b) and / or the lower camera (such as Figure 1 In the case where the camera device c) is tilted forward at a fourth angle, the first instruction may be replaced by a second instruction, wherein the first instruction is used to instruct the aircraft to fly backward, and the second instruction is used to instruct the aircraft to turn around and fly forward.

[0107] The forward tilt of the upper and / or lower cameras creates an area behind the aircraft that is not covered by at least three cameras. The aircraft may be unable to accurately measure the depth of objects (such as linear obstacles) in this area, and if it were to fly directly forward, it might not be able to circumvent the object. By replacing the backward flight instruction with a turnaround and then forward flight, the aircraft's forward field of view is covered by at least three cameras, allowing it to accurately obtain depth information for objects in this area, thereby improving flight safety.

[0108] Optionally, the first instruction may be replaced with the second instruction when the aircraft is in automatic flight mode. In the case where the aircraft is in remote control flight mode, instruction replacement is not necessary.

[0109] Optionally, the embodiments of the present application do not limit the values ​​and settings of the angle values ​​involved (such as the first angle, the second angle, the third angle, and the fourth angle). For example, the angle values ​​involved in the embodiments of the present application can be set by default by the aircraft, or can be set by the user operating the aircraft, or can be set by the aircraft controller. The angle values ​​involved in the embodiments of the present application can be modified by the aircraft or the controller.

[0110] In one possible implementation, S402 may be executed if the aircraft meets the second condition. The second condition may include, but is not limited to, one or more of the following: detecting that the target scene includes a target object, and that repeated textures exist on the surface of the target object; the aircraft entering a matching ambiguity prevention mode; the aircraft is executing a first mission, and the scene in which the first mission is executed includes the target object, and that repeated textures exist on the surface of the target object.

[0111] Optionally, the aircraft can use a ToF sensor or other sensor to detect whether the current scene includes the target object. If the current scene is detected to include the target object, the current scene is used as the target scene. In this case, the depth information of the target object is determined based on the stereo matching results of the L image groups. This helps reduce the probability of detecting stereo matching ambiguity during stereo matching of images that include the target object, thereby helping to improve the accuracy of the depth information.

[0112] Optionally, the aircraft can enter a match ambiguity prevention mode automatically or based on user instructions. Based on the flight mission the aircraft is to perform, the user can determine that the scene the aircraft is facing may include a target object. In this case, controlling the aircraft to enter match ambiguity prevention mode allows the aircraft to determine the depth information of the target object based on the stereo matching results of the L image groups. This helps reduce the probability of detecting stereo match ambiguity during stereo matching of images that include the target object, thereby improving the accuracy of the depth information.

[0113] Optionally, when the first mission being performed by the aircraft is a mission scenario that includes a target object, the depth information of the target object is determined based on the stereo matching results of the L image groups. This helps reduce the probability of detecting stereo matching ambiguities during stereo matching of images that include the target object, thereby improving the accuracy of the depth information. Exemplary first missions may include, but are not limited to, power line inspection missions and drone delivery missions.

[0114] Optionally, if the aircraft does not meet any of the second conditions, the aircraft can determine the depth information of the target object based on the stereo matching results of a single image group. For example, when the aircraft is flying forward and none of the second conditions are met, the aircraft can accurately calculate the depth information of obstacles in the forward field of view by performing stereo matching on a single image group captured by the front binocular camera. In this case, the obstacle in the forward field of view is not considered a target object. This method helps save power.

[0115] In a possible implementation, the M shooting devices may include a first shooting device and a second shooting device, the number of the first shooting devices may be one or more, and the second shooting device includes other shooting devices among the M shooting devices except the first shooting device.

[0116] In one possible implementation, the first angle is smaller than the second angle. The first angle is the angle between a first incident ray of light from a first camera and the optical axis of the first camera, where the first incident ray includes the line between the target object and the first camera; the second angle is the angle between a second incident ray of light from a second camera and the optical axis of the second camera, where the second incident ray includes the line between the target object and the second camera. In other words, among the M cameras, the angle between the first incident ray of light from the first camera and the optical axis of the first camera is the smallest, and the first incident ray is closest to the center of the field of view. This indicates that the camera with the best viewing angle for observing the target object in the target scene is the first camera. It is understood that there may be multiple angles with the same angle, all of which are minimum angles. In this case, there may be more than one first camera.

[0117] In one possible implementation, at least one image in the second image group is captured by the first camera. In other words, at least one camera in the second binocular camera is the first camera. Alternatively, at least one image in the first image group is captured by the first camera. In other words, at least one camera in the first binocular camera is the first camera. In this manner, the aircraft can first select the image captured by the first camera with the best observation for stereo matching. Alternatively, the aircraft can select the image captured by the first camera with the best observation for stereo matching after detecting that stereo matching has failed.

[0118] See also Figure 8 , Figure 8 This is a schematic diagram of the structure of a depth information determination device provided by an embodiment of the present application. Figure 8 As shown, the depth information determining apparatus 80 includes an acquiring unit 801 and a matching unit 802. Optionally, the depth information determining apparatus 80 may further include a replacing unit 803. Figure 8 The units shown by the dotted lines are optional units, that is, the depth information determination device 80 may not include Figure 8 The depth information determining device 80 can execute the steps related to the aircraft in the above method embodiment.

[0119] An acquisition unit 801 is configured to acquire N images of a target scene captured by N camera devices of an aircraft at a first moment, wherein one image is captured by one camera device, the target scene includes a target object, and N is an integer greater than or equal to 3;

[0120] Matching unit 802 is configured to perform stereo matching on at least two image groups to obtain depth information of a target object. One image group includes two images from M images, where the M images are a subset of N images, and the M images are captured by M cameras from N cameras. The at least two image groups include at least a first image group, wherein an image region of the target object in the first image group has a repetitive texture in a first direction, an angle between the first direction and a baseline of a first binocular camera is greater than a first angle, and the first binocular camera includes two cameras from the M cameras that captured the first image group.

[0121] In one implementation, L pairs of binocular camera devices consisting of M camera devices meet a first condition, and a pair of binocular camera devices consists of two camera devices; wherein the first condition includes: the baselines of the L pairs of binocular camera devices are not parallel, and / or the baselines of the L pairs of binocular camera devices do not belong to the same plane.

[0122] In one implementation, the shooting angles of the M shooting devices match the flight direction of the aircraft at the first moment.

[0123] In one implementation, the at least two image groups further include a second image group; if stereo matching ambiguity is detected during stereo matching of the second image group, the depth information of the target object is determined based on the matching result obtained by stereo matching of the first image group.

[0124] In one implementation, the two photographing devices that photograph the second image group are located on the left side and the right side of the nose of the aircraft, respectively.

[0125] In one implementation, the angle between the projections of the two optical axes of the second binocular camera device on the first plane is a second angle, where the first plane is a plane formed by the roll axis and pitch axis of the aircraft, and the second angle is less than 180 degrees and greater than or equal to 90 degrees; the angle between the two optical axes of the second binocular camera device and the positive direction of the yaw axis of the aircraft is a third angle, and the third angle is greater than 45 degrees and less than or equal to 90 degrees.

[0126] In one implementation, one of the first binocular photographing devices is located on the upper side of the nose, the lower side of the nose, or the lower side of the fuselage of the aircraft.

[0127] In one implementation, an angle between an optical axis of one of the first binocular camera devices and the yaw axis of the aircraft is a fourth angle, and the fourth angle is greater than 0 degrees and less than or equal to 45 degrees.

[0128] In one implementation, the replacement unit 803 is configured to replace the first instruction with a second instruction, wherein the first instruction is used to instruct the aircraft to fly backward, and the second instruction is used to instruct the aircraft to turn around and fly forward.

[0129] In one implementation, the matching unit 802 is used to perform stereo matching on at least two image groups to obtain depth information of the target object, and can be used to: when the aircraft meets the second condition, perform stereo matching on at least two image groups to obtain depth information of the target object; wherein the second condition includes one or more of the following: it is detected that the target scene includes the target object, and there are repeated textures on the surface of the target object; the aircraft enters a matching ambiguity prevention mode; the aircraft is performing a first task, the scene for performing the first task includes the target object, and there are repeated textures on the surface of the target object.

[0130] Specifically, in this case, the operations performed by the acquisition unit 801, the matching unit 802, and the replacement unit 803 can refer to Figure 1-Figure 7 The introduction of the aircraft in the corresponding embodiment.

[0131] See also Figure 9 , Figure 9 This is a schematic diagram of the structure of an aircraft 90 provided in an embodiment of the present application. It can be used to implement the functions of the aircraft in the above method embodiment. The aircraft 90 may include N shooting devices 901 and a processor 902. Optionally, the aircraft 90 may also include a memory 903. The N shooting devices 901, the processor 902, and the memory 903 may be connected via a bus 904 or other means. The bus is Figure 9 The connections between the other components are shown in bold lines, which are only for illustration and not intended to be limiting. The bus can be divided into address bus, data bus, control bus, etc. Figure 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0132] The coupling in the embodiments of the present application is an indirect coupling or communication connection between devices, units, or modules, which can be electrical, mechanical, or other forms, and is used for information exchange between devices, units, or modules. The embodiments of the present application do not limit the specific connection medium between the N cameras 901, processor 902, and memory 903.

[0133] The memory 903 may include a read-only memory and a random access memory, and provides instructions and data to the processor 902. A portion of the memory 903 may also include a nonvolatile random access memory.

[0134] The processor 902 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, or alternatively, the processor 902 may be any conventional processor.

[0135] In one example, when the aircraft uses Figure 9 When the form shown is Figure 9 The processor in can execute the method executed by the aircraft in any of the above method embodiments.

[0136] In an optional embodiment, the memory 903 is used to store computer programs or instructions; the processor 902 is used to call the computer programs or instructions stored in the memory 903 for execution Figure 1-Figure 7 The steps performed by the aircraft in the corresponding embodiment.

[0137] Specifically, Figure 8 The functions / implementation processes of the acquisition unit 801, the matching unit 802, and the replacement unit 803 can be realized by Figure 9 The processor 902 in the memory 903 calls the computer program or instruction stored in the memory 903 to implement it.

[0138] In the embodiments of the present application, a computer program (including program code) capable of executing the steps involved in the above method can be run on a general-purpose computing device, such as a computer, including processing elements and storage elements such as a CPU, random access memory (RAM), and read-only memory (ROM), and the methods provided in the embodiments of the present application can be implemented. The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into an aircraft via the computer-readable recording medium and executed therein.

[0139] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the aircraft 90 in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems provided by the aircraft in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0140] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program or computer instructions are stored. The computer program or computer instructions are suitable for being loaded by an aircraft and executing the method provided by the above method embodiment.

[0141] An embodiment of the present application also provides a computer program product comprising a computer program or instructions. When the computer program or instructions are executed on an aircraft, the aircraft is caused to execute the method provided in the above method embodiment.

[0142] Regarding the various modules / units contained in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or partly software modules / units and partly hardware modules / units. For example, for various devices and products applied to or integrated into a chip, the various modules / units contained therein can all be implemented in the form of hardware such as circuits, or at least part of the modules / units can be implemented in the form of software programs, which run on the processor integrated inside the chip, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated into a chip module, the various modules / units contained therein can all be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component of the chip module (such as a chip, circuit module, etc.) or in different components, or at least part of the modules / units can be implemented in the form of software programs. It is implemented in the form of a software program that runs on a processor integrated inside the chip module, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits; for various devices and products applied to or integrated into an aircraft, the various modules / units contained therein can be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (for example, a chip, circuit module, etc.) or different components in the aircraft, or at least some modules / units can be implemented in the form of a software program that runs on a processor integrated inside the aircraft, and the remaining (if any) modules / units can be implemented in the form of hardware such as circuits.

[0143] It should be noted that for the aforementioned various method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0144] In the above embodiments, the description of each embodiment has its own emphasis. Any multiple embodiments can be used in combination. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0145] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0146] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0147] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by program instructions and related hardware. The program instructions can be stored in a computer-readable storage medium, which may include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0148] The above disclosure is only one embodiment of the present application, which is only a part of the embodiments of the present application and cannot be used to limit the scope of rights of the present application.

Claims

1. A method for determining depth information, characterized in that: The method comprises: Obtaining N images of a target scene captured by N camera devices of the aircraft at a first moment, wherein one image is captured by one camera device, the target scene includes a target object, and N is an integer greater than or equal to 3; Stereo matching is performed on at least two image groups to obtain depth information of the target object; wherein one image group includes two images from M images, the M images are a subset of the N images, and the M images are captured by M of the N capturing devices; the at least two image groups include at least a first image group, an image region of the target object in the first image group has a repetitive texture in a first direction, an angle between the first direction and a baseline of a first binocular capturing device is greater than a first angle, and the first binocular capturing device includes two of the M capturing devices that captured the first image group.

2. The method according to claim 1, characterized in that The L pairs of binocular camera devices composed of the M camera devices meet the first condition that a pair of binocular camera devices consists of two camera devices; The first condition includes: the baselines of the L pairs of binocular camera devices are not parallel, and / or the baselines of the L pairs of binocular camera devices do not belong to the same plane.

3. The method according to claim 1, characterized in that The shooting angles of the M shooting devices match the flight direction of the aircraft at the first moment.

4. The method according to claim 1, wherein The at least two image groups further include a second image group; If stereo matching ambiguity is detected during stereo matching of the second image group, the depth information of the target object is determined based on a matching result obtained by stereo matching of the first image group.

5. The method according to claim 4, characterized in that The two shooting devices for shooting the second image group are respectively located on the left side and the right side of the nose of the aircraft.

6. The method according to claim 5, characterized in that The included angle between the projections of the two optical axes of the second binocular camera device on the first plane is a second angle, wherein the first plane is a plane formed by the roll axis and the pitch axis of the aircraft, and the second angle is less than 180 degrees and greater than or equal to 90 degrees; The angles between the two optical axes of the second binocular camera device and the positive direction of the yaw axis of the aircraft are both a third angle, and the third angle is greater than 45 degrees and less than or equal to 90 degrees.

7. The method according to claim 4, characterized in that One of the first binocular shooting devices is located on the upper side of the nose, the lower side of the nose, or the lower side of the fuselage of the aircraft.

8. The method according to claim 7, characterized in that An angle between the optical axis of one of the first binocular shooting devices and the yaw axis of the aircraft is a fourth angle, and the fourth angle is greater than 0 degrees and less than or equal to 45 degrees.

9. The method according to claim 8, characterized in that The method further comprises: The first instruction is replaced by a second instruction, wherein the first instruction is used to instruct the aircraft to fly backward, and the second instruction is used to instruct the aircraft to turn around and fly forward.

10. The method according to any one of claims 1 to 9, characterized in that The performing stereo matching on the at least two image groups to obtain depth information of the target object includes: When the aircraft satisfies a second condition, stereo matching is performed on at least two image groups to obtain depth information of the target object; wherein the second condition includes one or more of the following: detecting that the target scene includes the target object, and that a repeated texture exists on the surface of the target object; The aircraft enters a matching ambiguity prevention mode; The aircraft is executing a first mission, a scene in which the first mission is executed includes the target object, and a surface of the target object has repeated textures.

11. A depth information determination device, characterized in that: The method comprises means for performing the method according to any one of claims 1 to 10.

12. An aircraft, characterized in that: The invention comprises a memory, a processor and N shooting devices, wherein the memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions in the memory, and when the computer program or instructions are executed by the processor, the aircraft performs the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program or computer instructions. When the computer program or computer instructions are executed by an aircraft, the aircraft is enabled to perform the method according to any one of claims 1 to 10.

Images