Camera layout method of unmanned aerial vehicle, unmanned aerial vehicle, electronic equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of unmanned aerial vehicles, in particular to a camera layout method of an unmanned aerial vehicle, the unmanned aerial vehicle, electronic equipment and a storage medium, and the method comprises the steps: obtaining fuselage data of the unmanned aerial vehicle; according to the fuselage data, a camera position model is determined, the camera position model comprises position information of a plurality of cameras on the unmanned aerial vehicle, and the plurality of cameras are distributed on the fuselage of the unmanned aerial vehicle in pairs to form a plurality of camera groups; and according to a preset camera orientation rule, orientation information of each camera in the camera position model is determined, so that a plurality of mutually vertical base lines are formed among the cameras in each camera group. Based on the method, the base lines of the plurality of camera groups are perpendicular to each other, so that when depth measurement fails due to ambiguity caused by binocular cameras formed by some camera groups, the ambiguity can be eliminated by means of another pair of units perpendicular to the base line of the unit, the situation that stereo matching is invalid is effectively avoided, and the applicability of the unmanned aerial vehicle in various scenes is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of drones, and in particular to a camera layout method for a drone, a drone, an electronic device, and a storage medium. Background Art

[0002] A drone is an unmanned aircraft with a mission payload that is controlled by a remote control device or a self-contained program control device. If a drone collides with an obstacle while performing a predetermined mission, it may cause the aircraft to crash and cause losses. Therefore, low-altitude drones currently generally integrate obstacle avoidance and detour functions to reduce the occurrence of collision accidents. In related technologies, drones are usually equipped with image sensors such as cameras to enable them to have a certain three-dimensional spatial environment perception capability, thereby achieving the effect of all-round obstacle avoidance and detour of the drone.

[0003] Currently, most drones use a binocular stereo matching perception solution, which infers the depth of objects in the scene by analyzing the parallax information of the images captured by two cameras, and then uses a stereo matching algorithm to reconstruct the three-dimensional structure of the scene. However, this solution does not work well in certain specific scenarios. Any point in the environment can be observed by at most two cameras at the same time, so it may cause ambiguity when performing stereo matching, resulting in greater errors or even failure of depth measurement, which in turn affects the obstacle avoidance and circumvention effects of the drone. For example, assume that there is a linear obstacle in the environment, which is parallel to the binocular baselines of the two cameras. At this time, because the linear obstacle coincides with the camera binocular baseline and the linear obstacle usually lacks texture, the stereo matching algorithm will find many ambiguous matching points when searching along the baseline, and cannot obtain accurate results, which affects the obstacle avoidance and circumvention effects of the drone. Summary of the invention

[0004] One purpose of the embodiments of the present application is to provide a camera layout method for a drone, a drone, an electronic device and a storage medium, so as to solve the technical problem that related technologies are prone to ambiguity when performing stereo matching, resulting in larger errors or even failure of depth measurement, affecting the obstacle avoidance and circumvention effects of the drone.

[0005] In a first aspect, an embodiment of the present application provides a camera layout method for an unmanned aerial vehicle, comprising: obtaining body data of the unmanned aerial vehicle; determining a camera position model based on the body data, wherein the camera position model includes position information of multiple cameras on the unmanned aerial vehicle, and the multiple cameras are distributed in pairs on the body of the unmanned aerial vehicle to form multiple camera groups; determining the orientation information of each camera in the camera position model based on a preset camera orientation rule, so that multiple mutually perpendicular baselines are formed between each camera in each of the camera groups.

[0006] In combination with the first aspect, in a possible implementation method, determining the camera position model based on the body data of the drone includes: determining the body parameters of the drone based on the body data of the drone; determining the number of cameras based on the body parameters; and determining the position information of each camera on the body of the drone in sequence based on the number of cameras and preset camera distribution rules to obtain the camera position model.

[0007] In combination with the first aspect, in a possible implementation method, determining the number of the multiple cameras according to the body parameters of the drone includes: determining the body model of the drone according to the body parameters of the drone; determining a geometric model that matches the body model of the drone in a preset database according to the body model of the drone, the geometric model including multiple vertices; and determining the number of cameras according to the number of each vertices in the geometric model.

[0008] In combination with the first aspect, in a possible implementation method, determining the orientation information of each camera in the camera position model according to a preset camera orientation rule includes: determining a target camera, wherein the target camera is distributed on the fuselage of the drone; determining the position information of the target camera according to the camera position model; and determining the orientation information of the target camera according to the position information of the target camera.

[0009] In combination with the first aspect, in a possible implementation method, the camera position model includes a geometric model corresponding to the fuselage of the drone, the target camera is at a vertex of the geometric model, and determining the orientation information of the target camera based on the position information of the target camera includes: determining the vertex where the target camera is located in the geometric model as the target vertex; based on the target vertex, determining a target plane in the geometric model, the target vertex is outside the target plane; and determining the orientation information of the target camera based on the target vertex and the target plane.

[0010] In combination with the first aspect, in a possible implementation, determining a target plane that meets preset conditions in the geometric model based on the target vertex includes: determining at least three reference vertices adjacent to the target point in the geometric model based on the target vertex, each of the reference vertices being on a vertex of the geometric model and sharing an edge with the target vertex; determining a plurality of target straight lines based on the reference vertices, each of the target straight lines being connected between any two reference vertices; and determining the target plane based on the target straight lines, each of the target straight lines intersecting within the target plane.

[0011] In combination with the first aspect, in a possible implementation method, determining the orientation information of the target camera based on the target vertex and the target plane includes: determining a target normal based on the target vertex and the target plane, wherein the target normal passes through the target vertex and is perpendicular to the target plane; determining that the direction in which the target camera points to the outside of the geometric model along the target normal is the target direction; and generating the orientation information based on the target direction.

[0012] In a second aspect, an embodiment of the present application further proposes a drone, wherein a plurality of cameras are provided on the fuselage of the drone, and the plurality of cameras are distributed on the fuselage of the drone by the camera layout method as described in the first aspect.

[0013] In a third aspect, an embodiment of the present application further proposes an electronic device, comprising a memory and a processor, wherein the memory is connected to the processor, and the processor is used to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the electronic device implements the method described in the first aspect.

[0014] In a fourth aspect, an embodiment of the present application further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method described in the first aspect.

[0015] The embodiments of the present application can achieve the following technical effects:

[0016] Based on the method proposed in the embodiment of the present application, when used to layout the cameras on the fuselage of the drone, the fuselage data of the drone is first obtained, and then the camera position model is determined based on the fuselage data, the camera position model includes the position information of multiple cameras on the drone, and the multiple cameras are distributed in pairs on the fuselage of the drone to form multiple camera groups, and finally the orientation information of each camera in the camera position model is determined according to the preset camera orientation rules, so that multiple mutually perpendicular baselines are formed between the cameras in each camera group. Based on this method, multiple cameras are distributed on the fuselage of the drone to form multiple camera groups, and all-round multi-eye perception coverage is achieved through the combination of multiple camera groups, and the baselines between each camera group are perpendicular. When the binocular cameras formed by some camera groups cause ambiguity and cause the depth measurement to fail, the ambiguity can be eliminated by another pair of binocular groups perpendicular to the baseline of the binocular group, effectively avoiding the invalidity of stereo matching, thereby improving the obstacle avoidance and obstacle bypassing technology effects of the drone, and further improving the applicability of the drone in various specific scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 A schematic diagram of a drone obstacle avoidance scenario provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of a flow chart of a camera layout method for a drone provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a geometric model of a drone provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of a target plane in a geometric model of a drone provided in an embodiment of the present application;

[0022] Figure 5 A schematic diagram of the structure of a camera layout device for a drone provided in an embodiment of the present application;

[0023] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0025] It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional module division is performed in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in this application do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0026] To better explain this application, please first refer to Figure 1 , Figure 1The figure shows a schematic diagram of a drone obstacle avoidance scenario. Figure 1 As shown, the scene includes a drone and an obstacle P located in front of the drone, wherein the drone is provided with four cameras C1, C2, C3 and C4, and any two cameras can form a pair of binocular units, and cameras C1, C2, C3, and C4 are respectively used to collect images of the drone's surrounding environment. By comparing the disparity information of the images captured by each camera in the binocular unit ("disparity" is the pixel coordinate offset of the same object in two images), a stereo matching algorithm is further used to find the corresponding points in the two images to calculate the disparity, so as to calculate the depth map according to the internal and external parameters of the camera ("depth map" is an image in which the gray value or color of each pixel represents the distance of the corresponding point in the scene, and is usually used to represent the spatial structure of a three-dimensional scene). Finally, the depth of the object in the scene is inferred according to the depth map, and the three-dimensional structure of the scene can be reconstructed.

[0027] However, if Figure 1 As shown, if the obstacle P in front of the drone is a linear obstacle (such as a wire), and the obstacle P is parallel to the binocular baseline of the binocular unit on the drone (for example, the baseline L1 between cameras C1 and C2), since the linear obstacle coincides with the baseline and the linear obstacle usually lacks texture, the stereo matching algorithm will find many ambiguous matching points when searching along the baseline, and cannot obtain accurate results. It should be noted that in application scenarios such as drone wire inspection, it is a common situation that the linear obstacle is parallel to the binocular baseline, which greatly limits the application of drones in such scenarios.

[0028] Based on this, the embodiment of the present application proposes a camera layout method for a drone, which can enable the drone to be used in applications such as Figure 1 The obstacle avoidance scenario shown here effectively reduces the ambiguity of stereo matching.

[0029] For details, please refer to Figure 2 , Figure 2 Shown is a schematic flow diagram of the method, comprising:

[0030] Step S10, obtaining the fuselage data of the drone;

[0031] In this embodiment, the fuselage data of the drone mainly refers to data related to the three-dimensional structure and fuselage size of the drone fuselage, etc. This part of data can be obtained through a preset database, or can be measured and collected in real time through specific tools. For example, assuming that a preset database stores the fuselage data of various types of drones, when performing camera layout on the target drone (i.e., the drone of the current layout plan), the identifier on the drone fuselage, such as the SN number (Serial Number, i.e., the serial number), can be first identified, and then the fuselage data corresponding to the target drone can be extracted from the preset database through the SN number for use in subsequent steps.

[0032] In some embodiments, the camera layout method of the drone is executed by an electronic device with communication capabilities, which can be in the form of various mobile terminals, computers, tablet computers or other similar industrial equipment, etc. The electronic device can communicate with the background server and access the preset database to obtain the above-mentioned body data. For example, the fuselage or external accessories of the drone are usually provided with a unique identification code (such as a barcode, a QR code, etc.), and the electronic device is provided with a barcode scanner or other similar information collection device, which can scan the identification code, read the SN number uniquely corresponding to the drone, and transmit the SN number to the background server. The server-side database pre-stores the body data of various drones when they leave the factory, and the body data of each drone is uniquely corresponding to its SN number. Then, the background server can send the body data corresponding to the SN number to the electronic device, so that the electronic device can obtain the body data of the drone.

[0033] In other embodiments, the electronic device is provided with a device for collecting the body data of the drone, such as an image acquisition device or an infrared scanning device, which can scan the body of the drone to generate relevant data, and analyze the relevant data in combination with a corresponding computer vision algorithm to generate body data in a specific format. This embodiment does not impose too many restrictions on this.

[0034] Step S20, determining a camera position model according to the fuselage data, wherein the camera position model includes position information of a plurality of cameras on the drone, and the plurality of cameras are distributed in pairs on the fuselage of the drone to form a plurality of camera groups;

[0035] In this embodiment, the camera position model refers to a data model that can indicate the position information of multiple cameras on the drone body. Under the guidance of this model, the position of each camera on the drone body can be abstracted as a target point. The setting of this target point ignores the specific structure of the camera, and its position installed on the fuselage is regarded as a point outside the fuselage as a whole.

[0036] It can be understood by those skilled in the art that, according to the principle of multi-viewing, multiple cameras should be distributed in pairs on the fuselage of the drone, so that a pair of camera groups can be formed between every two cameras, so that the images collected by different cameras in each camera group can be used for 3D scene reconstruction. Of course, in addition to this, the number of cameras included in each camera group can also be a limited number of n (n≥2), and the n cameras in each camera group can collect different images from the environment to determine depth information, thereby achieving the effect of 3D scene reconstruction.

[0037] It should be noted that the camera position model includes at least two types of data, namely, the fuselage model used to indicate the fuselage of the drone, and the position information of each camera in the fuselage model, that is, the specific setting of the fuselage model should be able to indicate an abstract three-dimensional model. According to this understanding, the three-dimensional model can at least correspond to a specific reference system, such as the direction of Roll, Pitch and Yaw can be determined according to the fuselage of the drone, and then the coordinate axes in the three directions can be obtained, thereby obtaining the reference system of the drone fuselage. Therefore, the position information of each camera on the drone fuselage can be uniquely determined and represented by its coordinates mapped on the reference system. As a feasible implementation, the position information of each camera in the fuselage model is represented by a unique three-element array, and each element in the array is used to indicate its coordinates in the Roll, Pitch and Yaw directions respectively.

[0038] Step S30: determining the orientation information of each camera in the camera position model according to a preset camera orientation rule, so that a plurality of mutually perpendicular baselines are formed between the cameras in each of the camera groups.

[0039] In this embodiment, after determining the position of each camera, it is also necessary to determine the specific orientation of each camera so that the baselines between different camera groups and between cameras in groups are perpendicular to each other. Therefore, when an ambiguous image is captured in the environment by a certain camera group, the image can be captured by another camera group that is perpendicular to the baseline of the camera group, thereby achieving the effect of eliminating ambiguity.

[0040] Through the above method, this embodiment proposes a solution for camera layout of drones, which can effectively eliminate the ambiguity of stereo matching. Specifically, if each camera in the above camera position model is a fisheye camera, and the field of view (FoV) of each fisheye camera is 180° (it can be larger or smaller than this, but not too small, and the lower limit of FoV is to be calculated), it is easy to understand that any point around the drone can be observed by exactly four cameras, that is, all-round multi-view coverage is achieved. Figure 1Taking the application scenario shown as an example, the four cameras C1, C2, C3 and C4 can form at least two groups of binocular cameras with mutually perpendicular baselines, namely (C1, C2) and (C2-C3). In this way, when the linear obstacle P is ambiguous in one pair of binocular cameras C1-C2 (at this time, the linear obstacle P is parallel to the baseline L1 between the binocular cameras (C1, C2)), for the other pair of binocular cameras C2-C3, it is obvious that since the baseline L1 between the binocular cameras (C1, C2) is perpendicular to the baseline L2 between the binocular cameras (C2, C3), the ambiguity can be eliminated, thereby eliminating the ambiguity caused by the linear obstacle and the binocular baseline (i.e., the baseline between the binocular cameras) being horizontal or nearly horizontal, and avoiding the situation where the stereo matching is invalid.

[0041] For example, assuming that there is a horizontal wire in front of the drone, and the drone body includes but is not limited to three cameras, namely, "upper left front", "upper right front" and "upper and lower front", if the two cameras of "upper left front" and "upper right front" form a binocular, the horizontal wire and the baseline between the two cameras of "upper left front" and "upper right front" will be parallel to each other, which will cause ambiguity and lead to failure of depth estimation. However, another binocular unit can be formed by the two cameras of "upper left front" and "lower left front", and the baseline direction of the binocular unit is perpendicular to the horizontal wire, so the ambiguity can be eliminated. In addition, the camera layout in this embodiment can maximize the baseline, and at the same time evenly and reasonably distribute the FoV (Field of View, i.e., field of view or visual field) to all directions of the drone according to the baseline length, so as to maximize the use of the camera's visual coverage and the long baseline advantage to improve the ranging accuracy and achieve the best perception effect.

[0042] Furthermore, in some embodiments, the above-mentioned determining the camera position model based on the body data of the drone includes: determining the body parameters of the drone based on the body data of the drone; determining the number of cameras based on the body parameters; and determining the position information of each camera on the body of the drone in sequence based on the number of cameras and preset camera distribution rules to obtain the camera position model.

[0043] Among them, the fuselage parameters of the drone refer to the parameters related to the three-dimensional shape of the drone fuselage, such as the length, width, height of the drone fuselage, the endpoints and angles of the connection between the wing and the fuselage, etc. The fuselage parameters can be extracted from the drone fuselage data to obtain the relevant fuselage parameters. The number of cameras that need to be arranged on the drone fuselage can be determined from the relevant fuselage parameters. After determining the number of cameras, the position of each camera on the fuselage is calculated in a certain order. The position of each camera should correspond to a point outside the fuselage, and then the camera position model is obtained.

[0044] Specifically, according to the body parameters of the drone, the number of the multiple cameras is determined, including: according to the body parameters of the drone, the body model of the drone is determined; according to the body model of the drone, a geometric model matching the body model of the drone is determined in a preset database, wherein the geometric model includes multiple vertices; and according to the number of each vertex in the geometric model, the number of cameras is determined.

[0045] It is easy to understand that a unique fuselage model can be determined from the fuselage parameters of the drone. The fuselage model is an abstraction of the three-dimensional structure of the drone fuselage, including the structural features of the drone fuselage. The data stored in the preset database indicates the correspondence between multiple fuselage models and geometric models, so that the database can be traversed to determine the corresponding geometric model based on the fuselage model of the drone.

[0046] It should be noted that this geometric model ignores the structural features on the drone fuselage that are not related to the camera layout, and thus converts the complex fuselage model into a simple geometric model, which is conducive to simplifying the analysis of the entire fuselage to determine the number of cameras. For details, please refer to Figure 3 ,like Figure 3 As shown, according to the fuselage model (1) of the drone, it can be abstracted into a geometric model (2), and the geometric model (2) is a cuboid, including 8 vertices, and a camera can be set at each of the 8 vertices. Therefore, the number of vertices of the geometric model (2) is the number of determined cameras. Of course, in addition to this, according to the characteristics of the drone fuselage model, the geometric model can also be other geometric bodies that match the fuselage model, such as regular polyhedrons, spheres or ellipsoids of various shapes, etc., and those skilled in the art can set it according to the structural characteristics of the fuselage.

[0047] Furthermore, in the above embodiment, the orientation information of each camera in the camera position model is determined according to a preset camera orientation rule, including: determining a target camera, wherein the target camera is distributed on the fuselage of the drone; determining the position information of the target camera according to the camera position model; and determining the orientation information of the target camera according to the position information of the target camera.

[0048] In this embodiment, the target camera refers to any camera with a to-be-determined orientation in the camera position model after the camera position model is determined. After the camera position model is determined (i.e., the number of cameras is determined and the positions of each camera are determined), the specific orientation of each camera needs to be determined. Assuming that the camera position model determines a camera set C with a predetermined position: {c1, c2, c3...cn}, in a certain order, c1, c2, c3...cn in the camera set C are determined as target cameras in turn until the orientations of all cameras in the camera set C are determined.

[0049] It is easy to understand that in this embodiment, since each camera is distributed at the vertex of the above-mentioned geometric model (2), the angle formed by the specific orientation of each camera and any edge on which it is located should be between 0° and 90°.

[0050] Specifically, it can be seen from the above embodiments that any camera in the camera set C is located at a vertex of the geometric model, and the orientation information of the target camera is determined according to the position information of the target camera, including: determining the vertex at which the target camera is located in the geometric model as the target vertex; based on the target vertex, determining a target plane in the geometric model, and the target vertex is outside the target plane; and determining the orientation information of the target camera according to the target vertex and the target plane.

[0051] Those skilled in the art will appreciate that, in this embodiment, the orientation information of the target camera at the target vertex is determined by the target vertex and a target plane associated with the target vertex, thereby determining the orientation information of the target camera based on the geometric relationship between the target vertex and the target plane.

[0052] Wherein, based on the target vertex, determining a target plane that meets preset conditions in the geometric model includes: based on the target vertex, determining at least three reference vertices adjacent to the target vertex in the geometric model, each of the reference vertices being on a vertex of the geometric model and sharing an edge with the target vertex; based on the reference vertices, determining a plurality of target straight lines, each of the target straight lines being connected between any two reference vertices; based on the target straight lines, determining the target plane, each of the target straight lines intersecting within the target plane.

[0053] In this embodiment, the target plane is uniquely determined according to the plane where the target vertex is located. Specifically, still taking the geometric model given in the above embodiment as an example, please refer to Figure 4 , assuming that the target vertex is located at a, then the reference vertices b, c, and d can be determined from point a, and b, c, and d all share a common edge with point a, that is, the line connecting b, c, d and a is an edge in the rectangular parallelepiped geometric model. It is easy to understand that, in combination with the content of the above embodiment, cameras are provided on vertices a, b, c, and d. Since the size of the baseline in the binocular vision system directly affects the accuracy of the depth estimation of the binocular vision system, the FoV between two cameras with longer baselines should overlap as much as possible to maximize the advantage of the long baseline, that is, the orientation of different cameras in a binocular camera group should make the FOV of each camera evenly and reasonably distributed to all directions of the aircraft.

[0054] Based on this understanding, since any three points in three-dimensional space that are not in a straight line can determine a plane, the reference vertices b, c, and d can at least determine the line bc between b and c, and the line cd between c and d, and then bc and cd can determine a target plane, and points b, c, and d are all located in the target plane. The target plane can be used to indicate the relationship between the FOVs of a point outside the plane (i.e., the camera at point a) and the cameras at points b, c, and d in the target plane when forming a binocular system.

[0055] Of course, in addition to this, if the geometric model is a polyhedron other than a cuboid, the method proposed in this embodiment can still be referred to to determine the target plane corresponding to each target vertex.

[0056] Furthermore, in the above embodiment, the orientation information of the target camera is determined according to the target vertex and the target plane, including: determining a target normal according to the target vertex and the target plane, wherein the target normal passes through the target vertex and is perpendicular to the target plane; determining the direction in which the target camera points to the outside of the geometric model along the target normal as the target direction; and generating the orientation information according to the target direction.

[0057] Among them, the target normal passes through the target vertex and is perpendicular to the target plane. The target normal includes two directions, one end points to the inside of the drone, and the other end points to the outside of the drone, which is the target direction. It is easy to understand that for the camera, its optical center should point to the outside of the drone to collect images in the environment. Therefore, the above-mentioned target direction is the orientation of the target camera at the target vertex. According to the orientation, the corresponding orientation information can be generated. When the orientation information is read by other electronic devices, the target direction corresponding to the target vertex can be uniquely determined.

[0058] It should be noted that, in each of the above-mentioned embodiments, there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present application, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0059] As another aspect of the embodiment of the present application, the embodiment of the present application provides a camera layout device for a drone. The camera layout device for a drone can be a software module, which includes several instructions stored in a memory, and the processor can access the memory and call the instructions for execution to complete the camera layout method for the drone described in the above embodiments.

[0060] In some embodiments, the camera layout device of the drone can also be constructed by hardware devices. For example, the camera layout device of the drone can be constructed by one or more chips, and each chip can work in coordination with each other to complete the camera layout method of the drone described in each of the above embodiments. For another example, the camera layout device of the drone can also be constructed by various logic devices, such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0061] For details, please refer to Figure 5 , Figure 5 The figure is a schematic structural diagram of a camera layout device for a drone, the camera layout device for the drone comprising:

[0062] The data acquisition module 310 is used to acquire the fuselage data of the UAV;

[0063] A position determination module 320, configured to determine a camera position model according to the fuselage data, wherein the camera position model includes position information of a plurality of cameras on the drone, wherein the plurality of cameras are distributed in pairs on the fuselage of the drone to form a plurality of camera groups;

[0064] The orientation determination module 330 is used to determine the orientation information of each camera in the camera position model according to a preset camera orientation rule, so that a plurality of mutually perpendicular baselines are formed between each camera in each camera group.

[0065] In one possible implementation, the position determination module 320, when used to determine the camera position model based on the body data of the drone, is specifically used to: determine the body parameters of the drone based on the body data of the drone; determine the number of cameras based on the body parameters; and determine the position information of each camera on the body of the drone in sequence based on the number of cameras and preset camera distribution rules to obtain the camera position model.

[0066] In a possible implementation, the position determination module 320, when used to determine the number of the plurality of cameras according to the body parameters of the drone, is specifically used to: determine the body model of the drone according to the body parameters of the drone; determine a geometric model matching the body model of the drone in a preset database according to the body model of the drone, the geometric model comprising a plurality of vertices; and determine the number of the cameras according to the number of vertices in the geometric model.

[0067] In one possible implementation, the orientation determination module 330, when used to determine the orientation information of each camera in the camera position model according to a preset camera orientation rule, is specifically used to: determine a target camera, where the target camera is distributed on the fuselage of the drone; determine the position information of the target camera according to the camera position model; and determine the orientation information of the target camera according to the position information of the target camera.

[0068] In a possible implementation, the orientation determination module 330 is used to determine the orientation information of the target camera based on the position information of the target camera, including: determining the vertex at which the target camera is located in the geometric model as the target vertex; based on the target vertex, determining a target plane in the geometric model, the target vertex being outside the target plane; and determining the orientation information of the target camera based on the target vertex and the target plane.

[0069] In one possible implementation, the orientation determination module 330, when used to determine a target plane that meets preset conditions in the geometric model based on the target vertex, is specifically used to: determine at least three reference vertices adjacent to the target point in the geometric model based on the target vertex, each of the reference vertices being on a vertex of the geometric model and sharing an edge with the target vertex; determine a plurality of target straight lines based on the reference vertices, each of the target straight lines being connected between any two reference vertices; and determine the target plane based on the target straight lines, each of the target straight lines intersecting within the target plane.

[0070] In one possible implementation, the orientation determination module 330, when used to determine the orientation information of the target camera based on the target vertex and the target plane, is specifically used to: determine a target normal based on the target vertex and the target plane, wherein the target normal passes through the target vertex and is perpendicular to the target plane; determine that the direction in which the target camera points to the outside of the geometric model along the target normal is the target direction; and generate the orientation information based on the target direction.

[0071] Further, see Figure 6 , Figure 6 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes one or more processors 41 and a memory 42. The memory 42 is connected to the one or more processors 41, for example, connected to the processor 41 via a bus.

[0072] The processor 41 is configured to support the electronic device to perform the corresponding functions in the method in the above method embodiment. The processor 41 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0073] The memory 42 is used to store program codes, etc. The memory 42 may include a volatile memory (VM), such as a random access memory (RAM); the memory 42 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 42 may also include a combination of the above-mentioned types of memories.

[0074] The memory 42 can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the camera layout method of the drone in the embodiment of the present application. The processor 41 executes various functional applications and data processing of the camera layout method of the drone and the camera layout device of the drone by running the non-volatile software programs, instructions and modules stored in the memory 42, that is, realizes the functions of each module or unit of the camera layout method of the drone and the camera layout device of the drone provided in the above method embodiment.

[0075] The memory 42 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function. The data storage area may store data created according to the use of the camera layout device of the drone, etc. In some embodiments, the memory 42 may optionally include a memory remotely arranged relative to the processor 41, and these remote memories may be connected to the camera layout device of the drone via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0076] One or more modules are stored in the memory 42. When executed by one or more processors 41, the camera layout method of the drone in any of the above method embodiments is executed, for example, the method steps described in the above method embodiments are executed to realize the functions of the modules described in the above device embodiments.

[0077] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the camera layout method of the drone as described in the above embodiment.

[0078] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0079] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A camera layout method for an unmanned aerial vehicle, characterized in that: The camera layout method of the drone includes: Get the drone's fuselage data; Determine a camera position model according to the fuselage data, wherein the camera position model includes position information of a plurality of cameras on the drone, wherein the plurality of cameras are distributed in pairs on the fuselage of the drone to form a plurality of camera groups; According to a preset camera orientation rule, orientation information of each camera in the camera position model is determined so that a plurality of mutually perpendicular baselines are formed between each camera in each camera group.

2. The camera layout method of a drone according to claim 1, characterized in that: Determining the camera position model according to the fuselage data of the drone includes: Determining fuselage parameters of the UAV according to the fuselage data of the UAV; Determine the number of cameras according to the body parameters; According to the number of cameras and a preset camera distribution rule, the position information of each camera on the fuselage of the drone is determined in sequence to obtain the camera position model.

3. The camera layout method of a drone according to claim 2, characterized in that: The step of determining the number of the plurality of cameras according to the fuselage parameters of the drone comprises: Determining a fuselage model of the UAV according to the fuselage parameters of the UAV; According to the fuselage model of the drone, determining a geometric model matching the fuselage model of the drone in a preset database, wherein the geometric model includes a plurality of vertices; The number of the cameras is determined according to the number of each vertex in the geometric model.

4. The camera layout method of a drone according to claim 1, characterized in that: The step of determining the orientation information of each camera in the camera position model according to a preset camera orientation rule includes: Determine a target camera, where the target camera is distributed on a fuselage of the UAV; Determining the position information of the target camera according to the camera position model; The orientation information of the target camera is determined according to the position information of the target camera.

5. The camera layout method of a drone according to claim 4, characterized in that: The camera position model includes a geometric model corresponding to the fuselage of the drone, the target camera is located at a vertex of the geometric model, and determining the orientation information of the target camera according to the position information of the target camera includes: Determine the vertex where the target camera is located in the geometric model as the target vertex; determining a target plane in the geometric model based on the target vertex, the target vertex being outside the target plane; The orientation information of the target camera is determined according to the target vertex and the target plane.

6. The camera layout method of a drone according to claim 5, characterized in that: The step of determining a target plane satisfying a preset condition in the geometric model based on the target vertex includes: Based on the target vertex, determining at least three reference vertices adjacent to the target point in the geometric model, each of the reference vertices being located on a vertex of the geometric model and sharing a common edge with the target vertex; Determine a plurality of target straight lines according to the reference vertices, each of the target straight lines being connected between any two reference vertices; The target plane is determined according to the target straight lines, and each of the target straight lines intersects within the target plane.

7. The camera layout method of a drone according to claim 5, characterized in that: The step of determining the orientation information of the target camera according to the target vertex and the target plane includes: Determine a target normal according to the target vertex and the target plane, wherein the target normal passes through the target vertex and is perpendicular to the target plane; Determine the direction in which the target camera points to the outside of the geometric model along the target normal as the target direction; The orientation information is generated according to the target direction.

8. A drone, characterized in that: A plurality of cameras are arranged on the fuselage of the drone, and the plurality of cameras are distributed on the fuselage of the drone according to the camera layout method for a drone according to any one of claims 1 to 7.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory is connected to the processor, the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the electronic device implements the camera layout method of the drone as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the camera layout method for a drone as described in any one of claims 1 to 7.

Images