Camera layout method of unmanned aerial vehicle, unmanned aerial vehicle, electronic device and storage medium

Abstract

The embodiment of the application relates to the technical field of unmanned planes, and particularly relates to a camera layout method of an unmanned plane, an unmanned plane, an electronic device and a storage medium, the method comprising: acquiring fuselage data of the unmanned plane; determining a camera position model according to the fuselage data, the camera position model comprising position information of a plurality of cameras on the unmanned plane, the plurality of cameras being distributed in pairs on a fuselage of the unmanned plane to form a plurality of camera groups; and determining orientation information of each camera in the camera position model according to a preset camera orientation rule, so that a plurality of mutually perpendicular base lines are formed between each camera in each of the camera groups. According to the method, the base lines between the plurality of camera groups are mutually perpendicular, so when binocular cameras formed by certain camera groups cause ambiguity and lead to depth measurement failure, ambiguity can be eliminated by another pair of groups perpendicular to the base lines of the groups, effectively avoiding the invalidity of stereo matching, and improving the applicability of the unmanned plane in various scenes.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of unmanned aerial vehicles, and particularly relate to a camera layout method of an unmanned aerial vehicle, the unmanned aerial vehicle, an electronic device, and a storage medium. BACKGROUND

[0002] An unmanned aerial vehicle refers to a non-manned aircraft with a task load, which is controlled by a remote control device or a self-provided program control device. If a collision with an obstacle occurs during the execution of a predetermined task by the unmanned aerial vehicle, the unmanned aerial vehicle may be crashed, causing a loss. Therefore, the unmanned aerial vehicle for low-altitude flight generally integrates an obstacle avoidance function and an obstacle circumvention function at present, so as to reduce the occurrence of collision accidents. In related technologies, an image sensor such as a camera is generally arranged on the unmanned aerial vehicle, so that the unmanned aerial vehicle has a certain three-dimensional spatial environment perception capability, and thus achieves the effect of omnidirectional obstacle avoidance and circumvention of the unmanned aerial vehicle.

[0003] At present, most unmanned aerial vehicles use a binocular stereo matching perception scheme, that is, the depth of an object in a scene is inferred by analyzing the parallax information of images captured by two cameras, and then a stereo matching algorithm is used to reconstruct the three-dimensional structure of the scene. However, the effect of this scheme is not good in some specific scenes. Any point in the environment can be observed by at most two cameras, so when stereo matching is performed, ambiguity may be generated, the error of depth measurement becomes large or even invalid, and thus the obstacle avoidance and circumvention effects of the unmanned aerial vehicle are affected. For example, it is assumed that there is a linear obstacle in the environment, and the binocular baseline of the two cameras is parallel to the linear obstacle. At this time, since the linear obstacle coincides with the binocular baseline of the camera, and the linear obstacle generally lacks texture, when the stereo matching algorithm searches along the baseline, many ambiguous matching points are found, and an accurate result cannot be obtained, which affects the obstacle avoidance and circumvention effects of the unmanned aerial vehicle. SUMMARY

[0004] An object of embodiments of the present application is to provide a camera layout method of an unmanned aerial vehicle, the unmanned aerial vehicle, an electronic device, and a storage medium, so as to solve the technical problem that related technologies are prone to ambiguity when stereo matching is performed, the error of depth measurement becomes large or even invalid, and the obstacle avoidance and circumvention effects of the unmanned aerial vehicle are affected.

[0005] In a first aspect, embodiments of the present application provide a camera layout method of an unmanned aerial vehicle, comprising: acquiring body data of the unmanned aerial vehicle; determining a camera position model according to the body data, the camera position model comprising position information of a plurality of cameras on the unmanned aerial vehicle, the plurality of cameras being distributed in pairs on a body of the unmanned aerial vehicle to form a plurality of camera groups; and determining orientation information of each camera in the camera position model according to a preset camera orientation rule, so that a plurality of mutually perpendicular baselines are formed between the cameras in each camera group.

[0006] With reference to the first aspect, in a possible implementation manner, the determining the camera position model according to the fuselage data of the unmanned aerial vehicle comprises: determining a fuselage parameter of the unmanned aerial vehicle according to the fuselage data of the unmanned aerial vehicle; determining a number of cameras according to the fuselage parameter; sequentially determining position information of each camera on the fuselage of the unmanned aerial vehicle according to the number of cameras and a preset camera distribution rule, to obtain the camera position model.

[0007] With reference to the first aspect, in a possible implementation manner, the determining the number of the plurality of cameras according to the fuselage parameter of the unmanned aerial vehicle comprises: determining a fuselage model of the unmanned aerial vehicle according to the fuselage parameter of the unmanned aerial vehicle; determining a geometric model matching the fuselage model of the unmanned aerial vehicle in a preset database according to the fuselage model of the unmanned aerial vehicle, the geometric model comprising a plurality of vertices; and determining the number of the cameras according to a number of the vertices in the geometric model.

[0008] With reference to the first aspect, in a possible implementation manner, the determining the orientation information of each camera in the camera position model according to the preset camera orientation rule comprises: determining a target camera, the target camera being distributed on the fuselage of the unmanned aerial vehicle; determining position information of the target camera according to the camera position model; and determining orientation information of the target camera according to the position information of the target camera.

[0009] With reference to the first aspect, in a possible implementation manner, the camera position model comprises a geometric model corresponding to the fuselage of the unmanned aerial vehicle, the target camera being at a vertex of the geometric model, and the determining the orientation information of the target camera according to the position information of the target camera comprises: determining a vertex at which the target camera is located in the geometric model as a target vertex; determining a target plane in the geometric model based on the target vertex, the target vertex being outside the target plane; and determining the orientation information of the target camera according to the target vertex and the target plane.

[0010] With reference to the first aspect, in a possible implementation manner, the determining the target plane satisfying a preset condition in the geometric model based on the target vertex comprises: determining at least three reference vertices adjacent to the target point in the geometric model based on the target vertex, each of the reference vertices being at a vertex of the geometric model and sharing an edge with the target vertex; determining a plurality of target straight lines according to the reference vertices, each of the target straight lines being connected between any two reference vertices; and determining the target plane according to the target straight lines, each of the target straight lines intersecting in the target plane.

[0011] In conjunction with the first aspect, in one possible implementation, determining the orientation information of the target camera based on the target vertex and the target plane includes: determining a target normal based on the target vertex and the target plane, wherein the target normal passes through the target vertex and is perpendicular to the target plane; determining the direction in which the target camera points outward from the geometric model along the target normal as the target direction; and generating the orientation information based on the target direction.

[0012] In a second aspect, embodiments of this application also propose an unmanned aerial vehicle (UAV) having multiple cameras mounted on its fuselage, the multiple cameras being distributed on the fuselage of the UAV using the camera layout method described in the first aspect.

[0013] In a third aspect, embodiments of this application also provide an electronic device, including a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor causing the electronic device to perform the method as described in the first aspect when executing the one or more computer programs.

[0014] In a fourth aspect, embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect.

[0015] The embodiments of this application can achieve the following technical effects:

[0016] Based on the method proposed in this application embodiment, when arranging cameras on the drone's fuselage, the drone's fuselage data is first acquired. Then, based on the fuselage data, a camera position model is determined. The camera position model includes the position information of multiple cameras on the drone. The multiple cameras are distributed in pairs on the drone's fuselage to form multiple camera groups. Finally, according to a preset camera orientation rule, the orientation information of each camera in the camera position model is determined so that multiple mutually perpendicular baselines are formed between the cameras in each camera group. Based on this method, multiple cameras are distributed on the drone's fuselage to form multiple camera groups. The combination of multiple camera groups achieves omnidirectional multi-view perception coverage, and the baselines between each camera group are perpendicular. When ambiguity caused by the binocular cameras formed by some camera groups leads to depth measurement failure, the ambiguity can be eliminated by another pair of binocular cameras with a baseline perpendicular to that binocular camera group. This effectively avoids the situation of invalid stereo matching, thereby improving the drone's obstacle avoidance and obstacle bypass technology, and further improving the drone's applicability in various specific scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of an obstacle avoidance scenario for a drone provided in an embodiment of this application;

[0019] Figure 2 A flowchart illustrating a camera layout method for a drone provided in an embodiment of this application;

[0020] Figure 3 A schematic diagram of a geometric model of a drone provided in an embodiment of this application;

[0021] Figure 4 A schematic diagram of the target plane in a geometric model of a drone provided in this application embodiment;

[0022] Figure 5 A schematic diagram of a camera layout device for a drone provided in an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0025] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0026] To better explain this application, please first refer to... Figure 1 , Figure 1The diagram shown illustrates an obstacle avoidance scenario for a drone. Figure 1 As shown, the scene includes a drone and an obstacle P in front of the drone. The drone is equipped with four cameras, C1, C2, C3, and C4. Any two cameras can form a binocular camera group. Cameras C1, C2, C3, and C4 are used to capture images of the drone's surrounding environment. By comparing the disparity information ("disparity" being the pixel coordinate offset of the same object in two images) captured by each camera in the binocular camera group, a stereo matching algorithm is further used to find corresponding points in the two images to calculate the disparity. Based on the camera's intrinsic and extrinsic parameters, a depth map is calculated (a depth map is an image where the grayscale value or color of each pixel represents the distance to the corresponding point in the scene, typically used to represent the spatial structure of a 3D scene). Finally, the depth of objects in the scene is inferred from the depth map, thus reconstructing the 3D structure of the scene.

[0027] However, as Figure 1 As shown, if the obstacle P in front of the drone is a linear obstacle (such as a power line), and this obstacle P is parallel to the binocular baseline of the drone's stereo camera system (exemplarily, the baseline L1 between cameras C1 and C2), the stereo matching algorithm will find many ambiguous matching points when searching along the baseline because the linear obstacle coincides with the baseline and linear obstacles usually lack texture, thus failing to obtain accurate results. It should be noted that in application scenarios such as drone power line inspection, linear obstacles parallel to the binocular baseline are quite common, thus significantly limiting the application of drones in such scenarios.

[0028] Based on this, embodiments of this application propose a camera layout method for drones, which enables drones to be used in applications such as... Figure 1 The obstacle avoidance scenario shown effectively reduces ambiguity in stereo matching.

[0029] Specifically, please refer to Figure 2 , Figure 2 The diagram shown is a flowchart of this method, including:

[0030] Step S10: Obtain the drone's fuselage data;

[0031] In this embodiment, the drone's fuselage data mainly refers to data related to the drone's three-dimensional structure and dimensions. This data can be obtained through a preset database or measured and collected in real time using specific tools. For example, assuming a preset database stores fuselage data for various types of drones, when deploying cameras on a target drone (i.e., the drone in the current layout scheme), the identifiers on the drone's fuselage, such as the SN (Serial Number), can be identified first. Then, the fuselage data corresponding to the target drone can be extracted from the preset database using the SN for use in subsequent steps.

[0032] In some embodiments, the camera layout method for a drone is executed by a communication-enabled electronic device, which may take the form of various mobile terminals, computers, tablets, or other similar industrial equipment. This electronic device can communicate with a backend server and access a preset database to obtain the aforementioned drone body data. For example, the drone body or external accessories typically have a unique identification code (such as a barcode, QR code, etc.), and the electronic device is equipped with a barcode scanner or other similar information collection device, which can scan the identification code, read the drone's unique serial number (SN), and transmit the SN to the backend server. The server's database pre-stores the body data of various drones at the time of manufacture, and each drone's body data uniquely corresponds to its SN. Therefore, the backend server can send the body data corresponding to the SN to the electronic device, thereby enabling the electronic device to obtain the drone's body data.

[0033] In other embodiments, the electronic device is equipped with a device for collecting the fuselage data of the drone, such as an image acquisition device or an infrared scanning device, which can scan the fuselage of the drone to generate relevant data, and combine the relevant data with the corresponding computer vision algorithm to analyze the relevant data and generate fuselage data in a specific format. This embodiment does not impose too many limitations on this.

[0034] Step S20: Determine the camera position model based on the fuselage data. The camera position model includes the position information of multiple cameras on the UAV. The multiple cameras are distributed in pairs on the fuselage of the UAV to form multiple camera groups.

[0035] In this embodiment, the camera position model refers to a data model that can indicate the position information of multiple cameras on the drone body. Under the guidance of this model, the position of each camera on the drone body can be abstracted as a target point. The setting of the target point ignores the specific structure of the camera and regards its position on the drone body as a point outside the drone body.

[0036] Those skilled in the art will understand that, based on the principles of multi-view vision, multiple cameras should be distributed in pairs on the fuselage of the drone, forming a camera group between each pair of cameras. This allows the images jointly acquired by different cameras within each camera group to be used for 3D scene reconstruction. Alternatively, the number of cameras in each camera group can be a finite n (n≥2), with each of the n cameras in the group acquiring different images from the environment to determine depth information, thereby achieving the effect of 3D scene reconstruction.

[0037] It should be noted that the camera position model includes at least two types of data: the fuselage model used to indicate the drone's body, and the position information of each camera within that fuselage model. In other words, the specific settings of the fuselage model should be able to indicate an abstract three-dimensional model. Based on this understanding, this three-dimensional model can correspond to at least a specific reference frame. For example, the roll, pitch, and yaw directions of the drone can be determined based on its fuselage, thus obtaining coordinate axes in three directions, thereby obtaining a reference frame about the drone's fuselage. Therefore, the position information of each camera on the drone's fuselage can be uniquely determined and represented by its coordinates mapped onto this reference frame. As a feasible implementation, the position information of each camera within the fuselage model is represented by a unique ternary array, where each element of the array indicates its coordinates in the roll, pitch, and yaw directions, respectively.

[0038] Step S30: Determine the orientation information of each camera in the camera position model according to the preset camera orientation rules, so that multiple mutually perpendicular baselines are formed between each camera in each camera group.

[0039] In this embodiment, after determining the position of each camera, it is also necessary to determine the specific orientation of each camera so that the baselines of different camera groups and the groups of cameras are perpendicular to each other. Thus, when the image captured by a certain camera group in the environment is ambiguous, the image is captured by another camera group whose baseline is perpendicular to that camera group, thereby achieving the effect of eliminating ambiguity.

[0040] Using the above method, this embodiment proposes a camera layout scheme for UAVs, which can effectively eliminate the ambiguity of stereo matching. Specifically, if each camera in the above camera position model is a fisheye camera, and the field of view (FoV) of each fisheye camera is 180° (it can be larger or smaller than this, but cannot be too small; the lower limit of FoV needs to be calculated), then it is easy to understand that any point around the UAV can be observed by exactly four cameras, thus achieving omnidirectional multi-view coverage. Figure 1Taking the application scenario shown as an example, the four cameras C1, C2, C3 and C4 can form at least two sets of binocular cameras with mutually perpendicular baselines, namely (C1, C2) and (C2-C3). In this way, when the linear obstacle P is ambiguous in one pair of binocular cameras C1-C2 (at this time, the baseline L1 between the linear obstacle P and the binocular cameras (C1, C2) is parallel), for the other pair of binocular cameras C2-C3, obviously, since the baseline L1 between the binocular cameras (C1, C2) is perpendicular to the baseline L2 between the binocular cameras (C2, C3), the ambiguity can be eliminated. This eliminates the ambiguity caused by the linear obstacle being horizontal or nearly horizontal with the binocular baseline (i.e. the baseline between the binocular cameras), and avoids the situation of invalid stereo matching.

[0041] For example, suppose there is a horizontal power line directly in front of the drone. The drone's body has, but is not limited to, three cameras: "upper left front," "upper right front," and "upper and lower front." If two of the "upper left front" and "upper right front" cameras are used to form a binocular system, the horizontal power line will be parallel to the baseline of the two cameras, causing ambiguity and leading to depth estimation failure. However, two cameras can be used to form another binocular system, and the baseline of this system is perpendicular to the horizontal power line, thus eliminating the ambiguity. Furthermore, the camera layout in this embodiment maximizes the baseline and evenly and reasonably distributes the Field of View (FoV) to all directions of the drone according to the baseline length, maximizing the use of the camera's field of view coverage and long baseline advantage to improve ranging accuracy and achieve the best perception effect.

[0042] Furthermore, in some embodiments, determining the camera position model based on the drone's fuselage data includes: determining the drone's fuselage parameters based on the drone's fuselage data; determining the number of cameras based on the fuselage parameters; and sequentially determining the position information of each camera on the drone's fuselage based on the number of cameras and a preset camera distribution rule to obtain the camera position model.

[0043] The drone's fuselage parameters refer to parameters related to the three-dimensional shape of the drone's fuselage, such as its length, width, height, the endpoints and angles at which the wings connect to the fuselage, etc. These parameters can be extracted from the drone's fuselage data. From these parameters, the number of cameras to be deployed on the drone's fuselage can be determined. After determining the number of cameras, the position of each camera on the fuselage is calculated sequentially, with each camera's position corresponding to a point on the outside of the fuselage, thus obtaining a camera position model.

[0044] Specifically, determining the number of cameras based on the drone's fuselage parameters includes: determining the drone's fuselage model based on the drone's fuselage parameters; determining a geometric model matching the drone's fuselage model in a preset database based on the drone's fuselage model, the geometric model including multiple vertices; and determining the number of cameras based on the number of each vertex in the geometric model.

[0045] It is easy to understand that a unique fuselage model can be determined from the fuselage parameters of the drone. This fuselage model is an abstraction of the three-dimensional structure of the drone fuselage, including the structural features of the drone fuselage. The data stored in the preset database indicates the correspondence between various fuselage models and geometric models. Thus, the database can be traversed to determine the corresponding geometric model based on the fuselage model of the drone.

[0046] It should be noted that this geometric model ignores structural features on the drone's fuselage that are unrelated to the camera layout, thus transforming the complex fuselage model into a simpler geometric model. This facilitates a simplified analysis of the overall fuselage to determine the number of cameras. For details, please refer to... Figure 3 ,like Figure 3 As shown, based on the fuselage model (1) of the UAV, it can be abstracted into a geometric model (2). The geometric model (2) is a cuboid with 8 vertices, and a camera can be set at each of the 8 vertices. Therefore, the number of vertices of the geometric model (2) is the determined number of cameras. Of course, in addition to this, according to the characteristics of the UAV fuselage model, the geometric model can also be other geometric bodies that match the fuselage model, such as regular polyhedra, spheres or ellipsoids of various shapes, etc. Those skilled in the art can set it according to the structural characteristics of the fuselage.

[0047] Furthermore, in the above embodiments, determining the orientation information of each camera in the camera position model according to a preset camera orientation rule includes: determining a target camera, wherein the target camera is distributed on the fuselage of the UAV; determining the position information of the target camera according to the camera position model; and determining the orientation information of the target camera according to the position information of the target camera.

[0048] In this embodiment, the target camera refers to any camera whose orientation is yet to be determined within the camera position model after the model has been determined. After the camera position model is determined (i.e., the number of cameras is fixed and the positions of each camera are fixed), it is also necessary to determine the specific orientation of each camera. Assuming that the camera set C with fixed positions is determined by the camera position model: {c1,c2,c3...cn}, c1, c2, c3...cn in the camera set C are determined as target cameras in a certain order until the orientation of all cameras in the camera set C is determined.

[0049] It is easy to understand that, in this embodiment, since each camera is located at the vertex of the above-mentioned geometric model (2), the angle formed by the specific orientation of each camera and any edge it is located should be between 0° and 90°.

[0050] Specifically, as can be seen from the above embodiments, any camera in the camera set C is located at a vertex of the geometric model. The orientation information of the target camera is determined based on the position information of the target camera, including: determining the vertex where the target camera is located in the geometric model as the target vertex; determining a target plane in the geometric model based on the target vertex, wherein the target vertex is outside the target plane; and determining the orientation information of the target camera based on the target vertex and the target plane.

[0051] Those skilled in the art will understand that, in this embodiment, the orientation information of the target camera at the target vertex is determined by the target vertex and the target plane associated with the target vertex, thereby determining the orientation information of the target camera based on the geometric relationship between the target vertex and the target plane.

[0052] The step of determining a target plane that meets preset conditions in the geometric model based on the target vertex includes: determining at least three reference vertices adjacent to the target vertex in the geometric model, each reference vertex being located on a vertex of the geometric model and sharing an edge with the target vertex; determining a plurality of target lines based on the reference vertices, each target line connecting any two reference vertices; and determining the target plane based on the target lines, wherein all the target lines intersect within the target plane.

[0053] In this embodiment, the target plane is uniquely determined based on the plane where the target vertex is located. Specifically, taking the geometric model given in the above embodiment as an example, please refer to... Figure 4 Assuming the target vertex is located at position 'a', reference vertices b, c, and d can be determined from point 'a'. Furthermore, b, c, and d all share an edge with point 'a', meaning the lines connecting b, c, and d to 'a' are all edges in the rectangular prism geometric model. It's easy to understand that, based on the above embodiment, cameras are positioned at vertices a, b, c, and d. Since the baseline size in a binocular vision system directly affects the accuracy of depth estimation, the FoV (Field of View) between two cameras with longer baselines should overlap as much as possible to maximize the advantage of the longer baseline. In other words, the orientation of different cameras in a binocular camera group should ensure that the FoV of each camera is evenly and reasonably distributed across the various directions of the aircraft.

[0054] Based on this understanding, since any three points not on a straight line in three-dimensional space can determine a plane, the reference vertices b, c, and d can at least determine the line bc connecting b and c, and the line cd connecting c and d. Furthermore, bc and cd can determine a target plane, and points b, c, and d are all located within this target plane. This target plane can be used to indicate the FOV relationship between a point outside the plane (i.e., camera a) and the cameras at points b, c, and d within the target plane when forming a binocular camera group.

[0055] Of course, in addition to this, if the geometric model is a non-cubic polyhedron, the method proposed in this embodiment can still be used to determine the target plane corresponding to each target vertex.

[0056] Further, in the above embodiments, determining the orientation information of the target camera based on the target vertex and the target plane includes: determining a target normal based on the target vertex and the target plane, wherein the target normal passes through the target vertex and is perpendicular to the target plane; determining the direction in which the target camera points outward from the geometric model along the target normal as the target direction; and generating the orientation information based on the target direction.

[0057] The target normal passes through the target vertex and is perpendicular to the target plane. This target normal has two directions: one end points inward to the inside of the drone, and the other end points outward to the outside of the drone. This is the target direction. It is easy to understand that for a camera, its optical center should point outward to the outside of the drone to capture images of the environment. Therefore, the above-mentioned target direction is the orientation of the target camera at the target vertex. Based on this orientation, the corresponding orientation information can be generated. When the orientation information is read by other electronic devices, the target direction corresponding to the target vertex can be uniquely determined.

[0058] It should be noted that in the above embodiments, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0059] As another aspect of the embodiments of this application, this application provides a camera layout device for a drone. The camera layout device for the drone can be a software module, which includes several instructions stored in a memory. A processor can access the memory, invoke the instructions, and execute them to complete the camera layout method for the drone described in the various embodiments above.

[0060] In some embodiments, the camera placement device for the UAV can also be constructed from hardware components. For example, the camera placement device can be constructed from one or more chips, which can work in coordination to complete the camera placement method for the UAV described in the various embodiments above. As another example, the camera placement device can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0061] Specifically, please refer to Figure 5 , Figure 5 The diagram shows a schematic of a camera layout device for a drone, which includes:

[0062] The data acquisition module 310 is used to acquire the fuselage data of the drone;

[0063] The position determination module 320 is used to determine a camera position model based on the fuselage data. The camera position model includes the position information of multiple cameras on the UAV. The multiple cameras are distributed in pairs on the fuselage of the UAV to form multiple camera groups.

[0064] Orientation determination module 330 is used to determine the orientation information of each camera in the camera position model according to a preset camera orientation rule, so that multiple mutually perpendicular baselines are formed between each camera in each camera group.

[0065] In one possible implementation, the position determination module 320, when determining the camera position model based on the drone's fuselage data, specifically performs the following steps: determining the drone's fuselage parameters based on the drone's fuselage data; determining the number of cameras based on the fuselage parameters; and sequentially determining the position information of each camera on the drone's fuselage based on the number of cameras and a preset camera distribution rule, thereby obtaining the camera position model.

[0066] In one possible implementation, the position determination module 320, when determining the number of multiple cameras based on the fuselage parameters of the UAV, specifically performs the following steps: determining the fuselage model of the UAV based on the fuselage parameters of the UAV; determining a geometric model matching the fuselage model of the UAV in a preset database based on the fuselage model of the UAV, the geometric model including multiple vertices; and determining the number of cameras based on the number of each vertex in the geometric model.

[0067] In one possible implementation, the orientation determination module 330, when determining the orientation information of each camera in the camera position model according to a preset camera orientation rule, is specifically used to: determine a target camera, the target camera being distributed on the fuselage of the UAV; determine the position information of the target camera according to the camera position model; and determine the orientation information of the target camera according to the position information of the target camera.

[0068] In one possible implementation, the orientation determination module 330 is used to determine the orientation information of the target camera based on the position information of the target camera, including: determining the vertex where the target camera is located in the geometric model as a target vertex; determining a target plane in the geometric model based on the target vertex, wherein the target vertex is outside the target plane; and determining the orientation information of the target camera based on the target vertex and the target plane.

[0069] In one possible implementation, the orientation determination module 330, when determining a target plane in the geometric model that satisfies preset conditions based on the target vertex, is specifically configured to: determine at least three reference vertices adjacent to the target vertex in the geometric model, each reference vertex being located on a vertex of the geometric model and sharing an edge with the target vertex; determine a plurality of target lines based on the reference vertices, each target line connecting any two reference vertices; and determine the target plane based on the target lines, wherein all target lines intersect within the target plane.

[0070] In one possible implementation, the orientation determination module 330, when determining the orientation information of the target camera based on the target vertex and the target plane, specifically performs the following: determining a target normal line based on the target vertex and the target plane, wherein the target normal line passes through the target vertex and is perpendicular to the target plane; determining the direction in which the target camera points outward from the geometric model along the target normal line as the target direction; and generating the orientation information based on the target direction.

[0071] Further, see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device includes one or more processors 41 and a memory 42. The memory 42 is connected to one or more processors 41, for example, via a bus.

[0072] Processor 41 is configured to support the electronic device in performing the corresponding functions in the methods described in the above method embodiments. Processor 41 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0073] Memory 42 is used to store program code, etc. Memory 42 may include volatile memory (VM), such as random access memory (RAM); memory 42 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 42 may also include combinations of the above types of memory.

[0074] The memory 42 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the camera layout method of the UAV in the embodiments of this application. The processor 41 executes various functional applications and data processing of the UAV camera layout method and the UAV camera layout device by running the non-volatile software programs, instructions, and modules stored in the memory 42, that is, it realizes the functions of each module or unit of the UAV camera layout method and the UAV camera layout device provided in the above method embodiments.

[0075] The memory 42 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the drone's camera layout device, etc. In some embodiments, the memory 42 may optionally include memory remotely located relative to the processor 41, which can be connected to the drone's camera layout device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0076] One or more modules are stored in memory 42. When executed by one or more processors 41, they execute the camera layout method of the UAV in any of the above method embodiments. For example, they execute the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0077] This application also provides a computer-readable storage medium storing a computer program, which includes program instructions that, when executed by a computer, cause the computer to perform the camera layout method for a drone as described in the foregoing embodiments.

[0078] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0079] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A camera layout method for an unmanned aerial vehicle (UAV), characterized in that, The camera layout method for the drone includes: Acquire drone fuselage data; Based on the fuselage data, a camera position model is determined. The camera position model includes the position information of multiple cameras on the UAV and a geometric model corresponding to the fuselage of the UAV. The multiple cameras are distributed in pairs on the fuselage of the UAV to form multiple camera groups. Based on preset camera orientation rules, the orientation information of each camera in the camera position model is determined so that multiple mutually perpendicular baselines are formed between each camera in each camera group; including: Identify the target camera, which is distributed on the fuselage of the UAV and located at a vertex of the geometric model; The vertex where the target camera is located in the geometric model is defined as the target vertex; Based on the target vertex, a target plane is determined in the geometric model, wherein the target vertex is outside the target plane; Based on the target vertex and the target plane, a target normal is determined, wherein the target normal passes through the target vertex and is perpendicular to the target plane; The direction in which the target camera points outward from the geometric model along the target normal is defined as the target direction. The orientation information is generated based on the target direction.

2. The camera layout method for a UAV according to claim 1, characterized in that, The step of determining the camera position model based on the drone's fuselage data includes: Based on the fuselage data of the drone, determine the fuselage parameters of the drone; Determine the number of cameras based on the aforementioned body parameters; Based on the number of cameras and the preset camera distribution rules, the position information of each camera on the drone body is determined sequentially to obtain the camera position model.

3. The camera layout method for a drone according to claim 2, characterized in that, Determining the number of cameras based on the drone's fuselage parameters includes: Based on the fuselage parameters of the drone, determine the fuselage model of the drone; Based on the fuselage model of the UAV, a geometric model matching the fuselage model of the UAV is determined in a preset database, the geometric model including multiple vertices; The number of cameras is determined based on the number of vertices in the geometric model.

4. The camera layout method for a UAV according to claim 1, characterized in that, Based on the target vertex, a target plane satisfying preset conditions is determined in the geometric model, including: Based on the target vertex, at least three reference vertices adjacent to the target vertex are determined in the geometric model, each of the reference vertices being located on a vertex of the geometric model and sharing an edge with the target vertex; Based on the reference vertices, multiple target lines are determined, and each target line connects any two reference vertices. Based on the target straight lines, the target plane is determined, and all the target straight lines intersect within the target plane.

5. A drone, characterized in that, The drone has multiple cameras mounted on its fuselage, and the multiple cameras are distributed on the fuselage of the drone according to any one of claims 1-4.

6. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor causing the electronic device to implement the camera layout method for a drone as described in any one of claims 1-4 when executing the one or more computer programs.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the camera layout method for the UAV as described in any one of claims 1-4.

Images