A 3D printing method for drone navigation based on indoor scene positioning

Abstract

A 3D printing method for drone navigation based on indoor scene positioning, comprising: 1. Using strategically arranged infrared cameras to collect omnidirectional dynamic information of the drone flight area; 2. Inputting the collected data into a pre-prepared computing system for data processing and calculating the precise position of the camera in space; 3. Dynamically adjusting the attitude and position of the drone through triangulation and bundle adjustment to ensure that the drone can achieve precise and stable fixed-point flight in complex environments, especially under disturbed conditions; 4. After completing precise positioning, the system performs 3D printing modeling according to the target task, and the drone precisely executes the printing task, depositing printing materials layer by layer to complete the required three-dimensional structure. The system of this application not only improves the operation accuracy and safety of the drone, but also provides a low-cost and high-precision drone positioning and 3D printing solution for fields such as indoor construction and model making.

Description

Technical Field

[0001] The present invention belongs to the technical field of UAV navigation and 3D printing, and specifically relates to a UAV navigation 3D printing method based on indoor scene positioning. Background Art

[0002] During the flight of a UAV, as a common positioning technology, GPS (Global Positioning System) has the problem of non-autonomy relying on external satellite signals. In adverse environmental conditions (such as building occlusion, indoor environment, or bad weather), GPS signals may be lost or weakened, thereby affecting positioning accuracy. In addition, the accuracy of GPS is usually only a few meters, and even reaches dozens of meters in some cases, which is insufficient for precise navigation and difficult to meet actual requirements.

[0003] On the other hand, traditional 3D printing requires a large amount of space. Traditional 3D printers usually require a fixed printing platform, and the size and space of the printing platform are usually fixed. For larger printed objects, traditional 3D printers require a very large space, which limits their application in certain scenarios; and traditional 3D printing is limited to planar printing: traditional 3D printers generally can only print on a horizontal plane and need to provide enough ground space for each object. If vertical or inaccessible areas (such as building facades) need to be printed, traditional 3D printers are not competent. Summary of the Invention

[0004] Aiming at the problems of high price and low accuracy existing in traditional UAV navigation technology so far, the present invention proposes a UAV navigation 3D printing method based on indoor scene positioning.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A UAV navigation 3D printing method based on indoor scene positioning, comprising the following steps:

[0007] (1) Task definition and geometric modeling:

[0008] Before the task starts, first define the printing target and establish its geometric model. According to the geometric model of the task target, the path planning system divides the target task into multiple layers, designs the printing path of each layer layer by layer, sets the height of each layer as h, and the printing task realizes the layered printing of the task by modeling the geometric shape of each layer;

[0009] (2) Path planning and printing trajectory design:

[0010] Based on the task definition and geometric model, the system generates the printing path and flight path. The printing path is usually a circular trajectory. The printing trajectory of each layer is set according to the target radius and the speed of the nozzle. The nozzle moves uniformly along the circular trajectory to ensure the uniformity of material deposition.

[0011] (3) Flight control and real-time path optimization:

[0012] During the printing process, due to external disturbances, the system needs to optimize the flight path and nozzle trajectory in real time. Through real-time feedback and position tracking, the system adjusts the flight speed and nozzle angle in real time according to the error between the actual position of the UAV and the target trajectory, ensuring that the nozzle moves precisely along the designed trajectory.

[0013] As a further improvement of the present invention, in step (1), the task definition and geometric modeling of 3D printing include: realizing the layered printing of the task by modeling the geometric shape of each layer.

[0014] As a further improvement of the present invention, in step (2), the path planning and printing trajectory design include nozzle trajectory design. The nozzle deposits materials along the path trajectory to ensure the uniform distribution of materials in each layer and avoid printing errors; height control and interlayer adjustment. The system accurately tracks the position of the UAV through the infrared camera positioning system to ensure that its vertical height is accurate to the micron level of the layer height, and adjusts the flight height through the path planning system to ensure the accurate docking of the printing position of each layer.

[0015] As a further improvement of the present invention, in step (2),

[0016] (1) Constraint conditions:

[0017] During the printing process, due to possible external disturbances, it is necessary to optimize the path in real time. At this time, it is assumed that the target object is on the circular trajectory of the i-th layer with a radius of R i , then the three-dimensional coordinates M(x, y, z) of the target object are expressed as:

[0018] x = R i ·cosθ

[0019] y = R i ·sinθ

[0020] z = h i .

[0021] (2) Optimization objective:

[0022] All camera poses T i =(R i , t i ) and the positions X of all three-dimensional points j, minimizing the error between the projection results of all cameras and the actually observed image feature points, that is, minimizing the following objective function:

[0023]

[0024] The meaning of the objective function is to minimize the distance between the feature points observed by each camera and the projected points calculated based on the currently estimated camera parameters and three-dimensional point positions;

[0025] where N cams represents the number of cameras; N points represents the number of three-dimensional points; p ij represents the actual observation of the j-th feature point captured by the i-th camera; represents the projected point calculated according to the current camera pose T i and the three-dimensional point position X j ; ρ(·) is a robust kernel function used to reduce the influence of outliers. First, assume ρ(x) = x;

[0026] (3) Error function:

[0027] Define the residual of each observation point as:

[0028] r ij = p ij - p ij (T i , X j )

[0029] (4) The goal is to minimize the following error function:

[0030]

[0031] (5) Gauss-Newton method iterative optimization;

[0032] S1. Linearize the residual function;

[0033] Use Taylor expansion to linearize the error function r ij around the current parameters {T i , X j}:

[0034]

[0035] where:

[0036] ● J ij is the Jacobian matrix of the residual function with respect to the optimization parameters, divided into two parts:

[0037] The Jacobian with respect to the camera pose;

[0038] Jacobian of the three - dimensional point position;

[0039] S2. Construct the linearized objective function;

[0040] After linearization, the objective function of the sum of squared errors becomes:

[0041]

[0042] After expansion, it can be written as:

[0043]

[0044] Where:

[0045] ● J is the global Jacobian matrix of all observation points

[0046] S3. Hessian matrix and gradient;

[0047] H = J T J, g = J T r

[0048] S4. Update amount Δθ;

[0049]

[0050] Where: Δθ includes ΔT i and ΔX j

[0051] S5. Parameter update;

[0052] After solving for Δθ, the new parameters are obtained through the update rule:

[0053] T i,new = T i,old + ΔT i

[0054] X j,new = X j,old + ΔX j .

[0055] The advantages of the present invention are:

[0056] Compared with the prior art, the significant advantage of the present invention is that it improves the algorithm for UAV positioning, reduces the positioning error, and thus can quickly and accurately respond to control commands. Moreover, the UAV 3D printing system usually uses a lightweight UAV platform and realizes the printing function through a simple print head and nozzle design. The combination of this low - cost hardware and efficient software control greatly reduces the construction cost of the system. Description of the Drawings

[0057] Figure 1 It is a schematic flow chart of the UAV navigation 3D printing method based on indoor scene positioning of the present invention;

[0058] Figure 2 It is a step flow chart of UAV positioning in the present invention;

[0059] Figure 3 It is a software interface diagram of the UAV being real-time recognized by the camera in the present invention;

[0060] Figure 4 It is a schematic diagram of the UAV shape in the present invention;

[0061] Figure 5 It is a trajectory diagram of UAV positioning in the present invention. Specific implementation manner

[0062] Suppose there is a scene photographed by multiple cameras, and the goal is to optimize the positions of three-dimensional points and the poses of the cameras based on the observations of these cameras, and to print a cylinder with a height H of 2 meters and a radius of 0.5 meters. For the flow chart, please refer to Figure 1 As shown, for the UAV schematic diagram, please refer to Figure 4 As shown.

[0063] The most common RGB camera is used, such as the PS3 Eye camera. On this basis, we remove the infrared filter in the camera and add a red transparent plastic sheet, and then place 3 red LED lights on the UAV in a triangular pattern. Through the feature point detection method, a set of two-dimensional feature points will be projected on the images taken by each camera.

[0064] Now, task definition and geometric modeling are carried out for the UAV. First, the task objective needs to be defined, which is to print a cylinder. The geometric model of the cylinder includes the height H and the radius R. Based on this, the path planning system divides the task into multiple levels and designs paths based on the geometric shape of each level. The height of the cylinder is 2 meters and the radius is 0.5 meters. The printing will be divided into 400 layers, and the height of each layer is 5 millimeters.

[0065] Path planning mainly involves the flight trajectory of the UAV and the movement trajectory of the printing nozzle. To ensure printing accuracy and stability, we adopt circular trajectory planning and print layer by layer. During the printing process of each layer, the nozzle deposits materials along the predetermined circular trajectory to ensure uniform printing of each layer

[0066] Inter-layer height adjustment: After each layer of printing is completed, the UAV needs to fly up a certain height to complete the printing task of the next layer. Suppose the height of each layer is 5mm, and the flight path passes through InfraredThe positioning system of the camera tracks the altitude change of the drone in real time, ensuring that the flight altitude is accurately docked with the altitude of each printing layer, and avoiding printing misalignment caused by deviations during flight.

[0067] Nozzle trajectory design: The printing path of each layer is designed in advance by the host computer system as a circular path. The nozzle needs to deposit along this circular trajectory to ensure a uniform printing layer. The trajectory of the nozzle is determined by the target radius R and the printing speed. The movement speed of the nozzle on the circular trajectory is kept consistent to ensure the deposition quality of each layer. On the basis of this path design, we combine positioning algorithms such as triangulation and bundle adjustment to achieve precise adjustment of the real-time position of the drone and the nozzle trajectory. Specifically, triangulation and bundle adjustment can effectively correct the flight path of the drone and the nozzle position, ensuring that the nozzle accurately deposits materials along the circular trajectory.

[0068] The specific path design is as follows:

[0069]

[0070] Among them, is the radius of the current layer, θ is the angle, varying from 0 to 2π;

[0071] During the printing process, due to possible external interferences (such as air flow, drone control precision error, etc.), it is necessary to optimize the path in real time. At this time, assume that the target object (nozzle) is on the circular trajectory of the i-th layer with a radius of R i , then the theoretical three-dimensional coordinates M(x, y, z) of the target object can be expressed as:

[0072]

[0073]

[0074] z = h i

[0075] Then, the triangulation method is used to restore the three-dimensional position of the target object. There are two cameras C1 and C2, which respectively observe the same target object. Each camera captures the two-dimensional coordinates p1 = (u1, v1) and p2(u2, v2) of the target object, as well as the internal and external parameter information of the camera (for example, focal length, rotation matrix, translation matrix, etc.). Our goal is to calculate the position X = (X, Y, Z) of the target object in the three-dimensional space through the projection coordinates of these two-dimensional points.

[0076] (1) Camera projection model

[0077] The projection model of the camera can be represented by the projection matrix. Assume that the internal parameter matrix of the camera is K, and the external parameter matrix is [R i |ti , the projection process of the camera can be expressed as:

[0078] p i = K[R i t i X

[0079] where:

[0080] ● p i = (u i , v i , 1) is the two-dimensional coordinate captured by the i-th camera

[0081] ● X = (X, Y, Z, 1) is the three-dimensional coordinate of the target object

[0082] ● K is the internal parameter matrix of the camera (including information such as focal length and principal point)

[0083] ● [R i | t i is the external parameter of the i-th camera, representing the position and attitude of the camera. R i is the rotation matrix, representing the direction of the camera, and t i is the translation matrix, representing the position of the camera.

[0084] (1) Problem analysis

[0085] By obtaining two-dimensional coordinates from multiple perspectives, we can use the least squares method to solve the three-dimensional coordinates of the target object.

[0086] At this time, we have two-dimensional projections from two perspectives, p1 = (u1, v1) corresponding to camera C1, p2 = (u2, v2) corresponding to camera C2, and the internal parameter matrix of the camera

[0087] where:

[0088] ● f x and f y are the focal lengths of the camera in the horizontal and vertical directions

[0089] ● c x and c y are the principal point coordinates of the camera image (usually the image center)

[0090] (2) Fundamental matrix F

[0091] The fundamental matrix F is a 3×3 matrix that describes the geometric relationship between two cameras. It is calculated through the internal and external parameters of the cameras. The relationship formula of the fundamental matrix is:

[0092]

[0093] where:

[0094] ● p1 and p2 are the two-dimensional coordinates of corresponding points in two perspectives.

[0095] ● F is the fundamental matrix, which contains the relative position and attitude information of the cameras (the rotation matrix R and translation vector t of the cameras)

[0096] (3) Epipolar constraint

[0097] Through the fundamental matrix F, we can obtain the epipolar line of the target point in the second perspective. Given the point p1 in the first camera, the epipolar line l2 in the second perspective can be calculated by the following formula:

[0098] l2 = Fp1

[0099] Using this epipolar line, you can determine the possible positions of the target point in the second perspective. Epipolar geometry provides us with a matching constraint between two images, helping us reduce the search space for matching.

[0100] (4) Calculate depth Z

[0101] The parallax d is the offset of the object between the two cameras. Calculating the parallax is one of the key steps in triangulation. Assuming that the horizontal parallax between camera C1 and camera C2 is d = u1 - u2, then the depth Z can be calculated by the following formula:

[0102]

[0103] Where:

[0104] ● f is the focal length of the camera (which can be obtained from the internal parameter matrix K of the camera)

[0105] ● b is the baseline distance between the cameras (the horizontal distance between the two cameras)

[0106] ● d = u1 - u2 is the parallax, that is, the difference in the abscissa of the target object in the two images.

[0107] (5) Recover the three-dimensional coordinates of the target object

[0108] According to the projection model of the camera, the three-dimensional coordinates X = (X, Y, Z) of the target object can be recovered from the two-dimensional coordinates (u, v) and the depth Z by the following formula:

[0109]

[0110] Z = Z

[0111] Where:

[0112] ● u and v are the two-dimensional coordinates of the target object in the image;

[0113] ●c x and c y are the principal point coordinates of the camera;

[0114] ●f x and f y are the focal lengths of the camera in the horizontal and vertical directions;

[0115] Thus, the coordinates X of the UAV are initially calculated

[0116] Next, the goal is to optimize the pose of the camera and the positions of each feature point in 3D space to minimize the error between these projected points and the actually observed feature points. For the optimization results, please refer to Figure 3 as shown.

[0117] Optimization goal: It is hoped to obtain the poses T of all cameras through optimization i =(R i , t i ) and the positions X of all 3D points j , so as to minimize the error between the projection results of all cameras and the actually observed image feature points. That is, to minimize the following objective function (cost function):

[0118]

[0119] The meaning of the objective function is to minimize the distance between the feature points observed by each camera and the projected points calculated based on the currently estimated camera parameters and 3D point positions.

[0120] N cams represents the number of cameras; N points represents the number of 3D points; p ij represents the actual observation of the j-th feature point captured by the i-th camera; represents the projected point calculated according to the current camera pose T i and the 3D point position X j ; ρ(·) is a robust kernel function used to reduce the influence of outliers (it can be assumed that ρ(x)=x) first.

[0121] Error function: Define the residual of each observation point as:

[0122]

[0123] The goal is to minimize the following error function (cost function):

[0124]

[0125] Gauss-Newton method iterative optimization

[0126] (1) Linearize the residual function

[0127] Use Taylor expansion for the error function r ij At the current parameter {T i , X j} for linearization:

[0128]

[0129] Where:

[0130] ● J ij is the Jacobian matrix of the residual function with respect to the optimization parameters, divided into two parts:

[0131] Jacobian with respect to the camera pose

[0132] Jacobian with respect to the 3D point position

[0133] (2) Construct the linearized objective function

[0134] After linearization, the objective function of the sum of squared errors becomes:

[0135]

[0136] After expansion, it can be written as:

[0137]

[0138] Where:

[0139] ● J is the global Jacobian matrix of all observation points

[0140] (3) Hessian matrix and gradient

[0141] H = J T J, g = J T r

[0142] (4) Update amount Δθ

[0143] HΔθ = -g

[0144] Where: Δθ includes ΔT i and ΔX j

[0145] (5) Parameter update

[0146] After solving for Δθ, the new parameters are obtained through the update rule:[[ID=?]] [[ID=?]]

[0147] T[[ID=?]] i,new = T[[ID=?]] i,old + ΔT[[ID=?]] i

[0148] X It seems there are some tags in the English translation that might be incorrect or need further clarification in the original text. Please double-check the provided text for accuracy. If you have any specific questions or need further assistance, feel free to ask.j,new = X j,old + ΔX j

[0149] By continuously repeating steps (1) to (5), the complexity of the algorithm is reduced using epipolar geometry, and the joint optimization of the camera pose and the 3D point position is achieved using bundle adjustment, gradually reducing the projection error and improving the accuracy of UAV navigation and positioning.

[0150] Communication is carried out through the esp32 chip, and the orientation of the UAV is continuously adjusted using a double-layer nested PID. The positioning process is shown in Figure 2 as shown.

[0151] By minimizing this error function, we not only optimize the position of the target object (the nozzle), but also ensure that the UAV and the nozzle can perform accurate printing along a predetermined circular trajectory. The specific trajectory is shown in Figure 5 as shown.

[0152] The above description is only a preferred embodiment of the present invention and does not impose any other form of limitation on the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A 3D printing method for drone navigation based on indoor scene positioning, characterized in that: Including the following steps: (1) Task definition and geometric modeling: Before the task starts, first define the printing target and establish its geometric model. According to the geometric model of the task target, the path planning system divides the target task into multiple levels, designs the printing path of each level layer by layer, sets the height of each layer to h, and realizes the layered printing of the task by modeling the geometric shape of each layer; (2) Path planning and printing trajectory design: According to the task definition and geometric model, the system generates the printing path and the flight path. The printing path is a circular trajectory, and the printing trajectory of each layer is set according to the target radius and the speed of the nozzle. The nozzle moves uniformly along the circular trajectory to ensure the uniformity of material deposition; (3) Flight control and real-time path optimization: During the printing process, due to external interference, the system needs to optimize the flight path and the nozzle trajectory in real time. Through real-time feedback and position tracking, the system adjusts the flight speed and the nozzle angle in real time according to the error between the actual position of the UAV and the target trajectory, ensuring that the nozzle moves precisely along the designed trajectory; In the step (3), (1) Constraint conditions: During the printing process, due to external interference, real-time optimization of the path is required. At this time, it is assumed that the drone is on a circular trajectory at the th layer, with a radius of . Then the three-dimensional coordinates of the target object are expressed as: ； ； ； (2) Optimization objective: The poses of all cameras are obtained through optimization and the positions of all 3D points , minimizing the error between the projection results of all cameras and the image feature points actually observed, that is, minimizing the following objective function: ； The meaning of the objective function is to minimize the distance between the feature points observed by each camera and the projection points calculated by the current estimated camera parameters and the three-dimensional point positions; Among them represents the number of cameras; represents the number of 3D points; represents the th camera's actual observation of the th feature point; represents the projected point calculated based on the current camera pose and the 3D point position ; is a robust kernel function used to reduce the influence of outliers; (3) Error function: Define the residual of each observation point as: ； (4) The goal is to minimize the following error function: ； (5) Gauss-Newton method iterative optimization; S1. Linearize the residual function; Use Taylor expansion to linearize the error function around the current parameter as follows: ； Where: is the Jacobian matrix of the residual function with respect to the optimization parameters and is divided into two parts: : Jacobian of the camera pose; : Jacobian of the three-dimensional point position; S2. Construct the linearized objective function; After linearization, the objective function of the sum of squared errors becomes: ； Expanded and written as: ； Where: is the global Jacobian matrix of all the observation points S3. Hessian matrix and gradient; , ； S4. Update amount ; ； Wherein: including and ; S5. Parameter update; Solve After that, new parameters are obtained through the update rule: ； 。 2. The method for 3D printing of drone navigation based on indoor scene positioning according to claim 1, wherein: In the step (1), the task definition and geometric modeling of 3D printing include: realizing the layered printing of the task by modeling the geometric shape of each layer.

3. A 3D printing method for drone navigation based on indoor scene positioning according to claim 1, characterized in that: In the step (2), the path planning and printing trajectory design include the nozzle trajectory design. The nozzle deposits materials along the path trajectory to ensure the uniform distribution of materials in each layer; the system accurately tracks the position of the UAV through the infrared camera positioning system to ensure the accurate docking of the printing positions of each layer.

Images