An automated multi-drone cooperative device and method based on an active light source array

Abstract

This invention proposes an automatic multi-drone coordination device and method based on an active light source array. In the coordination method, the drones are divided into a host (1) and one or more slaves (2). Both the host and slaves are equipped with an active light source array (12), and the slaves are equipped with a coordination control device. The active light source array includes multiple active light sources, which are used to emit light signals representing drone attitude information data and light signals for drone positioning. The slaves read the host attitude information and positioning information in the light signals of the active light source array through the coordination control device, and control the drone flight through a tracking algorithm to achieve automatic multi-drone coordination. This invention has a relatively good coordination effect in both day and night and does not rely on radio communication.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to an automatic multi-UAV collaborative device and method based on an active light source array. Background Technology

[0002] With the rapid advancement of technology, drone technology has developed at a breakneck pace, its application areas are constantly expanding, and its application scenarios are becoming increasingly diverse. Drones can be seen everywhere, whether indoors, in forests, or day and night. Especially in multi-drone collaborative applications, drones have demonstrated unprecedented potential and value.

[0003] Multi-drone collaboration refers to multiple drones working together to complete a task or objective. This collaboration can greatly improve the operational efficiency, accuracy, and safety of drones. Currently, there are two main methods for multi-drone collaboration.

[0004] One approach involves establishing communication between multiple drones, using wireless communication modules to transmit information. The drones exchange information to achieve coordinated operations such as formation flying and tracking. The advantage of this method is its high communication speed and ability to achieve real-time coordination. However, wireless communication can be affected by interference in complex environments; obstacles such as trees and tall buildings can hinder signal transmission, leading to poor coordination.

[0005] Another approach is to achieve multi-machine collaboration through visual recognition technology. In this method, the slave devices use sensors such as cameras to identify the current posture or features of the master device, then calculate the master device's posture information, and predict its trajectory to achieve multi-machine tracking. The advantage of this method is that it does not rely on wireless communication and can achieve collaboration in environments where wireless communication is limited. However, visual recognition technology performs poorly in low-light or nighttime scenes, and the posture data it generates is less accurate than data from gyroscopes. Summary of the Invention

[0006] This invention proposes an automatic multi-drone coordination device and method based on an active light source array, which has good coordination effect in both day and night and does not rely on radio communication.

[0007] The present invention adopts the following technical solution.

[0008] An automatic multi-drone collaboration method based on an active light source array is disclosed. In the collaboration method, the drones are divided into a host (1) and one or more slaves (2). Both the host and the slaves are equipped with an active light source array (12), and the slaves are equipped with a collaborative control device. The active light source array includes multiple active light sources, which are used to emit light signals representing drone attitude information data and light signals for drone positioning. The slaves read the host attitude information and positioning information in the light signals of the active light source array through the collaborative control device, and control the drone flight through a tracking algorithm to achieve automatic multi-drone collaboration.

[0009] An automatic multi-drone collaboration device based on an active light source array is provided for implementing the aforementioned automatic multi-drone collaboration method based on an active light source array. The active light source array includes a white light active light source emitting white light as a drone positioning feature, and three colored light active light sources emitting colored light. The light signal of each active light source is a light signal synthesized from red, green, and blue lasers and processed by a light shield. During multi-drone automatic collaboration, the colored light active light sources are connected to the drone's drone attitude sensing module to obtain the drone's attitude information through a gyroscope. Based on the different attitude information of the drone's current flight state, the rotation angle is synthesized to synthesize light signals of different colors, providing the slave drones with target identification and the master drone's flight parameters.

[0010] The collaborative control device includes a visual sensing module (23) and a solution control module (24); the visual sensing module acquires image information through a depth camera with an infrared ranging module, acquires depth information in the image through the infrared ranging module, and transmits the image information and depth information to the solution control module.

[0011] The calculation and control module is an onboard computer at the UAV. The onboard computer identifies the position of the UAV in the image by receiving image information and depth information, further identifies the position of the active light source on the UAV, and converts the light source color into the attitude information data of the UAV through the recognition algorithm.

[0012] When the recognition algorithm is working, it first identifies the active light source array of the UAV, then identifies the position of the white active light source in the array. After identifying the white active light source, based on its position among the four active light sources, it marks three colored active light sources representing yaw angle data, roll angle data, and pitch angle data. Finally, it extracts the R, G, and B values ​​of the colors of the three colored active light sources representing yaw, roll, and pitch, and calculates the corresponding rotation angles from the R, G, and B values.

[0013] The onboard computer stores an algorithm for controlling the UAV slave to track based on attitude and depth information. This algorithm calculates the required movement speed and angle for the UAV to track the host machine ahead using rotation angle and depth information, and controls the UAV to perform the corresponding flight.

[0014] The active light source array device includes four active light source combining devices and a power supply control device with the same structure; each active light source includes an active light source combining device (121) and a power supply control device (122);

[0015] The active light source synthesis device includes a base (1210) and a colored light emitting device disposed on the base; a laser synthesis module and a laser adjustment module are sequentially arranged at the output optical path of the colored light emitting device;

[0016] The laser combining module is located above the light output end of the light-emitting device and includes a beam combining prism (1214) and a first plano-convex mirror (1215) arranged in sequence, with the plane of the first plano-convex mirror facing the beam combining prism.

[0017] The colored light-emitting device includes a red light-emitting device (1211), a blue light-emitting device (1213), and a green light-emitting device (1212) located on three sides of the light-combining prism respectively; the three-color lasers emitted by the red light-emitting device, the blue light-emitting device, and the green light-emitting device are combined into a single laser beam by the laser combining module and emitted from the light output end of the laser combining module;

[0018] The optical input end of the laser adjustment module is connected to the optical output end of the laser combining module; the laser adjustment module includes a second plano-convex mirror (1216), a reflector (1217), a third plano-convex mirror (1218), and a light shield (1219) arranged sequentially along the optical path; the convex surface of the second plano-convex mirror faces the combining prism to adjust the combined laser to a fixed size; the reflector is located behind the second plano-convex mirror and is placed at 45° to reflect the combined laser upward; the third plano-convex mirror and the light shield are located above the reflector, and the laser beam intercepted and converged by the third plano-convex mirror forms a recognizable light source at the light shield;

[0019] The power control module of the power supply control device includes a first controller (1221), a second controller (1222), a third controller (1223), and a fourth controller (1224);

[0020] The UAV attitude sensing module (1225) is connected to the main control module (1226) via a cable;

[0021] The light-combining prism is located in the groove in the middle of the base;

[0022] The UAV attitude sensing module acquires the UAV's attitude data through a gyroscope;

[0023] The red, blue, and green light-emitting devices are all laser diode light-emitting devices. Each laser diode light-emitting device includes a laser diode driver board (12111), a laser diode (12112), and a condenser lens (12113). When the laser diode light-emitting device is working, the laser diode driver board drives the laser diode to emit photons, and then the condenser lens focuses the photons into a beam of laser light.

[0024] Each of the three colored light-emitting devices incorporates a laser diode. By exciting different semiconductors, each laser diode emits red, green, and blue photons respectively. The light signal synthesized by the lasers of each colored light-emitting device in the active light source array changes color with the rotation angles of the represented yaw, roll, and pitch, as detailed below:

[0025] C = [r(x)*255, g(x)*255, b(x)*255] (Formula 1)

[0026] In the formula, C represents the composite laser color. The meaning of Formula 1 is that the mixed spectral color is visually formed by mixing and adding the three basic colors of red, green, and blue in their respective proportions r(x), g(x), and b(x). The relationship between r(x), g(x), b(x) and the rotation angle is as follows:

[0027]

[0028] Where x is the rotation angle of the yaw, pitch, or roll to be represented, and r(x), g(x), and b(x) are the calculated color coefficients. The main control module outputs the corresponding PWM signal value to the power control module based on the calculated color coefficients r(x), g(x), and b(x).

[0029] The method for automatic multi-drone collaboration includes the following steps;

[0030] Step S1: The UAV attitude sensing module acquires the UAV attitude information and sends the acquired UAV attitude information to the main control module;

[0031] Step S2: The UAV attitude information includes the three-axis rotation angles of Yaw, Roll, and Pitch. The main control module converts the rotation angles into color coefficients r(x), g(x), and b(x) of the color to be emitted according to Formulas 2, 3, and 4. The required power is calculated through the color coefficients, and the corresponding PWM signal value is then sent to each power control module. The PWM signal value includes the percentage of output power corresponding to the maximum control power of the power control module.

[0032] Step S3: Each power control module provides power corresponding to the PWM value to each corresponding colored light emitting device according to the PWM value output by the main control module, thereby controlling the emission power of the three corresponding light emitting devices; Step S4: The three colored light emitting devices of the active light source synthesis device emit red, blue, and green lasers that conform to the calculated color coefficients. That is, the three lasers obtained according to Formula 1 are synthesized by the laser synthesis module of the active light source synthesis device into a composite laser that is output along the same straight line and whose color can represent the corresponding rotation angle value;

[0033] Step S5: The active light source synthesis device first processes the composite laser with the laser adjustment module to form a laser with a constant size of laser points at near and far distances. Then, it passes through a reflector and a plano-convex mirror in sequence to reach the light shield. After being processed by the light shield, it forms a light source signal that is easy to be observed and recorded by external electronic devices or humans and is then emitted.

[0034] Step S6: The four active light source synthesis devices repeat the above process to form four active light source signals with identification characteristics on the UAV. One of them is a white light source that emits white light. This light source is used as the positioning feature of the UAV. The other three are colored light sources. The color of the light source represents the rotation data values ​​of the UAV's Yaw, Roll, and Pitch, respectively, so as to realize the transmission of UAV attitude information based on the active light source array.

[0035] When the solution control module identifies the active light source array of the UAV in the image, the specific method is as follows: It receives image and depth information from the visual sensing module, where the depth information value is represented by D. The solution control module's solution control algorithm identifies circles using the Hough transform, with the following formula:

[0036] (x i -a) 2 +(y i -b) 2 =r 2

[0037] By defining a Hough space and constructing a point in the space using the three parameters (a, b, r), in the formula, x... i and y i Let a and b be points in the image, a and b be the coordinates of the center of the circle, and r be the radius of the circle. Iterate through each point in the image (x, b, r). i y i For each point, calculate its distance r to the corresponding possible center (a, b), and increment the count S of point (a, b, r) in the Hough space. a，b，r S a，b，r The larger S is, the higher the certainty that a, b, and r can represent a circle. a，b，r When it is greater than a certain value, that is:

[0038] Sa，b，r ≥S 阈值

[0039] a, b, and r represent a circle centered at (a, b) with radius r, and S 阈值 It is a set value; different values ​​are set to adjust the accuracy of circle recognition.

[0040] By identifying the number of circles within a certain range, the location of the active light source array in the image can be determined. Then, by weighting the R, G, and B values ​​of each point within the active light source region, with higher weights closer to the center, the R, G, and B values ​​of the light source color can be calculated, thereby identifying the color of each active light source.

[0041] After identifying white, two lines are extended from the white active light source as the origin to adjacent light sources. If one line can coincide with the other line when rotated 90° clockwise, then this line is used as the positive Y-axis and the other as the positive X-axis to establish a light source array coordinate system. The light source representing Yaw is on the positive X-axis, and the light source representing Pitch is on the positive Y-axis. The positions of the light sources representing Yaw, Roll, and Pitch are then identified, and the R, G, and B values ​​of the three light source colors are calculated using the following formula:

[0042]

[0043] Obtain the color coefficients r(x), g(x), and b(x), and then use the following formula:

[0044]

[0045] Calculate the rotation angle x of Yaw, Roll, or Pitch. Note that x is expressed as a rotation angle only in Formula 5.

[0046] Repeat formula five to calculate the rotation angles of the host's Yaw, Roll, and Pitch in sequence, with Yaw, Roll, and Pitch representing the rotation angle values ​​of the above three respectively.

[0047] Furthermore, by identifying the position of the active light source array of the drone in front and calculating the pixel difference (Pix) between it and the image center, the algorithm can be further optimized. x差 And Pix y差 The calculated Yaw, Roll, and Pitch are obtained using the following formula:

[0048]

[0049] The speed of rotation of the slave device along the X, Y, Z axes and the yaw angle when it tracks the master device is calculated, and the slave device is controlled to track the master device in this way.

[0050] In Formulas 6 and 7, `max` and `min` are used to limit the maximum speed of the drone to 6 m / s. The negative sign does not represent magnitude, but rather the direction of flight.

[0051] When the drone's nose is oriented towards the positive X-axis, its forward speed is positive and its backward speed is negative. If the drone's nose is oriented towards the Y-axis, its rightward speed is positive and its leftward speed is negative. If the drone's nose is oriented towards the Z-axis, its upward speed is positive and its backward speed is negative.

[0052] In Formulas 6 and 7, ΔYaw represents the change in the Yaw angle in degrees over a certain period of time; the Y-axis is the X-axis in the image, the Z-axis is the Y-axis in the image, and the Y-axis in the image increases from top to bottom.

[0053] The method for automatic multi-drone collaboration also includes the following steps;

[0054] Step S7: The slave device acquires the image and depth information of the host through the depth camera and sends it to the calculation control module. The R, G, and B values ​​of different light source colors of the host active light source array are obtained through the image.

[0055] Step S8: Calculate the rotation data values ​​of Yaw, Roll, and Pitch represented by different light sources using Formula 5. Then, using the rotation data values ​​and depth information, calculate the speed control values ​​for the X, Y, and Z axes and the Yaw angle of the slave drone to track the master drone using Formulas 6, 7, 8, and 9. This controls the slave drone to track the master drone. For multiple slave drones, repeat the above steps to emit an active light source array and track the preceding slave drone or master drone, achieving automatic multi-drone collaboration based on the active light source array.

[0056] When performing multi-drone automatic collaborative operations, both the host and slave drones are equipped with active light source arrays, while the collaborative control device is only installed on the slave drones.

[0057] This invention provides an automatic multi-drone coordination device and method based on an active light source array. The device consists of an active light source array and an automatic coordination mechanism. The active light source array, mounted on the drone's main unit, comprises four active light sources. Each active light source is synthesized from three red, green, and blue lasers and is shielded with a light-shielding cover to ensure better observation by the visual acquisition device. One light source emits fixed white light as a positioning feature. The other three active light sources acquire the drone's attitude information via a gyroscope, rotating at different angles based on the attitude information to synthesize lasers of different colors, thereby providing the slave drones with clearer target identification and more accurate flight parameters. The active light sources provide a clear display effect both day and night, are independent of communication, and the slave drones are equipped with visual acquisition devices. By identifying the colors of the active light sources at different positions on the main unit's light source array, the slave drones' attitude information can be obtained and controlled through a tracking algorithm to achieve automatic multi-drone coordination in multiple scenarios.

[0058] Compared with the prior art, the present invention has the following advantages:

[0059] 1. This invention solves the problem that laser light sources are difficult for electronic devices to recognize by processing the synthesized laser through a light shield.

[0060] 2. This invention uses an active light source array to represent information about different attitudes of the UAV, which is beneficial for use in various scenarios, including but not limited to nighttime, daytime, no GPS, magnetic field interference and other complex scenarios. This enables the slave device to more effectively obtain host information and track the host in multiple scenarios.

[0061] 3. This invention uses attitude information fused with depth information and image information to control speed, enabling the slave device to track the master device relatively quickly and accurately. Attached Figure Description

[0062] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0063] Appendix Figure 1 This is a three-dimensional structural schematic diagram of the device of the present invention;

[0064] Appendix Figure 2 This is a three-dimensional structural diagram of the host device of the present invention;

[0065] Appendix Figure 3 This is a three-dimensional structural schematic diagram of the slave device 1 of the present invention;

[0066] Appendix Figure 4 This is a schematic diagram of the internal structure of the host device of the present invention;

[0067] Appendix Figure 5This is a three-dimensional structural diagram of an active light source synthesis device according to an embodiment of the present invention;

[0068] Appendix Figure 6 This is a three-dimensional structural diagram of the power supply control device in one embodiment of the present invention;

[0069] Appendix Figure 7 This is a three-dimensional structural schematic diagram of a red light-emitting device in one embodiment of the present invention;

[0070] Appendix Figure 8 This is a schematic diagram of the coordinate system of the light source array in one embodiment of the present invention;

[0071] Appendix Figure 9 This is a schematic diagram of the optical path of the laser synthesis module and the laser adjustment module in one embodiment of the present invention;

[0072] Appendix Figure 10 This is a schematic diagram of the control flow of a multi-drone automatic collaborative method based on an active light source array in one embodiment of the present invention;

[0073] In the diagram: 1. Master unit, 2. Slave unit, 3. Another slave unit, 11. Master unit UAV body, 12. Active light source array, 21. Slave unit UAV body, 22. Calculation control module, 23. Visual sensor module, 24. Slave unit active light source array device, 121. Active light source synthesis device, 122. Power supply control device, 1210. Base, 1211. Red light-emitting device, 1212. Green light-emitting device, 1213. Blue light-emitting device, 1214. Light-combining prism, 1215. Plano-convex mirror, 1216. Plano-convex mirror, 1217. Reflector, 1218. Plano-convex mirror, 1219. Sunshade, 1221. Three-way controller, 1222. Three-way controller, 1223. Three-way controller, 1224. Three-way controller, 1225. UAV attitude sensing module, 1226. Main control module, 12111. Laser diode driver board, 12112. Laser diode, 12113. Condenser lens. Detailed Implementation

[0074] As shown in the figure, a multi-drone automatic collaborative method based on an active light source array is presented. In this collaborative method, the drones are divided into a master drone 1, a slave drone 2, and another slave drone 3, with more than one slave drone. Both the master drone and the slave drones are equipped with an active light source array 12, and the slave drones are equipped with a collaborative control device. The active light source array includes multiple active light sources, which are used to emit light signals representing drone attitude information data and light signals for drone positioning. The slave drones read the master drone attitude information and positioning information from the light signals of the active light source array through the collaborative control device, and control the drone flight through a tracking algorithm to achieve multi-drone automatic collaborative operation.

[0075] An automatic multi-drone collaboration device based on an active light source array is provided for implementing the aforementioned automatic multi-drone collaboration method based on an active light source array. The active light source array includes a white light active light source emitting white light as a drone positioning feature, and three colored light active light sources emitting colored light. The light signal of each active light source is a light signal synthesized from red, green, and blue lasers and processed by a light shield. During multi-drone automatic collaboration, the colored light active light sources are connected to the drone's drone attitude sensing module to obtain the drone's attitude information through a gyroscope. Based on the different attitude information of the drone's current flight state, the rotation angle is synthesized to synthesize light signals of different colors, providing the slave drones with target identification and the master drone's flight parameters.

[0076] The collaborative control device includes a visual sensing module 23 and a solution control module 24. The visual sensing module acquires image information through a depth camera with an infrared ranging module, acquires depth information in the image through the infrared ranging module, and transmits the image information and depth information to the solution control module.

[0077] The calculation and control module is an onboard computer at the UAV. The onboard computer identifies the position of the UAV in the image by receiving image information and depth information, further identifies the position of the active light source on the UAV, and converts the light source color into the attitude information data of the UAV through the recognition algorithm.

[0078] When the recognition algorithm is working, it first identifies the active light source array of the UAV, then identifies the position of the white active light source in the array. After identifying the white active light source, based on its position among the four active light sources, it marks three colored active light sources representing yaw angle data, roll angle data, and pitch angle data. Finally, it extracts the R, G, and B values ​​of the colors of the three colored active light sources representing yaw, roll, and pitch, and calculates the corresponding rotation angles from the R, G, and B values.

[0079] The onboard computer stores an algorithm for controlling the UAV slave to track based on attitude and depth information. This algorithm calculates the required movement speed and angle for the UAV to track the host machine ahead using rotation angle and depth information, and controls the UAV to perform the corresponding flight.

[0080] The active light source array device includes four active light source combining devices and a power supply control device with the same structure; each active light source includes an active light source combining device 121 and a power supply control device 122.

[0081] The active light source synthesis device includes a base 1210 and a colored light emitting device disposed on the base; a laser synthesis module and a laser adjustment module are sequentially arranged at the output optical path of the colored light emitting device.

[0082] The laser combining module is located above the light output end of the light-emitting device and includes a beam combining prism 1214 and a first plano-convex mirror 1215 arranged in sequence, with the plane of the first plano-convex mirror facing the beam combining prism.

[0083] The colored light-emitting device includes a red light-emitting device 1211, a blue light-emitting device 1213, and a green light-emitting device 1212 located on the three sides of the light-combining prism, respectively; the three-color lasers emitted by the red light-emitting device, the blue light-emitting device, and the green light-emitting device are combined into a single laser beam by the laser combining module and emitted from the light output end of the laser combining module.

[0084] The optical input end of the laser adjustment module is connected to the optical output end of the laser combining module; the laser adjustment module includes a second plano-convex mirror 1216, a reflector 1217, a third plano-convex mirror 1218, and a light shield 1219 arranged sequentially along the optical path; the convex surface of the second plano-convex mirror faces the combining prism to adjust the combined laser to a fixed size; the reflector is located behind the second plano-convex mirror and is placed at 45° to reflect the combined laser upward; the third plano-convex mirror and the light shield are located above the reflector, and the laser beam intercepted and converged by the third plano-convex mirror forms a recognizable light source at the light shield;

[0085] The power control module of the power supply control device includes a first controller 1221, a second controller 1222, a third controller 1223, and a fourth controller 1224;

[0086] In this example, different controllers are used to control the power of the light-emitting devices in the active light sources at different locations in the active light source array.

[0087] The UAV attitude sensing module 1225 is connected to the main control module 1226 via a cable;

[0088] The light-combining prism is located in the groove in the middle of the base;

[0089] The UAV attitude sensing module acquires the UAV's attitude data through a gyroscope;

[0090] The red, blue, and green light-emitting devices are all laser diode light-emitting devices. Each laser diode light-emitting device includes a laser diode driver board 12111, a laser diode 12112, and a condenser lens 12113. When the laser diode light-emitting device is working, the laser diode driver board drives the laser diode to emit photons, and then the condenser lens focuses the photons into a beam of laser light.

[0091] Each of the three colored light-emitting devices incorporates a laser diode. By exciting different semiconductors, each laser diode emits red, green, and blue photons respectively. The light signal synthesized by the lasers of each colored light-emitting device in the active light source array changes color with the rotation angles of the represented yaw, roll, and pitch, as detailed below:

[0092] C = [r(x)*255, g(x)*255, b(x)*255] (Formula 1)

[0093] In the formula, C represents the composite laser color. The meaning of Formula 1 is that the resulting spectral color is visually formed by mixing and adding the three basic colors—red, green, and blue—in their respective proportions r(x), g(x), and b(x). In this example, if r(x), g(x), and b(x) are 0.4, 0.6, and 0 respectively, then the composite laser consists of red and green lasers; the blue laser does not produce any light and is in an off state. Since r(x) and g(x) are 0.4 and 0.6 respectively, green has higher power and brightness than red. Since red and green are mixed to produce yellow, the resulting composite laser appears yellowish-green. The relationship between r(x), g(x), b(x) and the rotation angle is as follows:

[0094]

[0095]

[0096] Where x is the rotation angle of the yaw, pitch, or roll to be represented, and r(x), g(x), and b(x) are the calculated color coefficients. The main control module outputs the corresponding PWM signal value to the power control module based on the calculated color coefficients r(x), g(x), and b(x).

[0097] The method for automatic multi-drone collaboration includes the following steps;

[0098] Step S1: The UAV attitude sensing module acquires the UAV attitude information and sends the acquired UAV attitude information to the main control module;

[0099] Step S2: The UAV attitude information includes the three-axis rotation angles of Yaw, Roll, and Pitch. The main control module converts the rotation angles into color coefficients r(x), g(x), and b(x) of the color to be emitted according to Formulas 2, 3, and 4. The required power is calculated through the color coefficients, and the corresponding PWM signal value is then sent to each power control module. The PWM signal value includes the percentage of output power corresponding to the maximum control power of the power control module.

[0100] In this example, if the color coefficients r(x), g(x), and b(x) are 0.5, 0.5, and 0, and the full-value power (maximum control power of the power control module) is 100W, then the power to be output corresponds to 0.5*100W, 0.5*100W, and 0*100W for red, green, and blue light-emitting devices, which is 50W, 50W, and 0W.

[0101] If the calculated required power is 50W, and assuming the power control module's upper limit is 100W, then That is, a PWM signal output of 50% allows the power control module to provide 50W of power to the controlled device;

[0102] Step S3: Each power control module provides power corresponding to the PWM value to each corresponding colored light emitting device according to the PWM value output by the main control module, thereby controlling the emission power of the three corresponding light emitting devices; Step S4: The three colored light emitting devices of the active light source synthesis device emit red, blue, and green lasers that conform to the calculated color coefficients. That is, the three lasers obtained according to Formula 1 are synthesized by the laser synthesis module of the active light source synthesis device into a composite laser that is output along the same straight line and whose color can represent the corresponding rotation angle value;

[0103] Step S5: The active light source synthesis device first processes the composite laser with the laser adjustment module to form a laser with a constant size of laser points at near and far distances. Then, it passes through a reflector and a plano-convex mirror in sequence to reach the light shield. After being processed by the light shield, it forms a light source signal that is easy to be observed and recorded by external electronic devices or humans and is then emitted.

[0104] Step S6: The four active light source synthesis devices repeat the above process to form four active light source signals with identification characteristics on the UAV. One of them is a white light source that emits white light. This light source is used as the positioning feature of the UAV. The other three are colored light sources. The color of the light source represents the rotation data values ​​of the UAV's Yaw, Roll, and Pitch, respectively, so as to realize the transmission of UAV attitude information based on the active light source array.

[0105] When the solution control module identifies the active light source array of the UAV in the image, the specific method is as follows: It receives image and depth information from the visual sensing module, where the depth information value is represented by D. The solution control module's solution control algorithm identifies circles using the Hough transform, with the following formula:

[0106] (x i -a) 2 +(y i -b) 2 =r 2

[0107] By defining a Hough space and constructing a point in the space using the three parameters (a, b, r), in the formula, x... i and y i Let a and b be points in the image, a and b be the coordinates of the center of the circle, and r be the radius of the circle. Iterate through each point in the image (x, b, r). i y i For each point, calculate its distance r to the corresponding possible center (a, b), and increment the count S of point (a, b, r) in the Hough space. a，b，r S a，b，r The larger S is, the higher the certainty that a, b, and r can represent a circle. a,b,r When it is greater than a certain value, that is:

[0108] S a，b，r ≥S 阈值

[0109] a, b, and r represent a circle centered at (a, b) with radius r, and S 阈值 It is a set value; different values ​​are set to adjust the accuracy of circle recognition.

[0110] By identifying the number of circles within a certain range, the location of the active light source array in the image can be determined. Then, by weighting the R, G, and B values ​​of each point within the active light source region, with higher weights closer to the center, the R, G, and B values ​​of the light source color can be calculated, thereby identifying the color of each active light source.

[0111] After identifying white, two lines are extended from the white active light source as the origin to adjacent light sources. If one line can coincide with the other line when rotated 90° clockwise, then this line is used as the positive Y-axis and the other as the positive X-axis to establish a light source array coordinate system. The light source representing Yaw is on the positive X-axis, and the light source representing Pitch is on the positive Y-axis. The positions of the light sources representing Yaw, Roll, and Pitch are then identified, and the R, G, and B values ​​of the three light source colors are calculated using the following formula:

[0112]

[0113] Obtain the color coefficients r(x), g(x), and b(x), and then use the following formula:

[0114]

[0115] Calculate the rotation angle x of Yaw, Roll, or Pitch. Note that x is expressed as a rotation angle only in Formula 5.

[0116] Repeat formula five to calculate the rotation angles of the host's Yaw, Roll, and Pitch in sequence, with Yaw, Roll, and Pitch representing the rotation angle values ​​of the above three respectively.

[0117] Furthermore, by identifying the position of the active light source array of the drone in front and calculating the pixel difference (Pix) between it and the image center, the algorithm can be further optimized. x差 And Pix y差 The calculated Yaw, Roll, and Pitch are obtained using the following formula:

[0118]

[0119] The speed of rotation of the slave device along the X, Y, Z axes and the yaw angle when it tracks the master device is calculated, and the slave device is controlled to track the master device in this way.

[0120] In Formulas 6 and 7, `max` and `min` are used to limit the maximum speed of the drone to 6 m / s. The negative sign does not represent magnitude, but rather the direction of flight.

[0121] When the drone's nose is oriented towards the positive X-axis, its forward speed is positive and its backward speed is negative. If the drone's nose is oriented towards the Y-axis, its rightward speed is positive and its leftward speed is negative. If the drone's nose is oriented towards the Z-axis, its upward speed is positive and its backward speed is negative.

[0122] In Formulas 6 and 7, ΔYaw represents the change in the Yaw angle in degrees over a certain period of time; the Y-axis is the X-axis in the image, the Z-axis is the Y-axis in the image, and the Y-axis in the image increases from top to bottom.

[0123] The method for automatic multi-drone collaboration also includes the following steps;

[0124] Step S7: The slave device acquires the image and depth information of the host through the depth camera and sends it to the calculation control module. The R, G, and B values ​​of different light source colors of the host active light source array are obtained through the image.

[0125] Step S8: Calculate the rotation data values ​​of Yaw, Roll, and Pitch represented by different light sources using Formula 5. Then, using the rotation data values ​​and depth information, calculate the speed control values ​​for the X, Y, and Z axes and the Yaw angle of the slave drone to track the master drone using Formulas 6, 7, 8, and 9. This controls the slave drone to track the master drone. For multiple slave drones, repeat the above steps to emit an active light source array and track the preceding slave drone or master drone, achieving automatic multi-drone collaboration based on the active light source array.

[0126] When performing multi-drone automatic collaborative operations, both the host and slave drones are equipped with active light source arrays, while the collaborative control device is only installed on the slave drones.

[0127] In this example, when the flight attitude of the drone is relatively stable and does not change frequently during operation, a long-afterglow luminescent material that can absorb laser energy can be placed at the sunshade. By absorbing laser energy, the long-afterglow luminescent material is excited to emit light, forming a delayed luminescence effect.

[0128] In this example, each active light source contains an active light source synthesis device and a power supply control device. The white light of the white active light source is also formed by mixing three-color lasers in a specific ratio, so that the white active light source and the colored active light source can switch between each other.

[0129] In this example, when multiple drones are working together, the slave drones learn about the actions that the master drone will perform based on the attitude changes broadcast by the master drone via optical signals, and cooperate with the master drone. Alternatively, the master drone can transmit information about the tasks to be performed to the slave drones by changing the attitude of its drones.

[0130] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be included in the scope of the present invention.

Claims

1. A multi-drone automatic collaborative device based on an active light source array, characterized in that: The drone consists of a master unit and one or more slave units. Both the master unit and slave units are equipped with an active light source array, and the slave units are equipped with a cooperative control device. The active light source array includes a white light active light source that emits white light as a positioning feature for the drone, and three colored light active light sources that emit colored light. The light signal of each active light source is a light signal synthesized from red, green, and blue lasers and processed by a light shield. In multi-drone automatic cooperation, the colored light active light sources are connected to the drone's attitude sensing module to obtain the drone's attitude information through a gyroscope. Based on the different attitude information of the drone's current flight state, the drone rotates at different angles and synthesizes light signals of different colors to provide the slave units with target identification and master unit flight parameters. The collaborative control device includes a visual sensing module and a solution control module; the visual sensing module acquires image information through a depth camera with an infrared ranging module, acquires depth information in the image through the infrared ranging module, and transmits the image information and depth information to the solution control module. The calculation and control module is an onboard computer at the UAV. The onboard computer receives image information and depth information to identify the position of the UAV in the image, identify the position of the active light source on the UAV, and convert the light source color into the attitude information data of the UAV through the recognition algorithm. When the recognition algorithm is working, it first identifies the active light source array of the UAV, then identifies the position of the white active light source in the active light source array. After identifying the white active light source, it marks the three colored active light sources representing yaw angle data, roll angle data, and pitch angle data according to the position of the white active light source among the four active light sources. Finally, it extracts the R, G, and B values ​​of the light source colors of the three colored active light sources representing yaw, roll, and pitch, and calculates the R, G, and B values ​​into the corresponding rotation angles. The onboard computer stores an algorithm for controlling the UAV slave to track based on attitude and depth information. This algorithm calculates the required movement speed and angle for the UAV to track the host machine ahead using rotation angle and depth information, and controls the UAV to perform the corresponding flight.

2. The UAV multi-drone automatic collaborative device based on an active light source array according to claim 1, characterized in that: The active light source array includes four active light source synthesis devices and power supply control devices with the same structure; each active light source includes an active light source synthesis device (121) and a power supply control device (122); The active light source synthesis device includes a base (1210) and a colored light emitting device disposed on the base; a laser synthesis module and a laser adjustment module are sequentially arranged at the output optical path of the colored light emitting device; The laser combining module is located above the light output end of the light-emitting device and includes a beam combining prism (1214) and a first plano-convex mirror (1215) arranged in sequence, with the plane of the first plano-convex mirror facing the beam combining prism. The colored light-emitting device includes a red light-emitting device (1211), a blue light-emitting device (1213), and a green light-emitting device (1212) located on three sides of the light-combining prism respectively; the three-color lasers emitted by the red light-emitting device, the blue light-emitting device, and the green light-emitting device are combined into a single laser beam by the laser combining module and emitted from the light output end of the laser combining module; The optical input end of the laser adjustment module is connected to the optical output end of the laser combining module; the laser adjustment module includes a second plano-convex mirror (1216), a reflector (1217), a third plano-convex mirror (1218), and a light shield (1219) arranged sequentially along the optical path; the convex surface of the second plano-convex mirror faces the combining prism to adjust the combined laser to a fixed size; the reflector is located behind the second plano-convex mirror and is placed at 45° to reflect the combined laser upward; the third plano-convex mirror and the light shield are located above the reflector, and the laser beam intercepted and converged by the third plano-convex mirror forms a recognizable light source at the light shield; The power control module of the power supply control device includes a first controller (1221), a second controller (1222), a third controller (1223), and a fourth controller (1224); The UAV attitude sensing module (1225) is connected to the main control module (1226) via a cable; The light-combining prism is located in the groove in the middle of the base; The UAV attitude sensing module acquires the UAV's attitude data through a gyroscope; The red, blue, and green light-emitting devices are all laser diode light-emitting devices. Each laser diode light-emitting device includes a laser diode driver board (12111), a laser diode (12112), and a condenser lens (12113). When the laser diode light-emitting device is working, the laser diode driver board drives the laser diode to emit photons, and then the condenser lens focuses the photons into a beam of laser light.

3. The UAV multi-drone automatic collaborative device based on an active light source array according to claim 2, characterized in that: Each of the three colored light-emitting devices incorporates a laser diode. By exciting different semiconductors, each laser diode emits red, green, and blue photons respectively. The light signal synthesized by the lasers of each colored light-emitting device in the active light source array changes color with the rotation angles of the represented yaw, roll, and pitch, as detailed below: C = [r(x)*255, g(x)*255, b(x)*255] (Formula 1) In the formula, C represents the composite laser color. The meaning of Formula 1 is that the mixed spectral color is visually formed by mixing and adding the three basic colors of red, green, and blue in their respective proportions r(x), g(x), and b(x). The relationship between r(x), g(x), b(x) and the rotation angle is as follows: Where x is the rotation angle of the yaw, pitch, or roll to be represented, and r(x), g(x), and b(x) are the calculated color coefficients. The main control module outputs the corresponding PWM signal value to the power control module based on the calculated color coefficients r(x), g(x), and b(x).

4. The UAV multi-drone automatic collaborative device based on an active light source array according to claim 2, characterized in that: The method for automatic multi-drone collaboration includes the following steps; Step S1: The UAV attitude sensing module acquires the UAV attitude information and sends the acquired UAV attitude information to the main control module; Step S2: The UAV attitude information includes the three-axis rotation angles of Yaw, Roll, and Pitch. The main control module converts the rotation angles into color coefficients r(x), g(x), and b(x) of the color to be emitted according to Formulas 2, 3, and 4. The required power is calculated through the color coefficients, and the corresponding PWM signal value is then sent to each power control module. The PWM signal value includes the percentage of output power corresponding to the maximum control power of the power control module. Step S3: Each power control module provides the corresponding PWM value of power to each corresponding colored light-emitting device according to the PWM value output by the main control module, so as to control the emission power of the three corresponding light-emitting devices; Step S4: The three colored light-emitting devices of the active light source synthesis device emit red, blue, and green lasers that conform to the calculated color coefficients. That is, the three lasers obtained according to Formula 1 are combined into a composite laser that is output along the same straight line and whose color can represent the corresponding rotation angle value through the laser synthesis module of the active light source synthesis device. Step S5: The active light source synthesis device first processes the composite laser with the laser adjustment module to form a laser with a constant size of laser points at near and far distances. Then, it passes through a reflector and a plano-convex mirror in sequence to reach the light shield. After being processed by the light shield, it forms a light source signal that is easy to be observed and recorded by external electronic devices or humans and is then emitted. Step S6: The four active light source synthesis devices repeat the above process to form four active light source signals with identification characteristics on the UAV. One of them is a white light source that emits white light. This light source is used as the positioning feature of the UAV. The other three are colored light sources. The color of the light source represents the rotation data values ​​of the UAV's Yaw, Roll, and Pitch, respectively, so as to realize the transmission of UAV attitude information based on the active light source array.

5. The UAV multi-drone automatic collaborative device based on an active light source array according to claim 1, characterized in that: When the solution control module identifies the active light source array of the UAV in the image, the specific method is as follows: It receives image and depth information from the visual sensing module, where the depth information value is represented by D. The solution control module's solution control algorithm identifies circles using the Hough transform, with the following formula: (x i -a) 2 +(y i -b) 2 =r 2 By defining a Hough space and constructing a point in the space using the three parameters (a, b, r), in the formula, x... i and y i Let a and b be points in the image, a and b be the coordinates of the center of the circle, and r be the radius of the circle; iterate through each point in the image (x, b, and r). i y i For each point, calculate its distance r to the corresponding center (a, b), and increment the count S of point (a, b, r) in the Hough space. a，b，r S a，b，r The larger S is, the higher the certainty that a, b, and r can represent a circle. a，b，r When it is greater than the set value: S a，b，r ≥S 阈值 a, b, and r represent a circle centered at (a, b) with radius r, and S 阈值 It is a set value; different values ​​are set to adjust the accuracy of circle recognition. By identifying the number of circles, the location of the active light source array in the image can be determined. Then, by weighting the R, G, and B values ​​of each point within the active light source region, with higher weights closer to the center, the R, G, and B values ​​of the light source color can be calculated, thus identifying the color of each active light source. After identifying white, two lines are extended from the white active light source as the origin to adjacent light sources. If one line can coincide with the other line when rotated 90° clockwise, then this line is used as the positive Y-axis and the other as the positive X-axis to establish a light source array coordinate system. The light source representing Yaw is on the positive X-axis, and the light source representing Pitch is on the positive Y-axis. The positions of the light sources representing Yaw, Roll, and Pitch are then identified, and the R, G, and B values ​​of the three light source colors are calculated using the following formula: Obtain the color coefficients r(x), g(x), and b(x), and then use the following formula: Calculate the rotation angle x of Yaw, Roll, or Pitch. Note that x is expressed as a rotation angle only in Formula 5. Repeat formula five to calculate the rotation angles of the host's Yaw, Roll, and Pitch in sequence, with Yaw, Roll, and Pitch representing the rotation angle values ​​of the above three respectively. The position of the active light source array of the drone in front is identified by the preceding data, and the pixel difference (Pix) between it and the image center is calculated. x差 And Pix y差 The calculated Yaw, Roll, and Pitch are obtained using the following formula: Calculate the rotational speeds of the X, Y, Z axes and the yaw angle when the slave device tracks the master device, and use this to control the slave device to track the master device; In Formulas 6 and 7, `max` and `min` are used to limit the maximum speed of the drone to 6 m / s. The negative sign does not represent magnitude, but rather the direction of flight. When the drone's nose is oriented towards the positive X-axis, its forward speed is positive and its backward speed is negative. If the drone's nose is oriented towards the Y-axis, its rightward speed is positive and its leftward speed is negative. If the drone's nose is oriented towards the Z-axis, its upward speed is positive and its backward speed is negative. In Formulas 6 and 7, ΔYaw represents the change in the Yaw angle in degrees; the Y-axis is the X-axis in the image, the Z-axis is the Y-axis in the image, and the Y-axis in the image increases from top to bottom.

6. The UAV multi-drone automatic collaborative device based on an active light source array according to claim 5, characterized in that: The method for automatic multi-drone collaboration also includes the following steps; Step S7: The slave device acquires the image and depth information of the host through the depth camera and sends it to the calculation control module. The R, G, and B values ​​of different light source colors of the host active light source array are obtained through the image. Step S8: Calculate the rotation data values ​​of Yaw, Roll, and Pitch represented by different light sources using Formula 5, and use the rotation data values ​​and depth information to calculate the speed control values ​​of the X, Y, Z axes and Yaw angle of the slave tracking the master using Formulas 6, 7, 8 and 9, so as to control the slave to track the master. For multiple slave drones, repeat the above steps to emit an active light source array and track the preceding slave or master drone to achieve automatic multi-drone collaboration based on the active light source array.

7. The UAV multi-drone automatic collaborative device based on an active light source array according to claim 1, characterized in that: When performing multi-drone automatic collaborative operations, both the host and slave drones are equipped with active light source arrays, while the collaborative control device is only installed on the slave drones.

Images