Octree map expansion method, device, equipment, product and medium

Abstract

The invention relates to the technical field of map expansion, and provides an octree map expansion method and device, equipment, a product and a medium. The method comprises the following steps: acquiring a plurality of point cloud data of a target area in front of an aircraft; dividing the multiple pieces of point cloud data to obtain point cloud data corresponding to at least one danger level; expanding the point cloud data corresponding to the at least one danger level based on an expansion size matched with the danger level to obtain expanded point cloud data; and updating the expanded point cloud data to the octree map of the target area to obtain an expanded octree map. According to the method, the differential expansion based on the danger level is performed on the point cloud data, so that the map expansion size is reasonable and accords with a real scene, the accuracy and efficiency of path planning are improved, the energy consumption of the aircraft on the planned path is reduced, and the flight safety of the aircraft is ensured.

Description

Technical Field

[0001] This application relates to the field of map inflation technology, specifically to an octree map inflation method, apparatus, equipment, product, and medium. Background Technology

[0002] In recent years, research and applications of various types of aircraft in civilian fields have been increasing, such as target tracking, image stitching, power line inspection, island monitoring, coastline patrol, post-disaster monitoring, and river flood season monitoring. However, for an aircraft to accomplish these tasks, it must have the ability to fly autonomously, meaning it can complete the task independently without external assistance in any complex environment.

[0003] The ability to build maps online is a crucial aspect of enabling autonomous flight for aircraft. Existing online map building methods mainly rely on octree map inflation. However, this method inflates the data in the octree map by expanding the circumscribed sphere according to the aircraft's maximum virtual diameter. On the one hand, all data is inflated to the maximum size, resulting in an excessively large map. Therefore, path planning under this map is inefficient, and the planned paths consume more energy. On the other hand, all data is inflated to the same size, assuming that all data retain the same safety margin, which does not match real-world scenarios. Therefore, the accuracy of path planning under this map is low.

[0004] In summary, existing octree map inflation methods, by maximizing inflation of all data, result in excessively inflated maps that do not match the real-world scenario. Consequently, path planning on such maps is less accurate and efficient, and the planned paths lead to higher energy consumption for the aircraft, making it difficult to ensure flight safety. Summary of the Invention

[0005] This application provides an octree map inflation method, apparatus, device, product, and medium to solve the technical problems of existing octree map inflation methods, which maximize the inflation of all data, resulting in excessive map inflation that does not match the real scene. Consequently, the accuracy and efficiency of path planning under such maps are low, and the planned paths cause high energy consumption for aircraft, making it difficult to ensure flight safety.

[0006] In a first aspect, the octree map expansion method of this application includes: Acquire multiple point cloud data points of the target area in front of the aircraft; The multiple point cloud data are divided to obtain point cloud data corresponding to at least one hazard level; Based on the expansion size adapted to the hazard level, the point cloud data corresponding to the at least one hazard level is expanded to obtain expanded point cloud data. The expanded point cloud data is then updated into the octree map of the target area to obtain the expanded octree map.

[0007] In one embodiment, dividing the plurality of point cloud data to obtain point cloud data corresponding to at least one hazard level includes: Voxel filtering is performed on the multiple point cloud data to obtain multiple filtered point cloud data; Based on a preset expansion radius, the multiple filtered point cloud data are downsampled to obtain multiple sampled point cloud data. The sampled point cloud data are input into a pre-trained point cloud classification model to obtain at least one point cloud data category output by the point cloud classification model. The at least one point cloud data category is classified into at least one danger level; Determine the point cloud data corresponding to the at least one hazard level.

[0008] In one embodiment, downsampling the plurality of filtered point cloud data based on a preset dilation radius to obtain a plurality of sampled point cloud data includes: Select any one point cloud data from the multiple filtered point cloud data as a sampling point and add it to the sampling point set; Calculate the distance between the filtered point cloud data outside the sampling point set and the sampling points to obtain multiple distances; Filter out target distances greater than the preset expansion radius from the multiple distances, determine the point cloud data corresponding to the target distance as new sampling points and add them to the sampling point set, and return to calculate the distance between the filtered point cloud data outside the sampling point set and the sampling points, thus obtaining multiple distance steps; If the number of sampling points reaches a preset number or the target distance is not found among the multiple distances, the multiple sampling points at this time are determined as the multiple sampled point cloud data.

[0009] In one embodiment, the step of dilating the point cloud data corresponding to the at least one hazard level based on an dilution size adapted to the hazard level to obtain dilated point cloud data includes: For each sampled point cloud data, a sphere with a preset expansion unit is generated centered on that point cloud data, resulting in multiple spheres; the preset expansion unit is adapted to the hazard level of the sampled point cloud data. The multiple spheres are merged and deduplicated to obtain the expanded region; The filtered point cloud data within the inflated region is determined as the inflated point cloud data.

[0010] In one embodiment, acquiring multiple point cloud data of the target area in front of the aircraft includes: The aircraft acquires multiple image data of the target area using its binocular camera and the reflective devices on both sides. The multiple image data are transformed from the pixel coordinate system to the inertial coordinate system of the aircraft to obtain multiple point cloud data.

[0011] In one embodiment, the transformation of the plurality of image data from the pixel coordinate system to the inertial coordinate system of the aircraft to obtain plurality of point cloud data includes: Based on the camera intrinsics of the binocular camera, the multiple image data are transformed from the pixel coordinate system to the camera coordinate system; Based on the camera mounting position of the binocular camera and the pose information of the aircraft, the multiple image data are transformed from the camera coordinate system to the inertial coordinate system to obtain multiple point cloud data.

[0012] In one embodiment, updating the dilated point cloud data to the octree map of the target region to obtain the dilated octree map includes: Based on spatial location, the expanded point cloud data is updated to the cubes in the octree map that do not contain child cubes; Determine whether the octree map has reached the minimum resolution; If so, then the current octree map is determined as the expanded octree map; If not, determine whether the cube contains multiple of the expanded point cloud data; If not, then the current octree map is determined to be the expanded octree map; If so, the cube is divided into eight sub-cubes, and the process returns to the step of determining whether the octree map has reached the minimum resolution, until the expanded octree map is determined.

[0013] Secondly, embodiments of this application provide an octree map expansion device, comprising: The point cloud data acquisition module is used to: acquire multiple point cloud data of the target area in front of the aircraft; The point cloud data segmentation module is used to: segment the multiple point cloud data to obtain point cloud data corresponding to at least one hazard level; The point cloud data inflation module is used to: inflate the point cloud data corresponding to the at least one hazard level based on an inflation size adapted to the hazard level, so as to obtain inflated point cloud data; The octree map inflation module is used to update the inflation point cloud data into the octree map of the target area to obtain the inflation octree map.

[0014] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the steps of the octree map expansion method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the octree map expansion method described in the first aspect.

[0016] Fifthly, embodiments of this application provide a non-transitory computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the octree map expansion method described in the first aspect.

[0017] This application provides an octree map inflation method, apparatus, device, product, and medium that acquires multiple point cloud data points of a target area in front of an aircraft, divides these point cloud data points into at least one point cloud data point corresponding to a hazard level, inflates the point cloud data point corresponding to the at least one hazard level based on an inflation size adapted to the hazard level, and updates the inflated point cloud data point to an octree map of the target area, resulting in an inflated octree map. This application divides the point cloud data according to hazard levels and sets inflation sizes corresponding to those hazard levels. Each point cloud data point is inflated according to an inflation size adapted to its hazard level, achieving differentiated inflation of point cloud data based on its hazard level. On the one hand, this avoids all point cloud data point being inflated to the maximum size, making the map inflation size more reasonable, thereby improving the efficiency of path planning and reducing the energy consumption of the aircraft along the planned path. On the other hand, it avoids all point cloud data point being inflated to the same size, allowing each point cloud data point to retain differentiated safety margins based on its hazard level, which is more consistent with real-world scenarios and thus improves the accuracy of path planning. In summary, this application improves the accuracy and efficiency of path planning by performing differentiated expansion of point cloud data based on hazard level, resulting in a reasonable map expansion size that matches the real scene. This also reduces the energy consumption of aircraft along the planned path and ensures the flight safety of the aircraft. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1This is one of the flowcharts illustrating the octree map expansion method provided in the embodiments of this application; Figure 2 This is the second flowchart illustrating the octree map expansion method provided in this application embodiment; Figure 3 This is the third flowchart illustrating the octree map expansion method provided in this application embodiment; Figure 4 This is the fourth flowchart illustrating the octree map expansion method provided in this application embodiment; Figure 5 This is the fifth flowchart illustrating the octree map expansion method provided in the embodiments of this application; Figure 6 This is the optical path diagram for image data acquisition in the octree map dilation method provided in this application embodiment; Figure 7 This is the sixth flowchart illustrating the octree map expansion method provided in this application embodiment; Figure 8 This is a schematic diagram of the camera coordinate system and pixel coordinate system in the octree map dilation method provided in this application embodiment; Figure 9 This is the seventh flowchart illustrating the octree map expansion method provided in this application embodiment; Figure 10 This is one of the schematic diagrams of the octree map in the octree map expansion method provided in the embodiments of this application; Figure 11 This is the second schematic diagram of the octree map in the octree map expansion method provided in the embodiments of this application; Figure 12 This is a schematic diagram of the octree map expansion device provided in the embodiments of this application; Figure 13 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] Figure 1 This is one of the flowcharts illustrating the octree map expansion method provided in this application. (Refer to...) Figure 1 This application provides an octree map inflation method, which may include: Step 101: Acquire multiple point cloud data of the target area in front of the aircraft; Step 102: Divide the multiple point cloud data into points cloud data corresponding to at least one hazard level; Step 103: Based on the expansion size adapted to the hazard level, expand the point cloud data corresponding to at least one hazard level to obtain expanded point cloud data. Step 104: Update the expanded point cloud data to the octree map of the target area to obtain the expanded octree map.

[0022] The aircraft can be any type of aircraft used in the civilian field, and there is no limitation here. In this embodiment, the aircraft can be a drone.

[0023] In step 101, since octree maps are typically generated based on point cloud data, it is necessary to first acquire point cloud data of the target area in front of the aircraft to lay the foundation for the expansion of the octree map. The point cloud data can be acquired directly or converted from other types of data; this is not limited here. When directly acquiring point cloud data is relatively easy, the direct acquisition method can be used. When directly acquiring other types of data is relatively easy, other types of data can be acquired first, and then converted into point cloud data based on certain conversion relationships between them and point cloud data.

[0024] In steps 102 and 103, since the hazard levels of different obstacles are usually not consistent, the point cloud data constituting these obstacles also correspond to different hazard levels. Therefore, it is necessary to classify these point cloud data and perform differentiated expansion for point cloud data of different hazard levels to retain a more accurate safety margin. For example, for obstacles with higher hazard levels, their point cloud data should also be classified as having a higher hazard level and a larger expansion size should be set accordingly to retain a larger safety margin; for obstacles with lower hazard levels, their point cloud data should also be classified as having a lower hazard level and a smaller expansion size should be set to retain a smaller safety margin. The classification of the hazard level of point cloud data can be performed using any algorithm or model, and is not limited here.

[0025] In step 104, the expanded point cloud data is updated to the octree map, which realizes differentiated expansion of the data according to the danger level, making the expanded octree map size more reasonable and more in line with the real scene, improving the accuracy and efficiency of route planning, and reducing the energy consumption of the aircraft on the planned route.

[0026] Furthermore, the octree map can be updated in real time during the flight of the aircraft, enabling flexible expansion of the octree map and flexible adjustment of the planned route.

[0027] The octree map inflation method provided in this embodiment acquires multiple point cloud data points of the target area in front of the aircraft, divides these point cloud data points into at least one point cloud data point corresponding to a hazard level, and inflates the point cloud data point corresponding to the at least one hazard level based on an inflation size adapted to the hazard level, obtaining inflated point cloud data. The inflated point cloud data is then updated in the octree map of the target area, resulting in an inflated octree map. This embodiment divides the point cloud data according to hazard level and sets an inflation size corresponding to each hazard level. Each point cloud data point is inflated according to an inflation size adapted to its hazard level, achieving differentiated inflation of point cloud data based on its hazard level. On the one hand, this avoids all point cloud data points being inflated to the maximum size, making the map inflation size more reasonable, thereby improving the efficiency of path planning and reducing the energy consumption of the aircraft on the planned path. On the other hand, it avoids all point cloud data points being inflated to the same size, allowing each point cloud data point to retain a differentiated safety margin based on its hazard level, which is more consistent with real-world scenarios, thereby improving the accuracy of path planning. In summary, this embodiment improves the accuracy and efficiency of path planning by performing differentiated expansion of point cloud data based on hazard level, resulting in a reasonable map expansion size that matches the real scene. This also reduces the energy consumption of aircraft along the planned path, ensuring the flight safety of the aircraft.

[0028] Figure 2 This is the second schematic flowchart of the octree map expansion method provided in this application embodiment. (Refer to...) Figure 2 In one embodiment, step 102 may include: Step 201: Perform voxel filtering on multiple point cloud data to obtain multiple filtered point cloud data; Step 202: Downsample multiple filtered point cloud data based on a preset expansion radius to obtain multiple sampled point cloud data; Step 203: Input multiple sampled point cloud data into the pre-trained point cloud classification model to obtain at least one point cloud data category output by the point cloud classification model; Step 204: Classify at least one point cloud data category into at least one hazard level; Step 205: Determine the point cloud data corresponding to at least one hazard level.

[0029] In step 201, voxel filtering involves dividing the point cloud space into regular cubic grids, i.e., voxels, and aggregating or filtering the point cloud data within each voxel to remove noise and retain meaningful geometric features.

[0030] In step 202, considering that if the amount of data is too large when classifying point cloud data later, it may trigger the performance bottleneck of the model, the filtered point cloud data is downsampled first to reduce the amount of data involved in classification. However, the reduction in quantity means that the data becomes sparse. Therefore, downsampling is performed with a preset expansion radius as a condition so that the sampled point cloud data meets the preset expansion radius condition. This condition can be set according to actual needs and is not limited here, as long as it can ensure that the relatively sparse point cloud data after sampling is not affected by the data sparsity in the expansion effect.

[0031] In step 203, the pre-trained point cloud classification model is the model trained using historically sampled point cloud data and their corresponding classification labels, based on any point cloud classification model.

[0032] In this embodiment, the point cloud classification model can be the PointNet model. The core of this model is to extract features from each point cloud data and use an aggregation layer to fuse these features to obtain global features, ensuring that the model is insensitive to the order of the point cloud data.

[0033] The aggregation layer can be a pooling layer, which can be either a max pooling layer or an average pooling layer. Max pooling layers can retain local maxima and highlight key features, while average pooling layers can retain overall features, suppress noise, and smooth the output. For point cloud classification tasks, highlighting key features and achieving accurate classification are more important; therefore, this embodiment uses a max pooling layer as the aggregation layer.

[0034] Depending on the requirements of the classification task, a classification layer can be added, and an activation function can be used to output category labels. These category labels represent the obstacle type to which the point cloud data belongs.

[0035] In steps 204 and 205, the hazard levels of different obstacle categories may be the same or different. If obstacle categories with the same hazard level are grouped together, then all point cloud data under these obstacle categories belong to that hazard level.

[0036] The classification of hazard levels can be set according to actual needs, and is not limited here. In this embodiment, the hazard levels can be divided into safe, low risk, medium risk, and high risk. Assuming that the hazard levels of obstacle categories A and B are both safe, then the hazard levels of all point cloud data under obstacle categories A and B are also safe. Assuming that the hazard level of obstacle category A is safe and the hazard level of obstacle category B is high risk, then the hazard levels of all point cloud data under obstacle category A are also safe, and the hazard levels of all point cloud data under obstacle category B are also high risk. It should be noted that before step 201, multiple point cloud data can be preprocessed, such as point cloud centralization and normalization, to eliminate the influence of point cloud location information and to use data augmentation techniques to improve data diversity.

[0037] In this embodiment, voxel filtering is first applied to the point cloud data to remove noise. Then, the point cloud data is downsampled based on a preset expansion radius to ensure that the expansion effect is not affected by the data sparsity while the point cloud is sparse. Next, a pre-trained point cloud classification model is used to identify and classify the relatively sparse point cloud data to improve classification efficiency. Finally, the point cloud data of each category is divided into different danger levels, thereby achieving accurate and efficient classification of the danger level of each point cloud data.

[0038] Figure 3 This is the third flowchart illustrating the octree map expansion method provided in this application. (Refer to...) Figure 3 In one embodiment, step 202 may include: Step 301: Determine any one point cloud data from the multiple filtered point cloud data as a sampling point and add it to the sampling point set; Step 302: Calculate the distance between the filtered point cloud data outside the sampling point set and the sampling point to obtain multiple distances; Step 303: Select the target distance that is greater than the preset expansion radius from multiple distances, determine the point cloud data corresponding to the target distance as a new sampling point and add it to the sampling point set, then return to step 302; Step 304: If the number of sampling points reaches the preset number or the target distance is not found among the multiple distances, the multiple sampling points at this time are determined as multiple sampled point cloud data.

[0039] In step 301, it is assumed that there are multiple filtered point cloud datasets. The set of sampling points is The preset expansion radius is , It can be the minimum expansion radius required for the octree map, or it can be set as the minimum resolution of the octree map, or it can be appropriately increased based on the obstacle avoidance requirements on the basis of the minimum resolution.

[0040] In steps 302 to 304, choose any one Add sampling points to In the middle, then ,calculate In addition to All point cloud data other than The distance between them is obtained That distance, from this Filter out those greater than a certain distance. The target distance, assuming the point cloud data corresponding to this target distance is... Then Identify as a new sampling point and add to In the middle, then Return to step 302 and continue the calculation. In addition to All point cloud data other than The distance between them is obtained Given a distance, continue filtering from it to find those greater than [a certain value]. The target distance is determined, and the corresponding point cloud data is identified as a new sampling point and added to the target distance. The process continues in a loop until a certain loop ends. The number of sampling points in the sample has reached the preset number or it is impossible to filter out more than the preset number. The target distance is then determined, and downsampling is completed at this point. The point cloud data in the image is the sampled point cloud data.

[0041] In this embodiment, because It is the minimum expansion radius required for an octree map, therefore, on the one hand, two distances are less than The point cloud data are approximately identical at the resolution of the octree map, making them difficult to distinguish effectively. Furthermore, regardless of the size at which a point cloud data point is inflated, its inflation radius will always be greater than [a certain value]. This ensures that the expanded region will necessarily contain another point cloud data, without affecting the subsequent inclusion of the other point cloud data into the expanded point cloud data. Therefore, by retaining one point cloud data and deleting the other point cloud data in the above manner, we can ensure data sparsity while also ensuring that the subsequent expansion effect is not affected by data sparsity.

[0042] Figure 4 This is the fourth flowchart illustrating the octree map expansion method provided in this application. (Refer to...) Figure 4 In one embodiment, step 103 may include: Step 401: For each sampled point cloud data in the multiple sampled point cloud data, generate a sphere with a radius of a preset expansion unit centered on it, and obtain multiple spheres; The preset inflation unit is adapted to the hazard level of the sampled point cloud data; Step 402: Merge multiple spheres to remove duplicates and obtain the expanded region; Step 403: Determine the filtered point cloud data within the dilated region as the dilated point cloud data.

[0043] In step 401, the preset expansion unit corresponds one-to-one with the hazard level. The higher the hazard level, the larger the preset expansion unit, which means that the safety margin that needs to be retained is larger. The specific size of the preset expansion unit can be set according to actual needs and is not limited here.

[0044] In this embodiment: When the hazard level is safe, the preset inflation unit is set to 0, meaning that point cloud data of this hazard level does not need to be inflated. When the hazard level is low risk, the preset expansion unit is set to 1, that is, the point cloud data of this hazard level is expanded with an expansion unit as the radius; When the hazard level is medium risk, the preset expansion unit is set to 2, that is, the point cloud data of this hazard level is expanded with a radius of 2 expansion units; When the hazard level is high risk, the preset expansion unit is set to 3, meaning that the point cloud data of this hazard level is expanded with a radius of 3 expansion units.

[0045] As can be seen from the above, as the danger level increases, the preset expansion unit also increases to ensure that the point cloud data of each danger level can retain a reasonable safety margin.

[0046] In steps 402 to 403, since there may be overlapping areas among the spheres, all spheres are merged and deduplicated, which effectively and concisely expands the influence range of the risk area. The filtered point cloud data is obtained from the obtained expanded area as the expanded point cloud data. This point cloud data includes some point cloud data that was removed during the previous downsampling, making the characterization of potential obstacles in the risk area more accurate and helping the UAV to avoid obstacles during navigation.

[0047] In this embodiment, the preset expansion unit is increased synchronously with the increase of the danger level, so that the safety margin of the point cloud data increases with the increase of the danger level, so as to accurately reflect the potential risks in the environment. The representative spherical space of the safety margin is merged and deduplicated to obtain an expansion area that meets the safety margin requirements of each point cloud data and allows the spherical space to be merged and expanded. The point cloud data in this expansion area represents the potential obstacles in the risk area. Updating the octree map with the expanded data helps to effectively avoid potential obstacles in subsequent route planning.

[0048] Figure 5 This is the fifth flowchart illustrating the octree map expansion method provided in this application. (Refer to...) Figure 5 In one embodiment, step 101 may include: Step 501: Acquire multiple image data of the target area based on the binocular camera of the aircraft and the reflective devices on both sides; Step 502: Transform multiple image data from the pixel coordinate system to the inertial coordinate system of the aircraft to obtain multiple point cloud data.

[0049] In step 501, the distance between the focal points of the two cameras in the binocular camera of the aircraft is its baseline. The length of the baseline determines the accuracy of long-distance ranging. The longer the baseline, the more obvious the parallax produced by objects at the same distance, and the more accurate the long-distance ranging.

[0050] Traditional binocular cameras for aircraft have a short baseline, resulting in poor accuracy in long-distance ranging. Therefore, this step involves adding symmetrical reflective devices to both sides of the binocular camera. In addition to receiving light directly reflected from the object, the binocular camera also receives light reflected again by the reflective devices. By adjusting the distance between the symmetrical reflective devices to be greater than their original baseline length, a longer virtual baseline is formed between the virtual focal points of the symmetrical reflective devices, thereby increasing the baseline length and achieving more accurate ranging of distant objects.

[0051] The number of symmetrical reflectors can be set based on actual needs. When a pair of reflectors is set, the virtual baseline length can be increased by continuously increasing the distance between the reflectors. When multiple pairs of reflectors are set, the distance between each pair of reflectors can be set to be different. At this time, the camera can receive light reflected multiple times from multiple pairs of reflectors, and its virtual baseline length is the distance between the virtual focal points of the pair of reflectors with the largest distance.

[0052] This embodiment uses a pair of reflecting devices as an example. The optical path diagram for image data acquisition is as follows: Figure 6 As shown; in addition, the reflector is not incorporated into the aircraft's servo system, so its increased cost and weight are more easily kept within an acceptable range.

[0053] In step 502, on the one hand, image data in the pixel coordinate system only contains two-dimensional pixel coordinates and local depth information, and cannot be directly spatiotemporally aligned with data from other sensors (such as inertial measurement units (IMU) and global positioning systems (GPS). Point cloud data in the inertial coordinate system can unify the reference frame of multiple source sensors and achieve spatiotemporal alignment of data from multiple sensors. On the other hand, image data in the pixel coordinate system only reflects instantaneous observations from the camera's perspective and cannot maintain the stability of the environmental representation when the aircraft is moving. Point cloud data in the inertial coordinate system can compensate for changes in the aircraft's attitude and help build a globally consistent 3D map. Therefore, it is necessary to convert image data in the pixel coordinate system to point cloud data in the inertial coordinate system. One approach is to pre-define the transformation relationship between the pixel coordinate system and the inertial coordinate system, and then transform the image data from the pixel coordinate system to the inertial coordinate system based on this transformation relationship. Alternatively, other coordinate systems can be used as intermediate transformation bridges between the pixel coordinate system and the inertial coordinate system. The transformation relationships between the pixel coordinate system, the inertial coordinate system, and these other coordinate systems can be calibrated respectively, and the image data can be first transformed from the pixel coordinate system to the other coordinate system based on this transformation relationship, and then transformed from the other coordinate system to the inertial coordinate system.

[0054] This embodiment introduces reflective devices on both sides of the binocular camera to increase the virtual baseline length, widen the ranging range of the binocular camera, and improve the ranging accuracy of the binocular camera, thereby obtaining more high-quality image data. Then, it transforms the image data from the pixel coordinate system to the inertial coordinate system of the aircraft to obtain high-quality point cloud data, which helps to increase the environmental perception capability of the aircraft.

[0055] Figure 7 This is the sixth flowchart illustrating the octree map expansion method provided in this application. (Refer to...) Figure 7 In one embodiment, step 502 may include: Step 701: Based on the camera intrinsics of the binocular camera, transform multiple image data from the pixel coordinate system to the camera coordinate system; Step 702: Based on the camera installation position of the binocular camera and the pose information of the aircraft, transform multiple image data from the camera coordinate system to the inertial coordinate system to obtain multiple point cloud data.

[0056] In acquiring point cloud data, it is necessary to first clarify the definitions of various coordinate systems and their interrelationships to ensure the accuracy of data transformation. The definitions of the camera coordinate system, inertial coordinate system, body coordinate system, and pixel coordinate system are as follows: 1. Camera coordinate system: such as Figure 8 As shown, middle, The axis points in front of the camera. Axis to the right, Axial direction downwards, following the right-hand rule, origin The optical center of the camera.

[0057] 2. Inertial coordinate system: The northeast coordinate system is adopted here, that is, due north is the reference coordinate system. The axis is due east. axis, The axes are established by the right-hand rule. The origin of the coordinate system is the coordinate point at the moment before the aircraft takes off. This coordinate system is fixed to the Earth and moves with the Earth's rotation.

[0058] 3. Aircraft coordinate system: The forward direction of the aircraft is... The axis, to the right is axis, The axes are determined by the right-hand rule, and the origin is the instantaneous center of gravity of the aircraft. The transformation relationship between the body coordinate system and the camera coordinate system is only related to the camera installation position. Therefore, the transformation relationship can be obtained through prior calibration. Here, it is assumed that the transformation matrix from the camera coordinate system to the body coordinate system is known.

[0059] 4. Pixel coordinate system: such as Figure 8 As shown, the origin Located in the top left corner of the image, Axis to the right of the camera coordinate system The axes are parallel. Axis downward relative to the camera coordinate system The transformation matrix between the pixel coordinate system and the camera coordinate system is called the camera intrinsic parameter. The camera intrinsic parameter is fixed after leaving the factory and will not change during use. It is also assumed here that the camera intrinsic parameter is known.

[0060] In step 701, based on the above description, multiple image data can be transformed from the pixel coordinate system to the camera coordinate system using the following formula: ; in, These are the image data coordinates in the pixel coordinate system. That is, camera internal parameters. , , , All are intrinsic parameters. This refers to the coordinates of the image data transformed into the camera coordinate system. Among them, That is, the depth information of each pixel can be obtained by knowing the size of the object or by a certain adaptation algorithm. There is no limitation here. In this embodiment, the disparity map can be calculated by a stereo matching algorithm.

[0061] In step 702, based on the above description, the following formula can be used to further... Transformation from camera coordinate system to inertial coordinate system: ; in, for Coordinates in an inertial coordinate system, i.e., point cloud data coordinates. This is the transformation matrix from the aircraft's body coordinate system to its inertial coordinate system, which can be obtained from the aircraft's attitude information. The transformation matrix from the known camera coordinate system to the body coordinate system is determined by the camera's mounting position.

[0062] In this embodiment, the camera coordinate system and the body coordinate system are used as bridges between the pixel coordinate system and the inertial coordinate system. First, the image data is transformed from the pixel coordinate system to the camera coordinate system using the camera intrinsic parameters. Then, the data in the camera coordinate system is transformed to the body coordinate system using the camera installation position. Finally, the data in the body coordinate system is transformed to the inertial coordinate system using the aircraft's pose information, thereby obtaining point cloud data with accurate spatial positioning, laying a good data foundation for subsequent processing.

[0063] Figure 9 This is the seventh flowchart illustrating the octree map expansion method provided in this application. (Refer to...) Figure 9 In one embodiment, step 104 may include: Step 901: Based on spatial location, update the dilated point cloud data to the cubes in the octree map that do not contain child cubes; Step 902: Determine if the octree map has reached the minimum resolution; Step 903: If so, then the octree map at this time is determined as the expanded octree map; Step 904: If not, determine whether the cube contains multiple dilated point cloud data. Step 905: If not, then the octree map at this time is determined to be the expanded octree map; Step 906: If so, divide the cube into eight sub-cubes and return to step 902 until the expanded octree map is determined.

[0064] Reference Figure 10 and Figure 11 An octree is a tree-like data structure used to describe three-dimensional space. Each node in an octree represents a volume element (voxel) of a cube in 3D space. Each node has eight child nodes, and the sum of the voxels represented by the eight child nodes equals the volume of the cube corresponding to the parent node. The root node corresponds to the largest cube in 3D space. Each level corresponds to the eight sub-cubes into which the cube is continuously divided. The level containing the minimum bounded volume of the sub-cubes is called the leaf level, which represents the minimum resolution of the octree map. As shown in the diagram, when all sub-cubes of a cube are occupied or not occupied, it is unnecessary to divide the cube equally.

[0065] In step 901, the expanded point cloud data is updated to the cube at the corresponding spatial location in the current octree map, and the cube has not yet been evenly divided.

[0066] In steps 902 to 904, if the octree map has reached the minimum resolution, it means that each cube has reached the minimum limited volume and cannot be divided equally. The octree map at this time can be determined as the expanded octree map. If the octree map has not reached the minimum resolution, it means that there are cubes that can be further divided equally. At this time, the number of point cloud data in each cube can be determined.

[0067] In steps 905 and 906, if there are no multiple point cloud data points located in the same cube, it means that the spatial accuracy of each point cloud data point has reached its maximum, and the octree map at this time can be determined as the expanded octree map. If there are multiple point cloud data points located in the same cube, the cube is divided into eight equal parts to divide the multiple point cloud data points into different sub-cubes, thereby further improving the spatial accuracy of the point cloud data. Then, return to step 902 to re-determine whether the octree map has reached the minimum resolution. This process is repeated until the expanded octree map is finally determined.

[0068] In this context, if a cube in the octree map is occupied by at least one point cloud data, an occupation flag can be set for the cube. If a cube in the octree map is not occupied by any point cloud data, a blank flag can be set for the cube. To facilitate compressed storage, the occupation flag can be set to 1 and the blank flag can be set to 0. Binarization can reduce memory usage.

[0069] In this embodiment, during the process of updating the expanded point cloud data to the octree map, the spatial accuracy of the point cloud data can be improved by dividing the cube containing multiple point cloud data into eight equal parts, and the occupancy of each cube can be represented by binarization, so that the expanded octree map occupies less memory, is easier to store, and improves the efficiency of subsequent route planning based on the map.

[0070] The octree map expansion device provided in the embodiments of this application is described below. The octree map expansion device described below can be referred to in correspondence with the octree map expansion method described above.

[0071] Figure 12 This is a schematic diagram of the octree map expansion device provided in an embodiment of this application. (Refer to...) Figure 12 This application provides an octree map inflation device, which may include: The point cloud data acquisition module 1201 is used to: acquire multiple point cloud data of the target area in front of the aircraft; The point cloud data segmentation module 1202 is used to: segment the multiple point cloud data to obtain point cloud data corresponding to at least one hazard level; The point cloud data inflation module 1203 is used to: inflate the point cloud data corresponding to the at least one hazard level based on an inflation size adapted to the hazard level, so as to obtain inflated point cloud data; Octree map inflation module 1204 is used to update the inflation point cloud data to the octree map of the target area to obtain the inflation octree map.

[0072] The octree map inflation device provided in this embodiment acquires multiple point cloud data points of the target area in front of the aircraft, divides these point cloud data points into at least one point cloud data point corresponding to a hazard level, and inflates the point cloud data point corresponding to the at least one hazard level based on an inflation size adapted to the hazard level, obtaining inflated point cloud data. The inflated point cloud data is then updated into the octree map of the target area, resulting in an inflated octree map. This embodiment divides the point cloud data according to hazard levels and sets inflation sizes corresponding to those levels. Each point cloud data point is inflated according to an inflation size adapted to its hazard level, achieving differentiated inflation of point cloud data based on its hazard level. On the one hand, this avoids all point cloud data point being inflated to the maximum size, making the map inflation size more reasonable, thereby improving the efficiency of path planning and reducing the energy consumption of the aircraft along the planned path. On the other hand, it avoids all point cloud data point being inflated to the same size, allowing each point cloud data point to retain a differentiated safety margin based on its hazard level, which is more consistent with real-world scenarios, thereby improving the accuracy of path planning. In summary, this embodiment improves the accuracy and efficiency of path planning by performing differentiated expansion of point cloud data based on hazard level, resulting in a reasonable map expansion size that matches the real scene. This also reduces the energy consumption of aircraft along the planned path, ensuring the flight safety of the aircraft.

[0073] In one embodiment, the point cloud data partitioning module 1202 is specifically used for: Voxel filtering is performed on the multiple point cloud data to obtain multiple filtered point cloud data; Based on a preset expansion radius, the multiple filtered point cloud data are downsampled to obtain multiple sampled point cloud data. The sampled point cloud data are input into a pre-trained point cloud classification model to obtain at least one point cloud data category output by the point cloud classification model. The at least one point cloud data category is classified into at least one danger level; Determine the point cloud data corresponding to the at least one hazard level.

[0074] In one embodiment, the point cloud data partitioning module 1202 is specifically used for: Select any one point cloud data from the multiple filtered point cloud data as a sampling point and add it to the sampling point set; Calculate the distance between the filtered point cloud data outside the sampling point set and the sampling points to obtain multiple distances; Filter out target distances greater than the preset expansion radius from the multiple distances, determine the point cloud data corresponding to the target distance as new sampling points and add them to the sampling point set, and return to calculate the distance between the filtered point cloud data outside the sampling point set and the sampling points, thus obtaining multiple distance steps; If the number of sampling points reaches a preset number or the target distance is not found among the multiple distances, the multiple sampling points at this time are determined as the multiple sampled point cloud data.

[0075] In one embodiment, the point cloud data inflation module 1203 is specifically used for: For each sampled point cloud data, a sphere with a preset expansion unit is generated centered on that point cloud data, resulting in multiple spheres; the preset expansion unit is adapted to the hazard level of the sampled point cloud data. The multiple spheres are merged and deduplicated to obtain the expanded region; The filtered point cloud data within the inflated region is determined as the inflated point cloud data.

[0076] In one embodiment, the point cloud data acquisition module 1201 is specifically used for: The aircraft acquires multiple image data of the target area using its binocular camera and the reflective devices on both sides. The multiple image data are transformed from the pixel coordinate system to the inertial coordinate system of the aircraft to obtain multiple point cloud data.

[0077] In one embodiment, the point cloud data acquisition module 1201 is specifically used for: Based on the camera intrinsics of the binocular camera, the multiple image data are transformed from the pixel coordinate system to the camera coordinate system; Based on the camera mounting position of the binocular camera and the pose information of the aircraft, the multiple image data are transformed from the camera coordinate system to the inertial coordinate system to obtain multiple point cloud data.

[0078] In one embodiment, the octree map expansion module 1204 is specifically used for: Based on spatial location, the expanded point cloud data is updated to the cubes in the octree map that do not contain child cubes; Determine whether the octree map has reached the minimum resolution; If so, then the current octree map is determined as the expanded octree map; If not, determine whether the cube contains multiple of the expanded point cloud data; If not, then the current octree map is determined to be the expanded octree map; If so, the cube is divided into eight sub-cubes, and the process returns to the step of determining whether the octree map has reached the minimum resolution, until the expanded octree map is determined.

[0079] Figure 13 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 13 As shown, the electronic device may include: a processor 1310, a communication interface 1320, a memory 1330, and a communication bus 1340, wherein the processor 1310, the communication interface 1320, and the memory 1330 communicate with each other via the communication bus 1340. The processor 1310 can call a computer program in the memory 1330 to execute the steps of the octree map expansion method, such as including: Acquire multiple point cloud data points of the target area in front of the aircraft; The multiple point cloud data are divided to obtain point cloud data corresponding to at least one hazard level; Based on the expansion size adapted to the hazard level, the point cloud data corresponding to the at least one hazard level is expanded to obtain expanded point cloud data. The expanded point cloud data is then updated into the octree map of the target area to obtain the expanded octree map.

[0080] Furthermore, the logical instructions in the aforementioned memory 1330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the octree map expansion method provided in the above embodiments, such as including: Acquire multiple point cloud data points of the target area in front of the aircraft; The multiple point cloud data are divided to obtain point cloud data corresponding to at least one hazard level; Based on the expansion size adapted to the hazard level, the point cloud data corresponding to the at least one hazard level is expanded to obtain expanded point cloud data. The expanded point cloud data is then updated into the octree map of the target area to obtain the expanded octree map.

[0082] On the other hand, embodiments of this application also provide a non-transitory computer-readable storage medium storing a computer program thereon, the computer program being used to cause a processor to execute the steps of the octree map expansion method provided in the above embodiments, for example including: Acquire multiple point cloud data points of the target area in front of the aircraft; The multiple point cloud data are divided to obtain point cloud data corresponding to at least one hazard level; Based on the expansion size adapted to the hazard level, the point cloud data corresponding to the at least one hazard level is expanded to obtain expanded point cloud data. The expanded point cloud data is then updated into the octree map of the target area to obtain the expanded octree map.

[0083] The non-transitory computer-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0084] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An octree map expansion method, characterized in that, include: Acquire multiple point cloud data points of the target area in front of the aircraft; The multiple point cloud data are divided to obtain point cloud data corresponding to at least one hazard level; Based on the expansion size adapted to the hazard level, the point cloud data corresponding to the at least one hazard level is expanded to obtain expanded point cloud data. The expanded point cloud data is then updated into the octree map of the target area to obtain the expanded octree map.

2. The octree map expansion method according to claim 1, characterized in that, The step of dividing the multiple point cloud data to obtain point cloud data corresponding to at least one hazard level includes: Voxel filtering is performed on the multiple point cloud data to obtain multiple filtered point cloud data; Based on a preset expansion radius, the multiple filtered point cloud data are downsampled to obtain multiple sampled point cloud data. The sampled point cloud data are input into a pre-trained point cloud classification model to obtain at least one point cloud data category output by the point cloud classification model. The at least one point cloud data category is classified into at least one danger level; Determine the point cloud data corresponding to the at least one hazard level.

3. The octree map expansion method according to claim 2, characterized in that, The downsampling of the multiple filtered point cloud data based on a preset expansion radius yields multiple sampled point cloud data, including: Select any one point cloud data from the multiple filtered point cloud data as a sampling point and add it to the sampling point set; Calculate the distance between the filtered point cloud data outside the sampling point set and the sampling points to obtain multiple distances; Filter out target distances greater than the preset expansion radius from the multiple distances, determine the point cloud data corresponding to the target distance as new sampling points and add them to the sampling point set, and return to calculate the distance between the filtered point cloud data outside the sampling point set and the sampling points, thus obtaining multiple distance steps; If the number of sampling points reaches a preset number or the target distance is not found among the multiple distances, the multiple sampling points at this time are determined as the multiple sampled point cloud data.

4. The octree map expansion method according to claim 2, characterized in that, The process of dilating the point cloud data corresponding to the at least one hazard level based on an expansion size adapted to the hazard level to obtain expanded point cloud data includes: For each sampled point cloud data, a sphere with a preset expansion unit is generated centered on that point cloud data, resulting in multiple spheres; the preset expansion unit is adapted to the hazard level of the sampled point cloud data. The multiple spheres are merged and deduplicated to obtain the expanded region; The filtered point cloud data within the inflated region is determined as the inflated point cloud data.

5. The octree map expansion method according to claim 1, characterized in that, The acquisition of multiple point cloud data of the target area in front of the aircraft includes: The aircraft acquires multiple image data of the target area using its binocular camera and the reflective devices on both sides. The multiple image data are transformed from the pixel coordinate system to the inertial coordinate system of the aircraft to obtain multiple point cloud data.

6. The octree map expansion method according to claim 5, characterized in that, The process of transforming the multiple image data from the pixel coordinate system to the inertial coordinate system of the aircraft to obtain multiple point cloud data includes: Based on the camera intrinsics of the binocular camera, the multiple image data are transformed from the pixel coordinate system to the camera coordinate system; Based on the camera mounting position of the binocular camera and the pose information of the aircraft, the multiple image data are transformed from the camera coordinate system to the inertial coordinate system to obtain multiple point cloud data.

7. The octree map expansion method according to claim 1, characterized in that, The step of updating the dilated point cloud data to the octree map of the target area to obtain the dilated octree map includes: Based on spatial location, the expanded point cloud data is updated to the cubes in the octree map that do not contain child cubes; Determine whether the octree map has reached the minimum resolution; If so, then the current octree map is determined as the expanded octree map; If not, determine whether the cube contains multiple of the expanded point cloud data; If not, then the current octree map is determined to be the expanded octree map; If so, the cube is divided into eight sub-cubes, and the process returns to the step of determining whether the octree map has reached the minimum resolution, until the expanded octree map is determined.

8. An octree map expansion device, characterized in that, include: The point cloud data acquisition module is used to: acquire multiple point cloud data of the target area in front of the aircraft; The point cloud data segmentation module is used to: segment the multiple point cloud data to obtain point cloud data corresponding to at least one hazard level; The point cloud data inflation module is used to: inflate the point cloud data corresponding to the at least one hazard level based on an inflation size adapted to the hazard level, so as to obtain inflated point cloud data; The octree map inflation module is used to update the inflation point cloud data into the octree map of the target area to obtain the inflation octree map.

9. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the octree map expansion method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the octree map expansion method according to any one of claims 1 to 7.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the octree map expansion method according to any one of claims 1 to 7.

Images