Indoor storage unmanned aerial vehicle positioning and navigation method

Abstract

The invention relates to the technical field of unmanned aerial vehicle navigation, in particular to an indoor storage unmanned aerial vehicle positioning and navigation method, which comprises the following steps of: controlling a checking unmanned aerial vehicle to move in a to-be-checked warehouse by adopting a checking track, and checking the corresponding position of a goods shelf by adopting the checking unmanned aerial vehicle; and in the process of controlling the inventory unmanned aerial vehicle to move, performing fusion processing on the laser synchronous positioning grid and the visual synchronous positioning grid by adopting a Bayesian method so as to determine an actual obstacle position for avoidance. In order to solve the problem that a warehousing unmanned aerial vehicle positioning method in the prior art is poor in effect in an indoor warehouse, a fusion scheme of visual synchronous positioning and laser synchronous positioning is selected in the scheme, the problem that a specific object can not be identified in a single positioning method is solved by integrating the two synchronous positioning methods, and the positioning accuracy of the warehousing unmanned aerial vehicle is improved. And the positioning accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of drone navigation technology, and in particular to a positioning and navigation method for indoor warehouse drones. Background Art

[0002] Indoor warehouse drones are drone systems used in logistics and warehousing. Functionally, they include surveillance drones, inventory drones, and transport drones. Based on their movement patterns, they can be equipped with rotors, wheels, or multiple legs. Incorporating drone systems into warehouse systems enables a variety of activities, including material flow, transportation, inventory, and inspection, effectively saving manpower and resources. Inventory drones are specifically designed for inventory operations. In modern warehouses, specialized shelves are often designed for bulk goods. Typical inventory processes involve counting individual items or determining the presence of specific items on corresponding shelves by scanning barcodes and entering data into the warehouse system. Regardless of the inventory method, manual inventory checks on shelves several meters high are time-consuming and require overhead work.

[0003] In this regard, existing technologies have begun to attempt to use rotary-wing drones to replace manual inventory of such scenes.

[0004] For example, patent document CN202210344350.4 provides a drone inventory system, comprising: a shelf with a first color mark on the ground in front of one end and a second color mark on the ground in front of the other end; a shelf information code is set on the ground in front of the shelf near the first color mark; a drone, after identifying the first color mark and the shelf information code, begins to identify the goods on the shelf; after identifying the second color mark, it ends the identification of the goods on the shelf. In this drone inventory system, the drone uses the first color mark and the second color mark to know when entering or leaving the shelf identification area. After entering the shelf identification area and identifying the shelf information code, the drone begins to identify the goods information code; after leaving the shelf identification area, the drone ends the identification, thereby realizing automatic inventory of goods, with high identification accuracy, improved inventory efficiency, reduced inventory error rate, reduced labor costs, and improved warehouse management efficiency.

[0005] For another example, patent document CN202411777928.0 discloses an intelligent dynamic inventory counting algorithm for drones. This invention relates to the field of warehouse management technology. Based on the warehouse layout, shelf distribution, and material storage area information, drone flight inventory routes are planned, and uninventoryed areas are marked. The drone flies to obtain radio frequency data and material image visual data on warehouse materials. The collected radio frequency data and material image visual data are preprocessed and feature extracted to obtain radio frequency features and visual features. This intelligent dynamic inventory counting algorithm for drones enhances inventory accuracy through the coordinated use of radio frequency and visual technologies. Radio frequency identification can accurately read material label information and obtain key data such as material numbers and category codes. The visual recognition system can capture material appearance features from multiple angles and distances to assist in confirming material categories and status. It can efficiently inventory large areas within the warehouse in a short period of time, ensuring smooth warehouse operations.

[0006] However, during actual implementation, the inventors discovered that existing warehouse inventory drones often suffer from inaccurate obstacle recognition. For example, relying solely on visual recognition makes it difficult to identify large obstacles like walls that lack texture. Another example is relying solely on LiDAR, which is prone to positioning drift when targeting continuous structures with similar appearances, such as large numbers of standard containers. These issues ultimately lead to positioning drift and inaccurate obstacle recognition. Summary of the Invention

[0007] In response to the above-mentioned problems existing in the prior art, a method for positioning and navigation of indoor warehouse drones is now provided.

[0008] The specific technical solutions are as follows:

[0009] A method for positioning and navigating an indoor warehouse drone, comprising:

[0010] Step S1: Pre-draw a three-dimensional spatial map for the warehouse to be counted, and collect the locations of the shelves to be counted and add them to the three-dimensional spatial map;

[0011] Step S2: performing path planning for the shelf locations to be counted in the three-dimensional space map to form a counting trajectory;

[0012] Step S3: Using the inventory trajectory to control the inventory drone to move in the warehouse to be inventoryed, and using the inventory drone to inspect the corresponding position of the shelf;

[0013] In the process of controlling the movement of the inventory drone, the Bayesian method is used to fuse the laser synchronous positioning grid and the visual synchronous positioning grid to determine the actual obstacle position for avoidance.

[0014] On the other hand, in step S3, the inventory drone updates the fusion map based on the synchronous positioning process during the movement, and determines the direction of travel at the next moment according to the location of the shelf to be inventoryed;

[0015] The synchronous positioning process includes:

[0016] Step A1: using a laser radar to collect laser point clouds of the surrounding environment, and establishing a real-time laser map based on the laser point clouds; and using a visual sensor to collect image information of the surrounding environment, and establishing a real-time image map based on the image information;

[0017] Step A2: registering the real-time laser map and the real-time image map, and then performing rasterization processing to obtain a laser raster map and an image raster map respectively;

[0018] Step A3: Probabilistically fuse the laser synchronous positioning grid in the laser grid map and the visual synchronous positioning grid in the image grid map using the Bayesian formula, and then eliminate grid units with too low probability to form a fused map.

[0019] On the other hand, the step A2 comprises:

[0020] Step A21: registering the real-time laser map and the real-time image map to form a registered laser map and a registered image map;

[0021] Step A22: performing the same rasterization process on the registered laser map and the registered image map to form the laser raster map and the image raster map;

[0022] Step A23: mapping corresponding grid probabilities to the laser synchronous positioning grid and the visual synchronous positioning grid according to the reconstruction confidence of the real-time laser map and the real-time image map.

[0023] On the other hand, step S3 includes:

[0024] Step S31: obtaining the next shelf location to be counted from the inventory trajectory;

[0025] Step S32: controlling the inventory drone to update the map based on the fusion positioning process, and determining the travel direction in combination with the shelf location to be inventoryed;

[0026] Step S33: Control the movement of the inventory drone based on the travel direction, then return to step S32 until it reaches the shelf location to be counted and then turn to step S34;

[0027] Step S34: perform inventory on the corresponding position of the shelf, and then return to step S31 to obtain the next shelf position to be inventoried, until the inventory trajectory is traversed.

[0028] On the other hand, the step S34 includes:

[0029] Step B341: determining the direction of the corresponding position of the shelf relative to the location of the shelf to be counted based on the fused map;

[0030] Step B342: Controlling the inventory drone to face the corresponding position of the shelf according to the pointing direction, performing laser scanning modeling on the corresponding position of the shelf to determine the plane position of the target shelf, and capturing a pre-scan image of the corresponding position of the shelf;

[0031] Step B343: Identify and obtain the corresponding position of the barcode from the pre-scanned image;

[0032] Step B344: Controlling the inventory drone to align with the shelf plane based on the plane position, and controlling the inventory drone to move according to the corresponding position of the barcode so that the barcode is located at the center of the visual sensor;

[0033] Step B345: Scan the barcode using the visual sensor, and then return to step S31 to obtain the next shelf location to be counted, until the inventory trajectory is traversed.

[0034] On the other hand, the step S34 includes:

[0035] Step C341: determining the direction of the corresponding position of the shelf relative to the location of the shelf to be counted based on the fused map;

[0036] Step C342: Controlling the inventory drone to face the corresponding position of the shelf according to the pointing direction, and performing laser scanning modeling on the corresponding position of the shelf to obtain a stacking volume model;

[0037] Step C343: Estimate the remaining items in the stacking based on the stacking volume modeling, and then return to step S31 to obtain the next shelf point to be counted until the inventory trajectory is traversed.

[0038] On the other hand, the step S1 includes:

[0039] Step S11: deploying the inventory drone in the warehouse to be inventoryed and configuring an initial search direction as a map update direction;

[0040] Step S12: determining whether all spaces in the warehouse to be counted are added to the fusion map;

[0041] If yes, go to step S14;

[0042] If not, perform loop space prediction according to the fused map to obtain the spatial orientation not added to the fused map as the map update direction, and then go to step S13;

[0043] Step S13: updating the fused map based on the synchronous positioning process, and then returning to step S12;

[0044] Step S14: outputting the fused map as the three-dimensional spatial map, and then adding the shelf locations to be counted into the three-dimensional spatial map.

[0045] On the other hand, the step S2 includes:

[0046] Step S21: determining the travel space boundary according to the three-dimensional space map;

[0047] Step S22: searching for adjacent points of the shelves to be counted according to the boundary of the travel space, and constructing node connection lines;

[0048] Step S23: selecting a shelf position to be counted as a starting point, and searching the node connection lines based on the starting point to construct the inventory trajectory.

[0049] The above technical solution has the following advantages or beneficial effects:

[0050] In response to the problem that the existing warehouse drone positioning method is not effective in indoor warehouses, this solution uses a fusion solution of visual synchronous positioning and laser synchronous positioning. By combining the two synchronous positioning methods, the problem that a single positioning method may be unable to identify specific objects is eliminated, thereby improving the positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The embodiments of the present invention will be described more fully with reference to the accompanying drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0052] Figure 1 is an overall schematic diagram of an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the synchronous positioning process in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of step A2 in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of step S1 in an embodiment of the present invention;

[0056] Figure 5 This is a schematic diagram of step S2 in an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of step S3 in an embodiment of the present invention;

[0058] Figure 7 This is a schematic diagram of step B34 in an embodiment of the present invention;

[0059] Figure 8 Schematic diagram of step C34 in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0062] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0063] The present invention comprises:

[0064] A method for positioning and navigating indoor warehouse drones, such as Figure 1 Shown, including:

[0065] Step S1: Pre-draw a three-dimensional spatial map for the warehouse to be counted, and collect the locations of the shelves to be counted and add them to the three-dimensional spatial map;

[0066] Step S2: planning a path for the shelf locations to be counted in the three-dimensional space map to form a counting trajectory;

[0067] Step S3: Using the inventory trajectory to control the inventory drone to move in the warehouse to be inventoryed, and using the inventory drone to check the corresponding position of the shelf;

[0068] In the process of controlling the movement of inventory drones, the Bayesian method is used to fuse the laser synchronous positioning grid and the visual synchronous positioning grid to determine the actual obstacle position for avoidance.

[0069] Specifically, in response to the problem that the existing warehouse drone positioning method is not effective in indoor warehouses, this solution selects a fusion solution of visual synchronous positioning and laser synchronous positioning. By combining the two synchronous positioning methods, the problem that a single positioning method may not be able to identify specific objects is eliminated, thereby improving the positioning accuracy.

[0070] Specifically, during the application process, the above technical solution is mainly configured as a software embodiment in the inventory drone system. The inventory drone system mainly includes an inventory drone and a ground station. The ground station is a computer device that is used to control the movement of the inventory drone and provide a corresponding interactive interface, so that users can configure the parts that need to be inventoryed in the map, the space size of the warehouse to be inventoryed, modify the map, export the inventory results, and other functions.

[0071] The inventory drone is a multi-rotor drone, mainly equipped with a lidar system and a binocular vision sensor system. It can scan and model the surrounding environment independently, and integrate it according to the above-mentioned algorithm, synchronously locate and execute corresponding flight missions.

[0072] The aforementioned inventory-taking drone system is primarily used in indoor warehouses, where multiple large shelves are used to stack goods. Shelf heights typically range from several meters to over ten meters, with spacing between shelves ranging from tens of centimeters to two meters. These shelves may contain loose goods or be stacked in specialized containers. The inventory-taking drone needs to sequentially count multiple storage locations along its trajectory, performing tasks such as modeling, counting, and scanning.

[0073] Since the warehouse is a closed space structure, the traditional industry drone route planning and fixed-point stabilization solutions based on satellite positioning are almost unusable in this scenario due to weak satellite signals. Accordingly, even if an RTK ground differential station is configured, or other wireless positioning methods are introduced, such as UWB ultra-wideband positioning technology, to solve the current position of the drone, it is easy to fail to accurately position the drone due to the characteristics of the shelves themselves. Specifically, when goods of various materials are stacked on shelves more than ten meters high, the communication process between the drone and the positioning base station becomes an extremely complex multipath transmission problem, which directly makes the usual time-of-flight method, received signal strength method and other positioning methods unusable. Even if the corresponding coding and phase solution methods are added, the above problems cannot be effectively solved.

[0074] To solve this problem, the existing technology usually introduces a synchronous positioning method to realize the automatic mapping and positioning process without relying on radio positioning. The synchronous positioning process includes the visual synchronous positioning and mapping process (VSLAM) and the laser radar synchronous positioning and mapping process (LIDAR SLAM). Both use corresponding sensors to scan the surrounding environment, match features, estimate pose, perform loop detection, and build maps to form a complete map and determine the positioning process of the vehicle itself.

[0075] However, relying solely on visual recognition solutions makes it difficult to identify obstacles with large areas and lacking texture features, such as walls. Relying solely on lidar, when targeting continuous structures with similar appearance features such as a large number of standard containers, it is easy to match a large number of similar structures during the loop detection process, making it difficult to accurately determine the actual position of the drone, resulting in positioning drift.

[0076] To address the above problems, this solution proposes a synchronous positioning process that performs map fusion on the visual synchronous positioning process and the lidar synchronous positioning process based on Bayesian probability, thereby improving the positioning robustness.

[0077] like Figure 2 As shown, the synchronous positioning process includes:

[0078] Step A1: using a laser radar to collect laser point clouds of the surrounding environment, and establishing a real-time laser map based on the laser point clouds; and using a visual sensor to collect image information of the surrounding environment, and establishing a real-time image map based on the image information;

[0079] Step A2: registering the real-time laser map and the real-time image map, and then rasterizing them to obtain a laser raster map and an image raster map respectively;

[0080] Step A3: The laser synchronous positioning grid in the laser grid map and the visual synchronous positioning grid in the image grid map are probabilistically fused using the Bayesian formula, and then grid cells with too low probabilities are eliminated to form a fused map.

[0081] Specifically, in order to achieve a better map fusion effect, in this embodiment, the corresponding laser radar synchronous positioning and mapping process is first performed based on the laser radar, and the corresponding visual sensor synchronous positioning and mapping process is performed based on the visual sensor, and then the real-time laser map and real-time image map established at the current moment are obtained respectively.

[0082] Since the lidar and vision sensor themselves have different installation positions on the drone, they need to be aligned and registered so as to be transferred to the same coordinate system. In this coordinate system, the two maps are rasterized in the same way to form a laser raster map and an image raster map.

[0083] Each grid cell is three-dimensional, and each is assigned a corresponding occupancy probability based on the obstacle recognition results during the mapping process. Based on the occupancy probability, a Bayesian formula can be used for probabilistic fusion to fuse the laser synchronous positioning grid in the laser grid map and the visual synchronous positioning grid in the image grid map. Grid cells with too low probabilities are eliminated to reduce the false alarm rate caused by the fusion process, ultimately achieving map fusion.

[0084] After obtaining the fused map, it can be matched based on the pre-built three-dimensional spatial map, and the currently updated fused map can be aligned with the pre-collected and constructed three-dimensional spatial map. Then, the three-dimensional spatial pose information of the drone in the current frame can be read in the fused map to realize the positioning process of the current position of the drone. Then, the aligned three-dimensional spatial map is searched for pre-annotated information such as trajectory and inventory points to determine the direction the drone should go to at the next moment.

[0085] Specifically, the corresponding laser radar synchronous positioning and mapping process based on laser radar includes:

[0086] First, the 3D point cloud data collected by the LiDAR is filtered. Common filtering processes include statistical filtering and inter-frame matching to reduce the corresponding noise data. The historical point cloud data is matched and timestamps are aligned based on the IMU data to eliminate additional noise and blur.

[0087] Then, the point cloud data is combined with the pose information estimated by the UAV's IMU data to perform 3D reconstruction to obtain a point cloud reconstruction model. This point cloud reconstruction model is then matched and spliced ​​with the historical point cloud model established at the previous moment, thereby splicing the current point cloud reconstruction model into the global point cloud model and eliminating the sampling differences of the point cloud.

[0088] During the stitching process, the global point cloud model should also be subjected to loop detection to determine whether the point cloud reconstruction model obtained by reconstructing the current frame has the same point cloud modeling at the same position as the global point cloud model after stitching, so as to determine whether the drone has entered the loop part, and perform local optimization on the repeated modeling part to reduce the corresponding error.

[0089] By repeating the above process, the laser point cloud of each frame of the drone can be reconstructed and spliced ​​to form a complete real-time laser map.

[0090] The process of performing simultaneous positioning and mapping based on the corresponding visual radar based on the visual sensor includes:

[0091] First, the binocular vision image collected by the binocular vision sensor at the current moment is reconstructed, including extracting the corresponding image features in each image, matching the extracted image features to form feature point pairs, and mapping changes according to the external parameter matrix of the binocular vision sensor, thereby realizing the three-dimensional reconstruction process of the binocular vision image of the current frame and obtaining a visual reconstruction model;

[0092] On this basis, the drone's posture information is estimated based on the drone's IMU data, and the drone's moving direction relative to the visual reconstruction model of the previous frame is determined at the image level based on the posture information. The visual reconstruction models of the current frame and the previous frame are matched and spliced ​​according to the corresponding directions to form a global visual reconstruction model.

[0093] During the stitching process, the global visual reconstruction model should also be subjected to loop detection to determine whether the visual reconstruction model obtained by reconstructing the current frame has the same visual image modeling at the same position as the global point cloud model after stitching, so as to determine whether the drone has entered the loop part, and perform local optimization on the repeated modeling part to reduce the corresponding error.

[0094] By repeating the above steps, a real-time image map can be obtained and output.

[0095] In one embodiment, Figure 3 As shown, step A2 includes:

[0096] Step A21: registering the real-time laser map and the real-time image map to form a registered laser map and a registered image map;

[0097] Step A22: performing the same rasterization process on the registered laser map and the registered image map to form a laser raster map and an image raster map;

[0098] Step A23: mapping the grid probabilities corresponding to the laser synchronous positioning grid and the visual synchronous positioning grid according to the reconstruction confidence of the real-time laser map and the real-time image map.

[0099] In order to realize the fusion process of the occupancy grids of the two sets of maps, the real-time laser map and the real-time image map are first aligned and transferred to the same set of coordinate systems.

[0100] Then, the registered laser map and the registered image map are subjected to the same rasterization process based on the same coordinate origin and step size to form a laser raster map and an image raster map.

[0101] During the pre-reconstruction process of the real-time laser map and the real-time image map, each reconstructed key point is marked with a confidence level.

[0102] When constructing a real-time laser map, laser point clouds at the edges of the map, where reflection intensity is low, are assigned a lower confidence level. The confidence level of each reflection point is assigned based on the reflection intensity. During the stitching process, the confidence levels of repeated point clouds and those within a certain proximity are averaged. In subsequent map reconstruction and loop detection processes, the confidence levels of the vertices around each reconstructed mesh surface are stitched together to form the confidence level for the corresponding mesh surface, which serves as the reconstruction confidence level for the real-time laser map.

[0103] During the construction of the real-time image map, for each frame of the visual 3D reconstructed image, nonlinearly decreasing confidence parameters are assigned in radial directions from the image center point. During the stitching process, the confidence values ​​of the image regions corresponding to the matching feature point pairs are averaged. In subsequent map reconstruction and loop closure detection processes, the confidence values ​​of the vertices around each reconstructed mesh surface are stitched together to form the confidence values ​​on the corresponding mesh surfaces, which serve as the reconstruction confidence values ​​for the real-time image map.

[0104] Based on the reconstruction confidence building process described above, after the registered laser map and the registered image map are rasterized in the same way, several grid surfaces are defined for each grid cell in the registered laser map and the registered image map. The reconstruction confidence of each grid surface is averaged in each grid cell to obtain the reconstruction confidence of the grid cell as a whole. Based on this reconstruction confidence, the grid probabilities corresponding to the laser synchronous positioning grid and the visual synchronous positioning grid mapping are converted.

[0105] After obtaining the corresponding grid probabilities, for each laser synchronous positioning grid in the laser grid map and the visual synchronous positioning grid in the image grid map, the grid cell probability of the corresponding grid cell in the previous fused map is used as the map prior probability. Then, based on the Bayesian formula, the joint probability is calculated for the grid cell at the current moment, and the updated grid cell probability is finally obtained. Obstacles and other objects in the map are updated according to the grid cell probabilities to obtain the current fused map and output it.

[0106] In one embodiment, Figure 4 As shown, step S1 includes:

[0107] Step S11: deploying an inventory drone in the warehouse to be inventoryed and configuring the initial search direction as the map update direction;

[0108] Step S12: Determine whether all spaces in the warehouse to be counted are added to the fusion map;

[0109] If yes, go to step S14;

[0110] If not, perform loop space prediction based on the fused map to obtain the spatial orientation not added to the fused map as the map update direction, and then go to step S13;

[0111] Step S13: updating the fused map based on the synchronous positioning process, and then returning to step S12;

[0112] Step S14: Output the fused map as a three-dimensional spatial map, and then add the location of the shelf to be counted into the three-dimensional spatial map.

[0113] Specifically, based on the aforementioned synchronous positioning process, this embodiment can implement mapping of the warehouse to be inventoried. Specifically, an inventory drone is first deployed to the warehouse to be inventoried, and an initial search direction is configured as the map update direction. The drone moves along this initial search direction, triggering the automatic mapping and stitching process during the synchronous positioning process. The map update direction consists of two parts: the search direction in three-dimensional space, and the spatial distance moved in that search direction during a single mapping process.

[0114] During a single map-building process, the drone primarily perceives and integrates the surrounding environment based on the previous synchronous positioning process, thereby updating the drone's self-built map. After completing a single synchronous positioning process, the drone will splice a certain length of map along the map update direction.

[0115] At this point, we first determine whether the fused map reaches the area set for the warehouse to be inventoried. If not, a further update process is required. This update process relies on the loop detection process of synchronous positioning. By detecting the parts of the map where loops have appeared, we can filter out directions with a lower probability of loops as new map update directions.

[0116] The above process is repeated to build a map of the entire warehouse to be counted. Manual annotations are then added to mark the shelves that need to be counted in the subsequent inventory process, as well as the usual form of stacking goods on the shelves, such as inventory by scanning codes, inventory by counting piles, etc.

[0117] In one embodiment, Figure 5 As shown, step S2 includes:

[0118] Step S21: determining the boundary of the travel space according to the three-dimensional space map;

[0119] Step S22: searching for adjacent points at the locations of the shelves to be counted according to the boundary of the travel space, and constructing node connection lines;

[0120] Step S23: Select a shelf location to be counted as a starting point, and search for node connection lines based on the starting point to construct an inventory trajectory.

[0121] Specifically, in order to realize the construction process of the inventory trajectory, in this embodiment, after completing the mapping work of the warehouse to be inventoried, the established three-dimensional space map is first read, mainly reading the boundary part of the object identified in the map, projecting the boundary part, and extracting the vertex to construct the plane, eliminating the concave and convex planes and gaps that appear in the modeling, etc., and corresponding safety rules may also be configured as needed. For example, some areas are not suitable for drones to pass through, thereby constructing the travel space boundary.

[0122] Since the locations of the shelves to be counted have been marked after the map is built, on the basis of constructing the travel space boundary, random lines can be connected for multiple shelves to be counted, and distances can be calculated to determine adjacent points. In the process of connecting lines, the travel space boundary is added as a constraint, and adjacent points are connected to form node connection lines.

[0123] Finally, a shelf location to be counted is selected as the starting point, and the node connection line is searched based on the starting point. The search algorithm can be implemented based on a greedy algorithm or local optimality to construct the inventory trajectory.

[0124] Based on the above synchronous positioning process, in step S3, the inventory drone can update the fusion map based on the synchronous positioning process during the movement process, and determine the direction of travel at the next moment according to the determined shelf point to be inventoryed;

[0125] Specifically, if Figure 6 As shown, step S3 includes:

[0126] Step S31: Obtain the next shelf location to be counted from the inventory trajectory;

[0127] Step S32: Based on the synchronous positioning process, the inventory counting drone is controlled to update the map and determine the direction of travel in combination with the location of the shelf to be counted;

[0128] Step S33: Control the movement of the inventory drone based on the direction of travel, then return to step S32 until it reaches the shelf to be counted and then go to step S34;

[0129] Step S34: perform inventory on the corresponding position of the shelf, and then return to step S31 to obtain the next shelf point to be inventoried until the inventory track is traversed.

[0130] Specifically, in order to achieve a more accurate positioning process during the inventory process, in this embodiment, after constructing the inventory trajectory, the shelf locations to be inventoried are selected in turn, and then the inventory drone is controlled to update the map based on the synchronous positioning process, and a new fused map is obtained and aligned and matched with the pre-built three-dimensional space map to realize the positioning process of the drone.

[0131] Then, the positioning results of the drone are transferred to the three-dimensional space map and the direction of travel is determined in combination with the shelf point to be counted. The process is repeated to realize the positioning and navigation process of the drone until it reaches the shelf point to be counted.

[0132] In one embodiment, Figure 7 As shown, step S34 includes:

[0133] Step B341: Determine the direction of the corresponding position of the shelf relative to the location of the shelf to be counted based on the fused map;

[0134] Step B342: Control the inventory drone to face the corresponding position of the shelf according to the pointing direction, perform laser scanning modeling on the corresponding position of the shelf, determine the plane position of the target shelf, and capture a pre-scan image of the corresponding position of the shelf;

[0135] Step B343: Identify and obtain the corresponding position of the barcode from the pre-scanned image;

[0136] Step B344: Aligning the inventory drone with the shelf plane based on the plane position, and controlling the inventory drone to move according to the corresponding position of the barcode so that the barcode is located at the center of the visual sensor;

[0137] Step B345: Use a visual sensor to scan the barcode, and then return to step S31 to obtain the next shelf location to be counted until the inventory track is traversed.

[0138] Specifically, to achieve optimal inventory counting results for barcoded goods, this embodiment first calculates the drone's pose upon arrival at the shelf to be counted, and uses the fused map to determine the shelf's orientation. The drone is then directed toward the shelf's corresponding location based on this orientation, and laser scanning is performed to determine the target shelf's planar position. A pre-scan image of the shelf's corresponding location is also captured.

[0139] Taking into account the influence of the stacking position of the goods and the position of the barcode affixed, the process of shooting the barcode facing the shelf may introduce large perspective and distortion problems. In this solution, the orientation of the barcode and the posture compensation parameters of the drone are determined based on the process of laser scanning modeling and image recognition. The inventory drone is aligned with the shelf plane based on the plane position control, and the inventory drone is controlled to move according to the corresponding position of the barcode, so that the barcode is located in the center of the visual sensor, thereby improving the accuracy of barcode recognition.

[0140] In one embodiment, Figure 8 As shown, step S34 includes:

[0141] Step C341: determining the direction of the corresponding position of the shelf relative to the location of the shelf to be counted based on the fused map;

[0142] Step C342: Control the inventory drone to face the corresponding position of the shelf according to the pointing direction, and perform laser scanning modeling on the corresponding position of the shelf to obtain stacking volume modeling;

[0143] Step C343: Estimate the remaining items in the stack based on the stacking volume modeling, and then return to step S31 to obtain the next shelf point to be counted until the inventory trajectory is traversed.

[0144] Specifically, in order to achieve effective statistics of stacked goods, in this embodiment, after determining the pointing direction of the corresponding position of the shelf relative to the shelf point to be counted based on the fusion map, the inventory drone is first controlled to face the corresponding position of the shelf according to the pointing direction.

[0145] Next, laser scanning is performed on the corresponding shelf locations to generate a stacking volume model. For goods whose units are cubic, the cubic output is generated directly from the volume modeling results. For other goods, the volume of each piece is estimated, and the quantity is output after inventory counting.

[0146] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for positioning and navigating an indoor warehouse drone, characterized in that: include: Step S1: Pre-draw a three-dimensional spatial map for the warehouse to be counted, and collect the locations of the shelves to be counted and add them to the three-dimensional spatial map; Step S2: performing path planning for the shelf locations to be counted in the three-dimensional space map to form a counting trajectory; Step S3: Using the inventory trajectory to control the inventory drone to move in the warehouse to be inventoryed, and using the inventory drone to inspect the corresponding position of the shelf; In the process of controlling the movement of the inventory drone, the Bayesian method is used to fuse the laser synchronous positioning grid and the visual synchronous positioning grid to determine the actual obstacle position for avoidance.

2. The indoor warehouse drone positioning and navigation method according to claim 1, characterized in that: In step S3, the inventory drone updates the fusion map based on the synchronous positioning process during the movement, and determines the direction of travel at the next moment according to the location of the shelf to be inventoryed; The synchronous positioning process includes: Step A1: using a laser radar to collect laser point clouds of the surrounding environment, and establishing a real-time laser map based on the laser point clouds; and using a visual sensor to collect image information of the surrounding environment, and establishing a real-time image map based on the image information; Step A2: registering the real-time laser map and the real-time image map, and then performing rasterization processing to obtain a laser raster map and an image raster map respectively; Step A3: Probabilistically fuse the laser synchronous positioning grid in the laser grid map and the visual synchronous positioning grid in the image grid map using the Bayesian formula, and then eliminate grid units with too low probability to form a fused map.

3. The indoor warehouse drone positioning and navigation method according to claim 2, characterized in that: The step A2 comprises: Step A21: registering the real-time laser map and the real-time image map to form a registered laser map and a registered image map; Step A22: performing the same rasterization process on the registered laser map and the registered image map to form the laser raster map and the image raster map; Step A23: mapping corresponding grid probabilities to the laser synchronous positioning grid and the visual synchronous positioning grid according to the reconstruction confidence of the real-time laser map and the real-time image map.

4. The indoor warehouse drone positioning and navigation method according to claim 2, characterized in that: The step S3 comprises: Step S31: obtaining the next shelf location to be counted from the inventory trajectory; Step S32: controlling the inventory drone to update the map based on the fusion positioning process, and determining the travel direction in combination with the shelf location to be inventoryed; Step S33: Control the movement of the inventory drone based on the travel direction, then return to step S32 until it reaches the shelf location to be counted and then turn to step S34; Step S34: perform inventory on the corresponding position of the shelf, and then return to step S31 to obtain the next shelf position to be inventoried, until the inventory trajectory is traversed.

5. The indoor warehouse drone positioning and navigation method according to claim 4, characterized in that: The step S34 includes: Step B341: determining the direction of the corresponding position of the shelf relative to the location of the shelf to be counted based on the fused map; Step B342: Controlling the inventory drone to face the corresponding position of the shelf according to the pointing direction, performing laser scanning modeling on the corresponding position of the shelf to determine the plane position of the target shelf, and capturing a pre-scan image of the corresponding position of the shelf; Step B343: Identify and obtain the corresponding position of the barcode from the pre-scanned image; Step B344: Controlling the inventory drone to align with the shelf plane based on the plane position, and controlling the inventory drone to move according to the corresponding position of the barcode so that the barcode is located at the center of the visual sensor; Step B345: Scan the barcode using the visual sensor, and then return to step S31 to obtain the next shelf location to be counted, until the inventory trajectory is traversed.

6. The indoor warehouse drone positioning and navigation method according to claim 4, characterized in that: The step S34 includes: Step C341: determining the direction of the corresponding position of the shelf relative to the location of the shelf to be counted based on the fused map; Step C342: Controlling the inventory drone to face the corresponding position of the shelf according to the pointing direction, and performing laser scanning modeling on the corresponding position of the shelf to obtain a stacking volume model; Step C343: Estimate the remaining items in the stacking based on the stacking volume modeling, and then return to step S31 to obtain the next shelf point to be counted until the inventory trajectory is traversed.

7. The indoor warehouse drone positioning and navigation method according to claim 2, characterized in that: The step S1 comprises: Step S11: deploying the inventory drone in the warehouse to be inventoryed and configuring an initial search direction as a map update direction; Step S12: determining whether all spaces in the warehouse to be counted are added to the fusion map; If yes, go to step S14; If not, perform loop space prediction according to the fused map to obtain the spatial orientation not added to the fused map as the map update direction, and then go to step S13; Step S13: updating the fused map based on the synchronous positioning process, and then returning to step S12; Step S14: outputting the fused map as the three-dimensional spatial map, and then adding the shelf locations to be counted into the three-dimensional spatial map.

8. The indoor warehouse drone positioning and navigation method according to claim 1, characterized in that: The step S2 comprises: Step S21: determining the travel space boundary according to the three-dimensional space map; Step S22: searching for adjacent points of the shelves to be counted according to the boundary of the travel space, and constructing node connection lines; Step S23: selecting a shelf position to be counted as a starting point, and searching the node connection lines based on the starting point to construct the inventory trajectory.

Images