Multi-agent collaborative guided positioning method and device

Abstract

The present invention proposes a guided positioning method and device for multi-agent collaboration. The multi-agent includes a portable terminal and a mobile platform. The method includes: aligning the pose coordinate systems of the portable terminal and the mobile platform; in real time, calculating the first six-degree-of-freedom pose of the portable terminal when it moves to the target point in the pose coordinate system to generate a local map of the path; the mobile platform is deployed on the mobile device, and based on the local map of the path, the mobile platform calculates the optimal planned path of the mobile device in the pose coordinate system, and calculates the second six-degree-of-freedom pose corresponding to the optimal planned path in real time to generate the vertices of the optimal planned path; the local map of the path and the vertices are compared and optimized to obtain the target movement trajectory of the mobile device to the target point, and guide the movement to the target point. As a result, the trajectory of the mobile platform is aligned with the trajectory of the portable terminal in real time, and the target movement trajectory is dynamically updated, improving the accuracy and efficiency of positioning.

Description

Technical Field

[0001] The present invention relates to the field of collaborative positioning technology, and in particular to a multi-agent collaborative guided positioning method, device, electronic device and storage medium. Background Art

[0002] Navigation and positioning issues in mobile vehicles, collaborative robots, and indoor scenarios. Existing positioning technologies primarily rely on independent positioning systems within individual devices, such as the Global Positioning System (GPS), inertial navigation system (INS), and visual-inertial odometry (VIO). These independent positioning systems perform well in a single environment, but are prone to cumulative positioning errors in complex environments or when moving over large areas, resulting in low positioning accuracy. Summary of the Invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0004] To this end, the first purpose of the present invention is to propose a multi-agent collaborative guided positioning method, in which the mobile platform trajectory is aligned to the portable terminal trajectory in real time, and the target movement trajectory is dynamically updated to improve the accuracy and efficiency of positioning.

[0005] The second object of the present invention is to provide a multi-agent collaborative guided positioning device.

[0006] A third object of the present invention is to provide an electronic device.

[0007] A fourth object of the present invention is to provide a non-transitory computer-readable storage medium storing computer instructions.

[0008] To achieve the above objectives, a first embodiment of the present invention provides a multi-agent collaborative guided positioning method, wherein the multi-agent includes a portable terminal and a mobile platform, and the method includes:

[0009] Aligning the pose coordinate systems of the portable terminal and the mobile platform;

[0010] In the pose coordinate system, a first six-degree-of-freedom pose of the portable terminal when it moves to a target point is calculated in real time by fusing a visual sensor and an inertial measurement unit's odometer, and a local map of the path of the mobile terminal when it moves is generated based on the first six-degree-of-freedom pose. The local map of the path includes visual feature points, descriptors, and landmarks corresponding to the movement of the portable terminal, and is sent to the mobile platform.

[0011] The mobile platform is deployed on the mobile device. The mobile platform calculates the optimal planned path of the mobile device in the pose coordinate system based on the local map of the path through a path planning algorithm, and calculates in real time the second six-degree-of-freedom pose of the mobile device moving to the target point based on the optimal planned path by fusing a visual sensor and an inertial measurement unit's odometer. Based on the second six-degree-of-freedom pose, the mobile platform generates vertices of the optimal planned path, wherein the vertices include visual feature points, descriptors, and landmarks corresponding to the movement of the mobile device based on the optimal planned path.

[0012] The local map and vertices of the path are compared and optimized by a service module deployed on the mobile platform to obtain a target movement trajectory of the mobile device to the target point, and guide the mobile device to move to the target point based on the target movement trajectory.

[0013] To achieve the above-mentioned objectives, a second embodiment of the present invention provides a multi-agent collaborative guided positioning device, wherein the multi-agent includes a portable terminal and a mobile platform, and the device includes:

[0014] A comparison module, used to align the pose coordinate systems of the portable terminal and the mobile platform;

[0015] a first calculation module, configured to calculate, in the pose coordinate system, a first six-degree-of-freedom pose of the portable terminal when it moves to a target point in real time by fusing a visual sensor and an odometer of an inertial measurement unit, and generate a local map of the path of the mobile terminal when it moves based on the first six-degree-of-freedom pose, and send the map to the mobile platform, wherein the local map of the path includes visual feature points, descriptors, and landmarks corresponding to the movement of the portable terminal;

[0016] A second computing module is configured to deploy the mobile platform on a mobile device, wherein the mobile platform calculates an optimal planned path for the mobile device in the pose coordinate system based on the local map of the path using a path planning algorithm, and calculates in real time a second six-degree-of-freedom pose of the mobile device moving to a target point based on the optimal planned path by integrating a visual sensor and an inertial measurement unit's odometer, and generates vertices of the optimal planned path based on the second six-degree-of-freedom pose, wherein the vertices include visual feature points, descriptors, and landmarks corresponding to the movement of the mobile device based on the optimal planned path;

[0017] The guidance module is used to compare and optimize the local map and vertices of the path through the service module deployed on the mobile platform to obtain the target movement trajectory of the mobile device to the target point, and guide the mobile device to move to the target point based on the target movement trajectory.

[0018] To achieve the above-mentioned purpose, the third aspect embodiment of the present invention proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.

[0019] In order to achieve the above-mentioned objectives, an embodiment of the fourth aspect of the present invention proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0020] The multi-agent collaborative guided positioning method, device, electronic device and storage medium provided by the embodiments of the present invention are applied to a portable terminal and a mobile platform to align the pose coordinate systems of the portable terminal and the mobile platform; in the pose coordinate system, the first six-degree-of-freedom pose of the portable terminal when it moves to the target point is calculated in real time to generate a local map of the path; the mobile platform is deployed on the mobile device, and the mobile platform calculates the optimal planned path of the mobile device in the pose coordinate system based on the local map of the path, and calculates the second six-degree-of-freedom pose corresponding to the optimal planned path in real time to generate the vertices of the optimal planned path; the local map of the path and the vertices are compared and optimized to obtain the target movement trajectory of the mobile device to the target point, and guide it to move to the target point. As a result, the trajectory of the mobile platform is aligned with the trajectory of the portable terminal in real time, and the target movement trajectory is dynamically updated, thereby improving the accuracy and efficiency of positioning.

[0021] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A schematic diagram of a flow chart of a multi-agent collaborative guided positioning method provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of a multi-agent collaborative positioning trajectory provided by an embodiment of the present invention;

[0025] Figure 3 This is an application flow chart of another multi-agent collaborative guided positioning method provided by an embodiment of the present invention;

[0026] Figure 4 A schematic structural diagram of a multi-agent collaborative guided positioning device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0028] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of the present invention comply with the relevant provisions of relevant laws and regulations.

[0029] The following describes the multi-agent collaborative guided positioning method, device, electronic device and storage medium of the embodiments of the present invention with reference to the accompanying drawings.

[0030] Figure 1 A flowchart of a multi-agent collaborative guided positioning method provided by an embodiment of the present invention.

[0031] like Figure 1 As shown, the multi-agent includes a portable terminal and a mobile platform, and the method includes the following steps:

[0032] Step 101: Align the pose coordinate systems of the portable terminal and the mobile platform.

[0033] In some possible implementations, the pose coordinate systems of the portable terminal and the mobile platform can be aligned using visual tags corresponding to the visual sensor or measurement data from an inertial measurement unit (IMU) to resolve the issue of heterogeneous device benchmark differences.

[0034] The visual sensor may be composed of one or two image sensors, and the inertial measurement unit may be a device for measuring the three-axis attitude angle (or angular rate) and acceleration of an object.

[0035] In other possible implementations, the pose coordinate systems of the portable terminal and the mobile platform may be aligned through ultra-wideband (UWB) positioning or lidar calibration to improve the alignment accuracy of the pose coordinate systems.

[0036] Optionally, the portable terminal may be a positioning pen, but is not limited thereto.

[0037] Optionally, the mobile platform may be a mobile vehicle or a service platform of a collaborative robot, but is not limited thereto.

[0038] Step 102: In the pose coordinate system, the first six-degree-of-freedom pose of the portable terminal when it moves to the target point is calculated in real time by fusing the visual sensor and the odometer of the inertial measurement unit. Based on the first six-degree-of-freedom pose, a local map of the path of the mobile terminal when it moves is generated and sent to the mobile platform. The local map of the path includes visual feature points, descriptors, and landmarks corresponding to the movement of the portable terminal.

[0039] Among them, the six-degree-of-freedom posture corresponds to the six degrees of freedom of an object in space, namely the degrees of freedom of movement along the three rectangular coordinate axes x, y, and z and the degrees of freedom of rotation around these three coordinate axes.

[0040] Visual feature points are unique and stable points in an image. These points remain unchanged when the camera's perspective changes and are often used for camera pose estimation. Feature points can be corners, edges, or blocks, among which corners are the easiest to identify and are often used in visual SLAM.

[0041] A descriptor is a data structure associated with a visual feature point, used to describe the appearance and attributes of the feature point. In a SLAM system, descriptors are used to match visual feature points in a new frame with feature points in a map. Each visual feature point has one or more descriptors, and matching visual feature points is achieved by calculating the distance or similarity between descriptors.

[0042] In SLAM systems, landmarks usually refer to feature points in an image. These feature points are sparsely represented in the map and used to construct a sparse map. Landmarks are not only used for positioning and navigation, but also participate in tasks such as path planning and obstacle avoidance.

[0043] A submap is a map built within a certain area and is commonly used in real-time SLAM systems. A submap contains keyframes and map points within that area and is used for real-time positioning and navigation. The construction and maintenance of submaps is crucial to improving the real-time performance and accuracy of SLAM systems.

[0044] In step 103, the mobile platform is deployed on the mobile device. Based on the local map of the path, the mobile platform calculates the optimal planned path of the mobile device in the pose coordinate system through a path planning algorithm, and calculates the second six-degree-of-freedom pose of the mobile device moving to the target point based on the optimal planned path in real time by integrating the visual sensor and the odometry of the inertial measurement unit. Based on the second six-degree-of-freedom pose, the vertices of the optimal planned path are generated. The vertices include visual feature points, descriptors, and landmark points corresponding to the movement of the mobile device based on the optimal planned path.

[0045] Optionally, the path planning algorithm is a computational method for determining the optimal path from a starting point to a target point, including but not limited to traditional graph search algorithms, sampling-based algorithms, and intelligent bionic algorithms.

[0046] A vertex, which contains visual feature points, descriptors, and landmarks, is a node in the SLAM system that represents a landmark in the map. It contains the landmark's 3D position, associated visual feature points, and descriptors. It is optimized along with other nodes (such as the visual sensor pose) through graph optimization to achieve accurate positioning and map construction.

[0047] In some possible implementations, after the mobile device's optimal planned path is calculated in real time using a fusion of visual sensors and an inertial measurement unit's odometry, a controller within the mobile platform can adjust the mobile device's speed and steering angle in real time to guide it to the target point along the optimal planned path. The mobile platform publishes Vertex data at a high frequency, supporting millisecond-level loop detection and global optimization.

[0048] In step 104 , the service module deployed on the mobile platform compares and optimizes the path local map and vertices to obtain a target movement trajectory of the mobile device to the target point, and guides the mobile device to move to the target point based on the target movement trajectory.

[0049] In some possible implementations, a multi-threaded service in a service module deployed on the mobile platform performs local graph optimization to correct the local path map, and a loop detection database is constructed based on the corrected local path map. A bag-of-words (BoW) model installed on the service module performs loop detection on vertices to obtain the target path corresponding to the vertex. A sliding window method is used to extract the real-time local path map corresponding to the target path from the loop detection database. The target path is then compared and optimized based on the real-time local path map to obtain a target trajectory for the mobile device to move to the target point, and the mobile device is guided to move to the target point based on the target trajectory. Through a hierarchical process of submap local optimization and vertex global optimization, the computational load is reduced, enabling the mobile platform to respond to changes in the portable terminal's target point in real time and adapt to complex scenarios.

[0050] In summary, the schematic diagram of multi-agent collaborative positioning trajectory is as follows Figure 2 As shown in the figure, the multi-agent collaborative positioning trajectory includes a portable terminal, a mobile platform, a portable path (local path map), a mobile path (vertices) and a target point location.

[0051] The multi-agent collaborative guided positioning method of the embodiment of the present invention uses a preset positioning pen module to digitally describe the position, shape, and posture information of the object to be manipulated by the robot. The digital object data is used as the input of the deep neural network of the robot's corresponding simulation platform, and the manipulation position of the robot-controlled object is used as the output to train the robot's initial position planning model. Based on the application scenario of the object, the corresponding observation space and task feedback logic are selected to optimize the initial position planning model and obtain the robot position planning model. The robot position planning model predicts the digital object data of the object to be manipulated and obtains the target position of the object to be manipulated by the robot. Therefore, based on the trained robot position planning model, the robot planning accuracy meets the implementation requirements of various industrial scenarios and greatly reduces the workload of manual operation.

[0052] In order to clearly illustrate the above embodiment, Figure 3 This is an application flowchart of a multi-agent collaborative guided positioning method provided by an embodiment of the present invention. Specifically, when the portable terminal is a positioning pen and the mobile platform is a service platform for a mobile vehicle, a local map (Submap_n, ..., Submap_0) of the path of the positioning pen during movement is collected in real time and sent to a service module deployed on the service platform. A multi-threaded service in the service module performs local map optimization to correct the local map of the path. A loop detection database (Submap_0, Submap_1, ..., Submap_n) is constructed based on the corrected local map of the path. Simultaneously, an optimal planned path for the mobile vehicle is calculated based on the local map of the path, and vertices (Vertex_n, ..., Vertex_0) of the optimal planned path are generated. A bag-of-words model installed on the service module performs loop detection on the vertices to obtain the target path corresponding to the vertices. A sliding window method is used to extract the real-time local map of the path corresponding to the target path from the loop detection database. The target path is then compared and optimized based on the real-time local map of the path to obtain a target trajectory for the mobile vehicle to reach the target point. The mobile vehicle is then guided to move to the target point based on the target trajectory. Therefore, through the dynamic collection of portable terminals and the optimization of paths by mobile platforms, the amount of redundant vertex data can be reduced and the positioning efficiency can be improved.

[0053] In addition, when communicating between the vehicle-side server, pen-side, and vehicle-side, if the wireless bandwidth is limited, compressed transmission can be used, including but not limited to the data structure serialization and deserialization framework ProtoBuf to compress Submap data.

[0054] In order to implement the above embodiment, the present invention also proposes a multi-agent collaborative guided positioning device.

[0055] Figure 4A schematic structural diagram of a multi-agent collaborative guided positioning device provided in an embodiment of the present invention.

[0056] like Figure 4 As shown, the multi-agent includes a portable terminal and a mobile platform, and the multi-agent collaborative guided positioning device 40 includes: a comparison module 41, a first calculation module 42, a second calculation module 43, and a guidance module 44.

[0057] A comparison module 41 is used to align the pose coordinate systems of the portable terminal and the mobile platform;

[0058] a first calculation module 42 configured to calculate, in the pose coordinate system, a first six-degree-of-freedom pose of the portable terminal when it moves to a target point in real time by fusing a visual sensor and an inertial measurement unit's odometer, and generate a local map of the path along which the mobile terminal moves based on the first six-degree-of-freedom pose, and transmit the map to the mobile platform, the local map comprising visual feature points, descriptors, and landmarks corresponding to the movement of the portable terminal;

[0059] A second computing module 43 is configured to deploy the mobile platform on the mobile device. The mobile platform calculates the optimal planned path of the mobile device in the pose coordinate system based on the local path map using a path planning algorithm. The mobile platform also calculates in real time a second six-degree-of-freedom pose of the mobile device moving to the target point based on the optimal planned path by integrating a visual sensor with an inertial measurement unit's odometer. The mobile platform also generates vertices of the optimal planned path based on the second six-degree-of-freedom pose. The vertices include visual feature points, descriptors, and landmarks corresponding to the movement of the mobile device based on the optimal planned path.

[0060] The guidance module 44 is used to compare and optimize the path local map and vertices through the service module deployed on the mobile platform to obtain the target movement trajectory of the mobile device to the target point, and guide the mobile device to move to the target point based on the target movement trajectory.

[0061] Furthermore, in a possible implementation of the embodiment of the present invention, the comparison module 41 is specifically configured to:

[0062] The pose coordinate systems of the portable terminal and the mobile platform are aligned using the visual tags corresponding to the visual sensor or the measurement data of the inertial measurement unit.

[0063] Furthermore, in a possible implementation of the embodiment of the present invention, the comparison module 41 is further specifically configured to:

[0064] The pose coordinate systems of the portable terminal and the mobile platform are aligned through ultra-wideband positioning or lidar calibration.

[0065] Furthermore, in a possible implementation of the embodiment of the present invention, the guiding module 44 is specifically configured to:

[0066] Perform local graph optimization through the multi-threaded service in the service module deployed on the mobile platform to complete the path local map correction, and build a loop detection database based on the corrected path local map;

[0067] Performing loop closure detection on the vertex using a bag-of-words model installed on a service module deployed on a mobile platform to obtain a target path corresponding to the vertex;

[0068] Extracting a real-time path local map corresponding to the target path from the loop detection database using a sliding window method;

[0069] The target path is optimized based on the comparison of the real-time path local map to obtain a target movement trajectory of the mobile device to the target point, and the mobile device is guided to move to the target point based on the target movement trajectory.

[0070] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment and will not be repeated here.

[0071] The multi-agent collaborative guided positioning device of the embodiment of the present invention comprises a portable terminal and a mobile platform, which align the pose coordinate systems of the portable terminal and the mobile platform; in the pose coordinate system, the first six-degree-of-freedom pose of the portable terminal when it moves to the target point is calculated in real time to generate a local map of the path; the mobile platform is deployed on the mobile device, and the mobile platform calculates the optimal planned path of the mobile device in the pose coordinate system based on the local map of the path, and calculates the second six-degree-of-freedom pose corresponding to the optimal planned path in real time to generate the vertices of the optimal planned path; the local map of the path and the vertices are compared and optimized to obtain the target movement trajectory of the mobile device to the target point, and guide the movement to the target point. As a result, the trajectory of the mobile platform is aligned to the trajectory of the portable terminal in real time, and the target movement trajectory is dynamically updated to improve the accuracy and efficiency of positioning.

[0072] In order to implement the above embodiment, the present invention further provides an electronic device, including:

[0073] at least one processor; and

[0074] a memory communicatively connected to the at least one processor; wherein,

[0075] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the aforementioned method.

[0076] In order to implement the above embodiment, the present invention further proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the above method.

[0077] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0078] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0079] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0080] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" is any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (not exhaustive) of computer-readable media include: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0081] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any of the following technologies known in the art, or a combination thereof, may be used: a discrete logic circuit having logic gates for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gates, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0082] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0083] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0084] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and are not to be construed as limiting the present invention. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A multi-agent collaborative guided positioning method, characterized in that: The multi-agent includes a portable terminal and a mobile platform, and the method includes: Aligning the pose coordinate systems of the portable terminal and the mobile platform; In the pose coordinate system, a first six-degree-of-freedom pose of the portable terminal when it moves to a target point is calculated in real time by fusing a visual sensor and an inertial measurement unit's odometer, and a local map of the path of the mobile terminal when it moves is generated based on the first six-degree-of-freedom pose. The local map of the path includes visual feature points, descriptors, and landmarks corresponding to the movement of the portable terminal, and is sent to the mobile platform. The mobile platform is deployed on the mobile device. Based on the local map of the path, the mobile platform calculates the optimal planned path of the mobile device in the pose coordinate system through a path planning algorithm, and calculates in real time the second six-degree-of-freedom pose of the mobile device moving to the target point based on the optimal planned path by fusing a visual sensor and an inertial measurement unit's odometer. Based on the second six-degree-of-freedom pose, the mobile platform generates vertices of the optimal planned path. The vertices include visual feature points and landmark points corresponding to the movement of the mobile device based on the optimal planned path. The visual feature points have one or more descriptors. The descriptor is a data structure related to the visual feature points and is used to describe the appearance and attributes of the feature points. The local map and vertices of the path are compared and optimized by a service module deployed on the mobile platform to obtain a target movement trajectory of the mobile device to the target point, and guide the mobile device to move to the target point based on the target movement trajectory.

2. The method according to claim 1, characterized in that Align the pose coordinate systems of the portable terminal and the mobile platform, including: The pose coordinate systems of the portable terminal and the mobile platform are aligned using the visual tags corresponding to the visual sensor or the measurement data of the inertial measurement unit.

3. The method according to claim 2, characterized in that The method further comprises: The pose coordinate systems of the portable terminal and the mobile platform are aligned through ultra-wideband positioning or lidar calibration.

4. The method according to claim 1, wherein The service module deployed on the mobile platform compares and optimizes the path local map and vertices to obtain a target movement trajectory of the mobile device to the target point, and guides the mobile device to move to the target point based on the target movement trajectory, including: Perform local graph optimization through the multi-threaded service in the service module deployed on the mobile platform to complete the path local map correction, and build a loop detection database based on the corrected path local map; Performing loop closure detection on the vertex using a bag-of-words model installed on a service module deployed on a mobile platform to obtain a target path corresponding to the vertex; Extracting a real-time path local map corresponding to the target path from the loop detection database using a sliding window method; The target path is optimized based on the comparison of the real-time path local map to obtain a target movement trajectory of the mobile device to the target point, and the mobile device is guided to move to the target point based on the target movement trajectory.

5. A multi-agent collaborative guided positioning device, characterized in that: The multi-agent includes a portable terminal and a mobile platform, and the device includes: A comparison module, used to align the pose coordinate systems of the portable terminal and the mobile platform; a first calculation module, configured to calculate, in the pose coordinate system, a first six-degree-of-freedom pose of the portable terminal when it moves to a target point in real time by fusing a visual sensor and an odometer of an inertial measurement unit, and generate a local map of the path of the mobile terminal when it moves based on the first six-degree-of-freedom pose, and send the map to the mobile platform, wherein the local map of the path includes visual feature points, descriptors, and landmarks corresponding to the movement of the portable terminal; A second computing module is configured to deploy the mobile platform on a mobile device, wherein the mobile platform calculates an optimal planned path for the mobile device in the pose coordinate system based on the local map of the path through a path planning algorithm, and calculates in real time a second six-degree-of-freedom pose of the mobile device moving to a target point based on the optimal planned path by fusing a visual sensor with an inertial measurement unit's odometer, and generates vertices of the optimal planned path based on the second six-degree-of-freedom pose, wherein the vertices include visual feature points and landmark points corresponding to the movement of the mobile device based on the optimal planned path, and the visual feature points have one or more descriptors, which are data structures associated with the visual feature points and are used to describe the appearance and attributes of the feature points; The guidance module is used to compare and optimize the local map and vertices of the path through the service module deployed on the mobile platform to obtain the target movement trajectory of the mobile device to the target point, and guide the mobile device to move to the target point based on the target movement trajectory.

6. The device according to claim 5, characterized in that The comparison module is specifically used to: The pose coordinate systems of the portable terminal and the mobile platform are aligned using the visual tags corresponding to the visual sensor or the measurement data of the inertial measurement unit.

7. The device according to claim 6, characterized in that The comparison module is further specifically used for: The pose coordinate systems of the portable terminal and the mobile platform are aligned through ultra-wideband positioning or lidar calibration.

8. The device according to claim 5, characterized in that The guiding module is specifically used to: Perform local graph optimization through the multi-threaded service in the service module deployed on the mobile platform to complete the path local map correction, and build a loop detection database based on the corrected path local map; Performing loop closure detection on the vertex using a bag-of-words model installed on a service module deployed on a mobile platform to obtain a target path corresponding to the vertex; Extracting a real-time path local map corresponding to the target path from the loop detection database using a sliding window method; The target path is optimized based on the comparison of the real-time path local map to obtain a target movement trajectory of the mobile device to the target point, and the mobile device is guided to move to the target point based on the target movement trajectory.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

Images