A collaborative localization method based on scene perception element semantics and knowledge base

By using a collaborative localization method based on scene-aware element semantics and knowledge base across platforms, and by employing pattern recognition and knowledge base filtering of static and dynamic element sets, high-precision relative positioning of multiple platforms in satellite navigation-constrained environments is achieved, making it suitable for dynamic application scenarios.

CN114814871BActive Publication Date: 2024-11-08THE 54TH RESEARCH INSTITUTE OF CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210455951.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-28
Publication Date
2024-11-08
Estimated Expiration
2042-04-28

AI Technical Summary

Technical Problem

In multi-platform application scenarios where satellite navigation is limited, there are no positioning base stations, and there is no line-of-sight measurement between platforms, existing technologies are difficult to achieve continuous relative positioning, especially in urban traffic and complex indoor environments where positioning accuracy is low or fails.

Method used

Scene data is acquired through visual or lidar sensors on Platform A and Platform B. Static and dynamic element sets are filtered using pattern recognition and knowledge base, semantics are assigned, and the relative spatial relationships between the platforms are calculated based on the scene element information exchanged through the communication network.

Benefits of technology

It achieves highly robust and continuous multi-platform relative positioning in non-cooperative and unfamiliar environments, solving the problem of low positioning accuracy or failure in existing technologies, and is suitable for dynamic application scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114814871B_ABST
    Figure CN114814871B_ABST
Patent Text Reader

Abstract

The present invention discloses a collaborative positioning method based on scene perception element semantics and knowledge base, and belongs to the field of navigation and positioning technology. Aiming at the technical approach of realizing collaborative positioning by using feature information in a high dynamic environment, the present invention adopts a collaborative positioning method based on scene perception element semantics and knowledge base, identifies elements in the perceived scene through each platform, and assigns corresponding semantic labels such as mobile / static elements and mobile element types, and interacts scene elements and semantic labels through a network; the platform screens out mobile elements with predictable spatial evolution laws based on the knowledge base, and then performs collaborative positioning based on the spatial relationship between these mobile elements and the static elements shared by the scene in the perception range of different platforms. The method is simple and easy to implement, and is of great significance for ensuring relative positioning between multiple platforms in multi-platform application scenarios where satellite navigation applications are limited, there is no positioning base station, there is no line-of-sight measurement between platforms, and relative positioning capabilities are required.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of navigation and positioning technology, and in particular refers to a collaborative positioning method based on scene perception element semantics and a knowledge base. Background Art

[0002] In the process of collaborative perception and inspection by multiple platforms, it is necessary to obtain the relative position information of each platform in real time to meet the needs of command and dispatch and behavior coordination. Usually, multiple platforms rely on satellite navigation to obtain absolute positioning information and then transmit the position information through the communication network; however, in scenarios where satellite navigation is interfered with or blocked, including urban canyons, mountains, indoors, underground, etc., where satellite navigation is unavailable, solving the problem of continuous relative positioning of multiple platforms becomes an important challenge.

[0003] Many studies have been conducted at home and abroad to address this issue, and the main technical approaches can be summarized into the following three categories: First, by deploying multiple radio or acoustic base stations, each platform can achieve continuous positioning through wireless ranging with multiple base stations; second, by sending and receiving wireless signals with ranging capabilities between multiple platforms to achieve relative ranging, and then achieve relative positioning based on ranging information; third, SLAM autonomous positioning based on sensors such as lidar and vision. The above technical approaches all have certain application conditions. For application scenarios that do not have the conditions for pre-installation of base stations, do not have line-of-sight wireless ranging capabilities, do not have high-value sensors, and require long-term relative positioning, a new positioning mode is needed.

[0004] Relying on the spatial mapping relationship of the same elements in the perception information of the application scene by different platforms is a way to achieve relative positioning of multiple platforms without line-of-sight measurement, which potentially relies on wireless communication between platforms. It can usually be achieved based on preset visual reference objects, but it is not applicable to application scenarios where it is difficult to preset known reference objects; another way is based on the perception, extraction and collaborative use of the same static feature information in the scene, which requires the identification and elimination of moving elements in the scene, but for scenes with many moving elements such as urban traffic and complex indoor environments, the removal of moving objects will result in fewer static reference elements, which will lead to poor positioning results. Summary of the invention

[0005] The purpose of the present invention is to provide a collaborative positioning method based on scene perception element semantics and knowledge base. The method can be used for multi-platform application scenarios where satellite navigation applications are limited, there is no positioning base station, there is no line-of-sight measurement between platforms, and relative positioning capabilities are required. The method can achieve the use of communication networks and perception scenes between multiple platforms to obtain relative positioning information.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A collaborative positioning method based on scene perception element semantics and knowledge base includes the following steps:

[0008] (1) Platform A at t 0 It can obtain the visual image or laser point cloud data set of the scene in real time through visual or laser radar sensors. It can filter out static feature sets and dynamic feature sets with predictable motion states from the data set through itself or other nodes O based on pattern recognition and knowledge base, obtain the relative spatial relationship between each feature set and platform A, and give semantics to each feature set.

[0009] (2) Platform B uses the same process as step (1) to obtain the platform B at t 1 The static element set, dynamic element set, element semantics, and the spatial position of each element set in its perceived image data relative to platform B at all times; the above information is sent to node O or platform A;

[0010] (3) Node O or platform A extracts the static and dynamic feature sets that are perceived by both platform A and platform B, and obtains the spatial relationship between platform A and platform B based on the spatial relationship between the static or dynamic feature sets and platform A and platform B.

[0011] Furthermore, the specific method of step (1) is:

[0012] (101) Platform A at t 0 Real-time acquisition of visual images or laser point cloud datasets of the scene through visual or lidar sensors:

[0013] {a k}, k = 1 ~ n

[0014] In the formula, a k Represents a single visual pixel or laser point obtained, and n is the number of data points obtained at a single time;

[0015] Platform A converts the dataset {a k} and its acquisition time information are sent to other nodes O with pattern recognition computing capabilities, or the platform A itself uses pattern recognition to send the scene data {a k} is divided into static element sets {b p} and dynamic feature sets {C m}={{c q1},{c q2}, …{c qm}}:

[0016] {a k}={b p}+{C m}={b p}+{{c q1},{c q2}, …{c qm}}

[0017] Among them, {b p}The subscript represents the pth static data point; {C m} subscript represents the mth group of dynamic element sets; the same group of dynamic element sets {c q} represents a set of perceptual data points of a certain object identified by pattern recognition; different {c q} represents different objects identified through pattern recognition; {C m} represents the set of perceptual elements corresponding to an object with the ability to move, but does not mean that actual movement has occurred;

[0018] (102) Platform A or node O performs {C m} are assigned semantic labels Y = {y 1 ,y 2 ,…y m}, represented by different values ​​{c qm}, and objects that do not have a certain type but are identified as movable objects; in addition, based on pattern recognition, the static element set {b p} contains multiple objects or static reference data to give semantics G = {g 1 , g 2 , …g j};

[0019] (103) Platform A or node O predicts or obtains a knowledge base through the network, and the content of the knowledge base is the movement laws of different types of objects at different times;

[0020] (104) Platform A or node O has a dynamic element set {C m} to screen, the screening method is to identify m dynamic element sets {c q The semantics of {y m}, whether the acquisition time is within the time and object type range contained in the knowledge base, and filter out the dynamic element set within the knowledge base range and its semantic tags

[0021] (105) Platform A or node O calculates the static element set {b p} and dynamic element set {c q1}~{c qx}The spatial position relative to platform A in the image data perceived by platform A.

[0022] Furthermore, the specific method of step (3) is:

[0023] (301) Platform A or node O extracts the same static and dynamic perception elements of platform A and platform B according to the semantic information of the static and dynamic element sets;

[0024] (302) If platform A and platform B only have the same static perception elements, then based on the same static perception elements at t 0 Time and t 1 The spatial position relative to platform A and platform B at the moment is calculated to get t 0 Moment platform A and t 1 The relative spatial relationship of platform B at each moment;

[0025] (303) If platform A and platform B have the same dynamic perception elements at the same time, platform A or node O calculates the spatial movement characteristics of the same moving object at t based on the spatial movement characteristics of different dynamic objects reflected in the knowledge base. 0 Time to t 1 The spatial displacement at the moment t is based on the same dynamic perception element. 0 Time and t 1 The spatial position relative to platform A and platform B at the moment and the obtained spatial displacement are calculated to obtain t 0 Moment platform A and t 1 The relative spatial relationship of platform B at each moment.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] (1) Compared with the collaborative positioning method of pre-arranging known visual image references in the application environment, such as pre-placing targets or images with known patterns and then utilizing multi-platform visual processing, the present invention completes positioning based on the recognition of scene elements and the identification of the same reference element set. Therefore, there is no need to preset reference objects in advance, and the need for collaborative positioning in non-cooperative unfamiliar application environments can be met.

[0028] (2) Compared with the collaborative positioning method based on static reference feature points, the present invention proposes a collaborative positioning method based on the identification of scene static and dynamic element sets, semantics and knowledge base. The motion characteristics of the dynamic elements of the scene are obtained through the knowledge base, and then collaborative positioning based on the dynamic elements in the scene is achieved. The method solves the problem of low or failed positioning accuracy due to the lack of static reference information in application scenarios such as urban transportation with a large number of moving elements, or positioning errors caused by mistakenly taking large moving objects in the scene as static references when relying on the collaborative positioning method of static feature points. The method has high robustness and continuous positioning capabilities in dynamic application environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0031] A collaborative positioning method based on scene perception element semantics and knowledge base includes the following steps:

[0032] (1) Platform A at t 0 The visual image or laser point cloud data set of the scene is obtained instantly through the visual or laser radar sensor. Through itself or other nodes O, based on pattern recognition and knowledge base, the data set is filtered out into static element sets and dynamic element sets with predictable motion states, the relative spatial relationship between each element set and platform A is obtained, and each element set is given semantics. Specifically, it includes:

[0033] (101) Platform A at t 0 Real-time acquisition of visual images or laser point cloud datasets of the scene through visual or lidar sensors

[0034] {a k}, k = 1 ~ n

[0035] In the formula, a k Represents a single visual pixel or laser point. n data points can be obtained at a time.

[0036] Platform A converts the dataset {a k} and its acquisition time information are sent to other nodes O with pattern recognition computing capabilities, or the platform A itself uses pattern recognition to send the scene data {a k} is divided into static element sets {b p} and dynamic feature sets {C m}={{c q1},{c q2}, …{c qm}}:

[0037] {a k}={b p}+{C m}={b p}+{{c q1},{c q2}, …{c qm}}

[0038] Among them, {b p}The subscript represents the pth static data point; {C m} subscript represents the mth group of dynamic element sets. The same group of dynamic element sets {c q} represents a set of perceptual data points of a certain object identified by pattern recognition; different {c q} represents different objects identified through pattern recognition; {Cm} means the set of perceptual elements corresponding to objects with the ability to move, and does not mean that actual movement has occurred;

[0039] (102) Platform A or node O performs {C m} are assigned semantic labels Y = {y 1 ,y 2 ,…y m}, represented by different values ​​{c qm} corresponding to the object type, and the object type that does not have a certain object type but is identified as a movable object; and based on pattern recognition, the static element set {b p} contains multiple objects or static reference data to give semantics G = {g 1 , g 2 , …g j};

[0040] (103) Platform A or node O predicts or obtains a knowledge base through the network, and the content of the knowledge base is the movement laws of different types of objects at different times;

[0041] (104) Platform A or node O has a dynamic element set {C m} for screening. The screening method is to sequentially identify m dynamic element sets {c q The semantics of {y m}, whether the acquisition time is within the time and object type range contained in the knowledge base, and filter out the dynamic element set within the knowledge base range and its semantic tags

[0042] (105) Platform A or node O calculates the static element set {b p} and dynamic element set {c q1}~{c qx} The spatial position relative to platform A in the image data sensed by platform A;

[0043] (2) Based on the same process as step (1), platform B obtains the value of platform B at t 1 The static element set, dynamic element set, element semantics, and the spatial position of each element set in its perceived image data relative to platform B at all times; and the above information is sent to node O or platform A;

[0044] (3) Node O or platform A extracts the static and dynamic feature sets that are perceived by both platform A and platform B, and obtains the spatial relationship between platform A and platform B based on the spatial relationship between the static or dynamic feature sets and platform A and platform B. Specifically, it includes:

[0045] (301) Platform A or node O extracts the same static and dynamic perception elements of platform A and platform B according to the semantic information of the static and dynamic element sets;

[0046] (302) If platform A and platform B only have the same static perception elements, then based on the same static perception elements at t 0 Time and t 1 The spatial position relative to platform A and platform B at the moment is calculated to get t 0 Moment platform A and t 1 The relative spatial relationship of platform B at each moment.

[0047] (303) If platform A and platform B have the same dynamic perception elements at the same time, platform A or node O calculates the spatial movement characteristics of the same moving object at t based on the spatial movement characteristics of different dynamic objects reflected in the knowledge base. 0 Time to t 1 The spatial displacement at the moment t is based on the same dynamic perception element. 0 Time and t 1 The spatial position relative to platform A and platform B at the moment and the obtained spatial displacement are calculated to obtain t 0 Moment platform A and t 1 The relative spatial relationship of platform B at each moment.

[0048] Furthermore, the platform refers to a device that has or can be connected to a visual sensor, a lidar sensor or other sensor that can perceive and map environmental information, and has communication capabilities, including special detection equipment, terminal equipment mounted on a mobile platform, general mobile phone devices, etc.

[0049] Furthermore, the node refers to a device that has the ability to receive information sent by the platform and has certain computing capabilities, including background servers, computing terminals mounted on mobile platforms, fixed computing terminals, etc.

[0050] This method is simple and easy to implement, and is of great significance for ensuring relative positioning between multiple platforms in multi-platform application scenarios where satellite navigation applications are limited, there are no positioning base stations, there is no line-of-sight measurement between platforms, and relative positioning capabilities are required.

[0051] Here is a more specific example:

[0052] Figure 1 This is a flowchart of a collaborative positioning method based on scene perception element semantics and knowledge base. This method can be used for multi-platform application scenarios where satellite navigation applications are limited, there is no positioning base station, there is no line-of-sight measurement between platforms, and relative positioning capabilities are required. It can achieve relative positioning information between multiple platforms using communication networks and perception scenes. Here, taking multiple drones using perception payloads with visual cameras to achieve collaborative relative positioning in indoor or underground scenes as an example, the implementation steps are described in detail:

[0053] (1) UAV A at t 0 The visual image dataset of the scene is obtained instantly through the visual sensor at all times. Based on pattern recognition and knowledge base, the dataset is filtered out into static feature sets and dynamic feature sets with predictable motion states through itself or the background server O, and the relative spatial relationship between each feature set and the drone A is obtained, and semantics is given to each feature set. Specifically, it includes:

[0054] (101) UAV A in t 0 Real-time acquisition of visual image data sets of scenes through visual sensors

[0055] {a k}, k = 1 ~ n

[0056] In the formula, a k Represents a single visual pixel point obtained. n data points can be obtained at a time.

[0057] Drone A sends the dataset {a k} and its acquisition time information are sent to other background servers O with pattern recognition computing capabilities, or the drone A itself uses pattern recognition to send scene data {a k} is divided into static element sets {b p} and dynamic feature sets {C m}={{c q1},{c q2}, …{c qm}}:

[0058] {a k}={b p}+{C m}={b p}+{{c q1},{c q2}, …{c qm}}

[0059] Among them, {b p}The subscript represents the pth static data point; {C m} subscript represents the mth group of dynamic element sets. The same group of dynamic element sets {c q} represents a set of perceptual data points of a certain object identified by pattern recognition; different {c q} represents objects that are identified as different objects through pattern recognition. m} means the set of perception elements corresponding to objects with the ability to move, and does not mean that they are actually moving, such as a vehicle that is temporarily static at the moment of perception;

[0060] (102) UAV A or backend server O performs {Cm} are assigned semantic labels Y = {y 1 ,y 2 ,…y m}, represented by different values ​​{c qm}, and the object type corresponding to the object type that does not have a certain object type but is identified as a movable object, such as the mobile element set that is judged to be mobile but the type cannot be identified, and the semantic label of the mobile element set such as personnel is assigned to D1, D2...; and the static element set {b p} contains multiple objects or static reference data to give semantics G = {g 1 , g 2 , …g j}, such as the semantic label of the environmental background data is assigned as J0, the window is J1, the table and chair are J2...;

[0061] (103) UAV A or the backend server O predicts or obtains a knowledge base through the network, and the content of the knowledge base is the movement rules of different types of objects at different times;

[0062] (104) UAV A or backend server O processes the dynamic element set {C m} for screening. The screening method is to sequentially identify m dynamic element sets {c q The semantics of {y m}, whether the acquisition time is within the time and object type range contained in the knowledge base, and filter out the dynamic element set within the knowledge base range and its semantic tags

[0063] (105) UAV A or backend server O calculates the static element set {b p} and dynamic element set {c q1}~{c qx} The spatial position relative to UAV A in the image data sensed by UAV A;

[0064] (2) Based on the same process as step (1), drone B obtains the information about drone B at t 1 The static feature set, dynamic feature set, feature semantics, and spatial position of each feature set in its perceived image data relative to drone B at all times; and the above information is sent to drone A or backend server O;

[0065] (3) Drone A or backend server O extracts the static and dynamic feature sets that are perceived by drone A and drone B in common, and obtains the spatial relationship between drone A and drone B based on the static or dynamic feature sets and the spatial relationship between drone A and drone B. Specifically, it includes:

[0066] (301) UAV A or the backend server O extracts the same static and dynamic perception elements of UAV A and UAV B according to the semantic information of the static and dynamic element sets;

[0067] (302) If UAV A and UAV B only have the same static perception elements, then based on the same static perception elements at t 0 Time and t 1 The spatial position of UAV A and UAV B at the moment is calculated to get t 0 Moment Drone A and T 1 The relative spatial relationship of UAV B at this moment.

[0068] (303) If platform A and platform B have the same dynamic perception elements at the same time, drone A or backend server O calculates the spatial movement characteristics of the same moving object at t based on the spatial movement characteristics of different dynamic objects reflected in the knowledge base. 0 Time to t 1 For example, if UAV A and UAV B both perceive the same uniformly moving object in the application scenario, they can obtain the moving speed of the reference object based on the knowledge base and calculate t 0 Time to t 1 Reference the displacement of the object at all times; then based on the same dynamic perception elements at t 0 Time and t 1 The spatial position of drone A and drone B at the moment and the obtained spatial displacement are calculated to obtain t 0 Moment Drone A and T 1 The relative spatial relationship of UAV B at this moment.

[0069] The present invention aims at a technical approach to realize collaborative positioning by utilizing feature information in a high-dynamic environment, and adopts a collaborative positioning method based on the semantics of scene perception elements and a knowledge base. Elements in the perceived scene are identified through various platforms, and corresponding semantic labels such as mobile / static elements and mobile element types are assigned, and scene elements and semantic labels are interacted through a network. The platform screens out mobile elements with predictable spatial evolution laws based on the knowledge base, and then performs collaborative positioning according to the spatial relationship between these mobile elements and the static elements shared by the scene in the perception range of different platforms.

[0070] In summary, the present invention can meet the needs of collaborative positioning in non-cooperative unfamiliar application environments, while having high robustness and continuous positioning capabilities in dynamic application environments.

[0071] The present invention can be applied to multi-platform application scenarios where satellite navigation applications are limited, there is no positioning base station, there is no line-of-sight measurement between platforms, and relative positioning capabilities are required, so as to achieve the acquisition of relative positioning information by using communication networks and perception scenarios between multiple platforms.

Claims

1. A collaborative localization method based on scene perception element semantics and knowledge base, characterized in that: The following steps are involved: (1) Platform A obtains the visual image or laser point cloud data set of the scene in real time through the visual or laser radar sensor at time t0. Through itself or other nodes O, based on pattern recognition and knowledge base, it selects static feature sets and dynamic feature sets with predictable motion states from the data set, obtains the relative spatial relationship between each feature set and platform A, and assigns semantics to each feature set. The specific method is as follows: (101) Platform A obtains the visual image or laser point cloud data set of the scene in real time through the visual or laser radar sensor at time t0: {a k },k=1~n In the formula, a k Represents a single visual pixel or laser point obtained, and n is the number of data points obtained at a single time; Platform A converts the dataset {a k } and its acquisition time information are sent to other nodes O with pattern recognition computing capabilities, or the platform A itself uses pattern recognition to send the scene data {a k } is divided into static element sets {b p } and dynamic feature sets {C m }={{c q1 },{c q2 }, …{c qm }}: {a k }={b p }+{C m }={b p }+{{c q1 },{c q2 },…{c qm }} Among them, {b p }The subscript represents the pth static data point; {C m } subscript represents the mth group of dynamic element sets; the same group of dynamic element sets {c q } represents a set of perceptual data points of a certain object identified by pattern recognition; different {c q } represents different objects identified through pattern recognition; {C m } represents the set of perceptual elements corresponding to an object with the ability to move, but does not mean that actual movement has occurred; (102) Platform A or node O performs {C m } are assigned semantic labels Y = {y1, y2, ...y m }, represented by different values ​​{c qm }, and objects that do not have a certain type but are identified as movable objects; in addition, based on pattern recognition, the static element set {b p } contains multiple objects or static reference data to give semantics G = {g1, g2, ... g j }; (103) Platform A or node O predicts or obtains a knowledge base through the network, and the content of the knowledge base is the movement laws of different types of objects at different times; (104) Platform A or node O has a dynamic element set {C m } to screen, the screening method is to identify m dynamic element sets {c q The semantics of {y m }, whether the acquisition time is within the time and object type range contained in the knowledge base, and filter out the dynamic element set within the knowledge base range and its semantic tags (105) Platform A or node O calculates the static element set {bp} and the dynamic element set {c q1 }~{c qx } The spatial position relative to platform A in the image data sensed by platform A; (2) Platform B uses the same process as step (1) to obtain the static feature set, dynamic feature set, feature semantics, and the spatial position of each feature set relative to platform B in its perceived image data at time t1; and sends the above information to node O or platform A; (3) Node O or platform A extracts the static and dynamic feature sets that are perceived by both platform A and platform B, and obtains the spatial relationship between platform A and platform B based on the spatial relationship between the static or dynamic feature sets and platform A and platform B.

2. The collaborative positioning method based on scene perception element semantics and knowledge base according to claim 1, characterized in that: The specific method of step (3) is: (301) Platform A or node O extracts the same static and dynamic perception elements of platform A and platform B according to the semantic information of the static and dynamic element sets; (302) If platform A and platform B only have the same static perception elements, then based on the spatial positions of the same static perception elements relative to platform A and platform B at time t0 and time t1, the relative spatial relationship between platform A at time t0 and platform B at time t1 is calculated; (303) If platform A and platform B have the same dynamic perception elements at the same time, platform A or node O calculates the spatial displacement of the same moving object from time t0 to time t1 based on the spatial movement characteristics of different dynamic objects reflected in the knowledge base, and calculates the relative spatial relationship between platform A at time t0 and platform B at time t1 based on the spatial positions of the same dynamic perception elements relative to platform A and platform B at time t0 and time t1 and the obtained spatial displacement.