Group cooperation positioning method and device and group platform system

Abstract

The invention provides a group collaborative positioning method and device and a group platform system, and belongs to the technical field of navigation positioning. The method comprises the following steps: receiving and responding to a collaborative positioning request which is sent by first member equipment in the group platform system and carries a current observation image; identifying an identification element in the current observation image to obtain absolute position information of each second member device in the world coordinate system; and based on the absolute position information of each second member device in the world coordinate system, calculating the pose information of the first member device, and transferring the pose information of the first member device to the first member device, and the first member equipment initializes an inertial navigation device arranged in the first member equipment or corrects an accumulated error by using the pose information of the first member equipment. According to the group cooperation positioning method and device and the group platform system provided by the invention, the pose of any equipment can be solved by adopting a group cooperation mode, and the light weight and the positioning precision of the platform are both considered.

Description

Technical Field

[0001] The present invention relates to the field of navigation and positioning technology, and in particular to a group collaborative positioning method, device and group platform system. Background Art

[0002] When using augmented reality, mixed reality devices or small drone platforms to interact with geographic information outdoors, it is necessary not only to obtain accurate geographic locations, but also accurate angular elements. Augmented reality, mixed reality devices or small drone platforms are limited by size, power consumption and other aspects, and cannot be installed with GNSS orientation devices with long baselines, and magnetic compasses are susceptible to interference. Therefore, gyroscopes are usually used to solve angular elements. The use of gyroscopes requires a method that meets the needs of the application scenario to assign initial values ​​and continuously calibrate during use. Therefore, the existing navigation and positioning technology is restricted by size, power consumption and other aspects, and cannot take into account both positioning accuracy and platform lightweight. Summary of the invention

[0003] The present invention provides a group collaborative positioning method, device and group platform system, which are used to solve the defects in the prior art that are restricted by volume, power consumption and the like and cannot take into account both positioning accuracy and platform lightweight at the same time.

[0004] The present invention provides a group collaborative positioning method, comprising: Receiving and responding to a collaborative positioning request carrying a current observation image sent by a first member device in a group platform system; wherein the first member device is any smart wearable device in the group platform system; the current observation image includes identification elements on other member devices except the first member device; the identification elements are used to distinguish smart wearable devices in the same group platform system; Obtaining the absolute position information of each second member device in the world coordinate system by identifying the identification element in the current observation image; wherein the second member device is the member device corresponding to the identification element contained in the current observation image; Based on the absolute position information of each second member device in the world coordinate system, the posture information of the first member device is calculated, and the posture information of the first member device is transferred to the first member device, so that the first member device can use the posture information of the first member device to initialize the inertial navigation device set in the first member device or correct the accumulated error.

[0005] According to a group collaborative positioning method provided by the present invention, the method of calculating the position information of the first member device based on the absolute position information of each second member device in the world coordinate system includes: When it is determined that the number of the second member devices is greater than 2, determining, based on the current observed image, projection position information of each second member device in the camera coordinate system; Based on the projection position information of each second member device in the camera coordinate system and the absolute position information of each second member device in the world coordinate system, the direction angle, pitch angle and roll angle in the position and posture information of the first member device are determined.

[0006] According to a group collaborative positioning method provided by the present invention, the method of calculating the position information of the first member device based on the absolute position information of each second member device in the world coordinate system includes: When it is determined that the number of the second member devices is equal to 2, the direction angle in the position and posture information of the first member device is determined according to the absolute position information of the second member devices in the world coordinate system.

[0007] According to a group collaborative positioning method provided by the present invention, after calculating the position information of the first member device based on the absolute position information of each second member device in the world coordinate system, the method includes: The position and posture information of the first member device is converted into Euler angle information or axis angle information.

[0008] According to a group collaborative positioning method provided by the present invention, the absolute position information of each second member device in the world coordinate system is obtained by identifying the identification elements in the current observation image, including: Based on the current observed image, determine the timestamp of the current observed image and identification information of each second member device; Filter out the satellite positioning information whose timestamp coincides with the timestamp of the current observed image and matches the identification information of the second member device from the satellite positioning information uploaded by each member device in real time, as the absolute position information of the second member device in the world coordinate system; The identification information of any second member device is determined according to the identification element belonging to the corresponding second member device in the current observed image.

[0009] According to a group collaborative positioning method provided by the present invention, the identification elements include characters, barcodes, QR codes, and color rings set on the smart wearable device.

[0010] The present invention also provides a group collaborative positioning device, comprising: A request response module, configured to receive and respond to a collaborative positioning request carrying a current observation image sent by a first member device in a group platform system; wherein the first member device is any smart wearable device in the group platform system; the current observation image includes identification elements on other member devices except the first member device; the identification elements are used to distinguish smart wearable devices in the same group platform system; A first positioning module, configured to obtain the absolute position information of each second member device in a world coordinate system by identifying the identification element in the current observation image; wherein the second member device is the member device corresponding to the identification element contained in the current observation image; The second positioning module is used to calculate the posture information of the first member device based on the absolute position information of each second member device in the world coordinate system, and transfer the posture information of the first member device to the first member device, so that the first member device can use the posture information of the first member device to initialize the inertial navigation device set in the first member device or correct the accumulated error.

[0011] The present invention also provides a group platform system, comprising a host device at a group center node, and a plurality of smart wearable devices at group edge nodes; The smart wearable device includes a smart wearable device body, and a processor, a positioning device and an inertial navigation device arranged in the smart wearable device body, each of the processors is used to forward the absolute position information of the positioning device in the world coordinate system periodically collected by the positioning device to the host device; If the host device is a remote server terminal, the processor is further used to send a collaborative positioning request carrying a current observation image to the host device, and use the posture information of the smart wearable device fed back by the group collaborative positioning method as described in any of the above items when the host device responds to the collaborative positioning request to initialize the inertial navigation device in the smart wearable device to which it belongs or correct the accumulated error; Among them, the smart wearable device body includes augmented reality glasses, mixed reality glasses and extended reality glasses.

[0012] According to a group platform system provided by the present invention, if the host device is any smart wearable device, the processor is also used to execute the group collaborative positioning method as described in any of the above items.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the group collaborative positioning method as described in any one of the above is implemented.

[0014] The group collaborative positioning method, device and group platform system provided by the present invention, when responding to the collaborative positioning request issued by the first member device, identifies all the second member devices observed by the first member device through the identification elements of the current observation image encapsulated in the collaborative positioning request, and then uses the absolute position information of the second member device in the world coordinate system to convert the position and orientation information of the first member device. It can provide a theoretical basis for the positioning and orientation of any member device in the group, especially for the group platform composed of smaller devices, which only needs to sense and deploy individual member devices, that is, it can solve the position and orientation of any member device through the group collaborative method adopted in the group platform system, and maximize the positioning accuracy on the basis of lightweight platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is a flow chart of the group collaborative positioning method provided by the present invention; Figure 2 It is a structural schematic diagram of a group collaborative positioning device provided by the present invention; Figure 3 It is a structural schematic diagram of the group platform system provided by the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0019] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present invention, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0020] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0021] Figure 1 FIG. 1 is a flow chart of the group collaborative positioning method provided by the present invention. Figure 1 As shown, the group collaborative positioning method provided by the embodiment of the present invention includes: step 101, receiving and responding to a collaborative positioning request carrying a current observation image sent by a first member device in a group platform system.

[0022] The first member device is any smart wearable device in the group platform system. The current observation image includes identification elements on other member devices except the first member device.

[0023] It should be noted that the execution subject of the group collaborative positioning method is a group collaborative positioning device, which can be integrated on a control chip in any member device in the group platform system.

[0024] The group platform system is formed by multiple smart wearable devices communicating with each other. Each smart wearable device in the group is integrated with a camera for environmental perception, and the camera and the smart wearable device are always facing the same direction. At the same time, the identification elements set on the surface of the smart wearable device ensure its independence and uniqueness within the group.

[0025] The application scenario of the group collaborative positioning method provided by an embodiment of the present invention is to capture images of other members in the group through a camera arranged in a head-mounted device, obtain the geographic locations of other members through the head-mounted device identification recorded in the image, and then use the geographic locations of other members to locate and orient the observer.

[0026] The embodiment of the present invention does not specifically limit the user role of the member device where the group collaborative positioning device is located in the group platform system.

[0027] Exemplarily, the group collaborative positioning device can be integrated on a control chip in a member device whose user role is a data switch in the group platform system, so that the first member device can interact with the member device acting as a data switch, and the group collaborative positioning device can calculate the posture of the first member device and send it to the first member device.

[0028] Exemplarily, the group collaborative positioning device may also be integrated on the control chip of the first member device itself. After the first member device receives the location information of other member devices transmitted by the member device serving as a data switch, the first member device calculates its own posture.

[0029] It should be noted that, before step 101, the first member device with positioning requirements can be used as an observer to shoot other members in the group using a camera to ensure that the identification elements on other observed member devices in the current observation image are evenly distributed. Then, the processor in the first member device encapsulates the current observation image in a collaborative positioning request and transmits the request to the positioning device of the group collaboration.

[0030] Specifically, in step 101, in the process of each member device in the group platform system periodically uploading perception data to the group collaborative positioning device, the group collaborative positioning device can not only summarize and classify the received perception data through the identification elements of each member device, but also receive a collaborative positioning request from the first member device, and respond to the request, and parse out the current observation image obtained when the first member device observes other member devices except itself.

[0031] Step 102: Obtain absolute position information of each second member device in a world coordinate system by identifying identification elements in the current observation image.

[0032] The second member device is a member device corresponding to the identification element contained in the current observed image.

[0033] Specifically, in step 102, the group collaborative positioning device identifies the identification elements in the current observation image parsed in step 101, marks the smart wearable device corresponding to the identifiable identification elements as a second member device, and obtains the absolute position information of the second member device in the world coordinate system perceived by the second member device in the real world through data interaction with the second member device.

[0034] The absolute position information describes the location of a place based on a fixed point on the earth. The most common method is to use coordinates to determine the location, such as longitude, latitude, and elevation. The embodiment of the present invention does not specifically limit the data interaction method for obtaining the absolute position information of the second member device in the world coordinate system.

[0035] Exemplarily, the group collaborative positioning device can extract the absolute position information of the second member device in the world coordinate system sensed and uploaded by the second member device from the perception data summarized for each member device according to the identified identification element.

[0036] Exemplarily, the group collaborative positioning device can issue a perception control instruction to the corresponding second member device based on the recognized identification element, and then collect the absolute position information of each second member device in the world coordinate system collected in response to the perception control instruction.

[0037] Step 103: Based on the absolute position information of each second member device in the world coordinate system, calculate the posture information of the first member device, and transfer the posture information of the first member device to the first member device, so that the first member device can use the posture information of the first member device to initialize the inertial navigation device set in the first member device or correct the accumulated error.

[0038] Specifically, in step 103, the group collaborative positioning device applies the absolute position information of all second member devices in the world coordinate system observed by the first member device to the rear intersection principle, calculates the posture information of the first member device as the observer in the world coordinate system, and transfers the posture information of the first member device in the world coordinate system to the first member device.

[0039] Among them, the position and posture information of the first member device in the world coordinate system includes the position information and attitude information of the first member device in the world coordinate system. Any rigid body can use the position information and attitude information to accurately and uniquely represent its position state in the spatial coordinate system (OXYZ). The position information can have the same description dimension as the absolute position information, or it can be the x, y, and z coordinates in the spatial coordinate system (OXYZ). Correspondingly, the attitude information is the angle between the first member device and the OX axis, the angle between the first member device and the OY axis, and the angle between the first member device and the OZ axis.

[0040] It can be understood that after step 103, the first member device uses the position information of the first member device in the world coordinate system sent by the group collaborative positioning device to position itself. If the first member device itself is not deployed with a positioning device, it can be positioned under the lightweight condition of the first member device. If the first member device itself is deployed with a positioning device, it can assist in positioning error correction.

[0041] At the same time, the first member device orients itself using the posture information of the first member device in the world coordinate system sent by the group collaborative positioning device. If the first member device itself is not equipped with an inertial navigation device, it can be oriented under the lightweight condition of the first member device. If the first member device itself is equipped with an inertial navigation device, an initial value can be assigned to the inertial navigation device to continuously accumulate increments during use after initialization, and it can also be used as a reference value to correct the accumulated error generated by the inertial navigation device during perception during subsequent use.

[0042] When responding to the collaborative positioning request issued by the first member device, the embodiment of the present invention identifies all the second member devices observed by the first member device through the identification elements of the current observation image encapsulated in the collaborative positioning request, and then uses the absolute position information of the second member device in the world coordinate system to convert the position and orientation information of the first member device. It can provide a theoretical basis for the positioning and orientation of any member device in the group, especially for the group platform composed of smaller devices, which only needs to sense and deploy individual member devices, that is, it can solve the position and orientation of any member device through the group collaboration method adopted within the group platform system, and maximize the positioning accuracy on the basis of lightweight platform.

[0043] Based on any of the above embodiments, the posture information of the first member device is calculated based on the absolute position information of each second member device in the world coordinate system, including: when it is determined that the number of the second member devices is greater than 2, based on the current observed image, determining the projection position information of each second member device in the camera coordinate system.

[0044] Specifically, in step 103, when the group collaborative positioning device identifies the identification elements in the current observation image in step 102 and determines that the number of second member devices observed in the image is greater than 2, the 2D point projected on the image by the 3D point corresponding to each second member device is extracted from the current observation image, and the two-dimensional pixel coordinates (i.e., 2D point coordinates) of each second member device are combined with the internal parameters pre-calibrated for the camera to restore the projection position information of each second member device in the camera coordinate system.

[0045] Based on the projection position information of each second member device in the camera coordinate system and the absolute position information of each second member device in the world coordinate system, the direction angle, pitch angle and roll angle in the position and posture information of the first member device are determined.

[0046] Specifically, the group collaborative positioning device uses the PnP algorithm to solve the correspondence between the 3D coordinate values ​​of three or more second member devices in the world coordinate system and their 2D projection coordinate values ​​in the camera coordinate system, so as to determine the yaw, pitch and roll angles of the camera of the first member device, and output them as the posture information of the first member device.

[0047] Among them, since there are three or more known point information in the image, the linear equation established around the correspondence between the 3D coordinate value and its 2D projection coordinate value in the camera coordinate has multiple solutions, so the P3P, direct linear transformation, EPnP and other methods of estimating the pose of three pairs of points can be used to determine the only solution. In addition, the least squares problem can be constructed and solved iteratively in a nonlinear optimization manner, which is the common bundle adjustment (Bundle Adjustment, BA). The embodiment of the present invention does not specifically limit the solution method of the PnP algorithm.

[0048] When the number of second member devices identified in the current observation image of the first member device is greater than 2, the embodiment of the present invention decides to substitute the projection position information of the second member device in the camera coordinate system and the absolute position information of the second member device in the world coordinate system as matching point pairs into the PnP algorithm to solve the heading angle, pitch angle and roll angle of the first member device itself. The camera posture of the member device as the observer during observation can be obtained through the geographic location information collected by the observed device in the group platform system, and it can be used as its own posture to initialize and calibrate the orientation, thereby improving the positioning and orientation accuracy of the device.

[0049] On the basis of any of the above embodiments, the posture information of the first member device is calculated based on the absolute position information of each second member device in the world coordinate system, including: when it is determined that the number of the second member devices is equal to 2, the direction angle in the posture information of the first member device is determined according to the absolute position information of the second member device in the world coordinate system.

[0050] Specifically, in step 103, the group collaborative positioning device determines that the number of second member devices observed in the image is equal to 2 when identifying the identification elements in the current observation image in step 102, without considering the elevation information in the absolute position information of the second member devices, and only solves the position of the second member device in the XY horizontal plane through trigonometric functions to obtain the direction angle of the photographer's first member device itself, and outputs it as the first member device posture information.

[0051] When the number of second member devices identified in the current observation image of the first member device is equal to 2, the embodiment of the present invention decides to use the absolute position information of the second member device in the world coordinate system to perform trigonometric function solution on the horizontal plane to obtain the direction angle of the first member device itself. The geographic location information collected by the observed device in the group platform system can be used to obtain the camera posture of the member device as the observer during observation, and use it as its own posture to initialize and calibrate the orientation, thereby improving the positioning and orientation accuracy of the device.

[0052] On the basis of any of the above embodiments, after calculating the position and posture information of the first member device based on the absolute position information of each second member device in the world coordinate system, the method includes: The position and posture information of the first member device is converted into Euler angle information or axis angle information.

[0053] Specifically, after step 103,

[0054] Based on any of the above embodiments, the absolute position information of each second member device in the world coordinate system is obtained by identifying the identification elements in the current observation image, including: based on the current observation image, determining the timestamp of the current observation image and the identification information of each second member device.

[0055] The identification information of any second member device is determined according to the identification element belonging to the corresponding second member device in the current observed image.

[0056] Specifically, in step 102, the group collaborative positioning device extracts the corresponding timestamp of the current observation image parsed from the collaborative positioning request, and obtains the identification information corresponding to the identification element fixedly set on each second member device through the mapping relationship pre-set between the identification element and the identification information.

[0057] The identification information can be used as a search field, so that the positioning device of the group collaboration can uniquely locate the perception information uploaded by the second member device observed by the first member device from the perception information summarized by each member device. The storage form of the identification information includes but is not limited to digital codes, character strings, etc., and the embodiments of the present invention do not specifically limit this.

[0058] From the satellite positioning information uploaded by each member device in real time, the satellite positioning information whose timestamp coincides with the timestamp of the current observed image and matches the identification information of the second member device is screened out as the absolute position information of the second member device in the world coordinate system.

[0059] Specifically, the group collaborative positioning device summarizes the satellite positioning information uploaded by each member device in real time, and uses the timestamp and identification information to uniquely determine the satellite positioning information of the perception content with the same timestamp and belonging to the second member device, so as to obtain the absolute position information of the second member device in the world coordinate system.

[0060] The embodiment of the present invention does not specifically limit the relationship between the satellite positioning information and the absolute position information.

[0061] Exemplarily, the group-coordinated positioning device may directly output the satellite positioning information (eg, longitude, latitude and altitude) acquired by the positioning device through satellite communication as the absolute position information of the second member device in the world coordinate system.

[0062] Exemplarily, the group collaborative positioning device can convert the satellite positioning information (for example, longitude, latitude and elevation) obtained by the positioning device during satellite communication into a spatial coordinate system (OXYZ), and use the x, y, and z coordinates under the spatial coordinate system (OXYZ) as the absolute position information of the second member device in the world coordinate system for direct output.

[0063] The embodiment of the present invention extracts the timestamp and identification information corresponding to various identification elements from the current observation image, and uses the two as search fields to uniquely map the absolute position information of the second member device that not only matches the timestamp overlap but also matches the information source from the perception data uploaded by the member device in real time. By establishing periodic data interaction between members in the group, a data basis can be provided for solving the position and posture of any member device, so as to achieve lightweight and improve positioning accuracy.

[0064] Based on any of the above embodiments, the identification elements include characters, barcodes, QR codes and color rings set on the smart wearable device.

[0065] It should be noted that characters refer to glyph-like units or symbols, including letters, numbers, operation symbols, punctuation marks and other symbols, as well as some functional symbols.

[0066] A barcode is a graphic identifier that arranges multiple black bars and spaces of varying widths according to certain coding rules to express a set of information.

[0067] A QR code is a black and white graphic that records data symbol information using certain geometric shapes distributed in a plane (in two dimensions) according to a certain pattern.

[0068] The color ring is a ring with a certain color painted on the surface of the smart wearable device to distinguish different devices by using different color combinations.

[0069] Specifically, one or more identifications can be set on any smart wearable device, and different target recognition methods and different rules corresponding to the smart wearable device can be set according to different identification information. Different identification information can also be combined to further improve the accuracy and versatility of recognition through the combined identification information.

[0070] It is understandable that the identification element can also be an optical mark. When the first member device emits scattered light and projects it onto the surface of the second member device, the optical mark set on the second member device will be activated by the light and reflect light of different wavelengths, so as to distinguish different second member devices by the wavelength of the reflected light.

[0071] The embodiments of the present invention set different types of identification information for independent or combined identification. On the one hand, it distinguishes smart wearable devices from multiple dimensions to prevent the inability to obtain weight information through only one identification due to force majeure factors, improves the applicability of identifying other member devices, and further improves the accuracy and reliability of group collaborative positioning.

[0072] Figure 2 FIG. 1 is a schematic diagram of the structure of a group cooperative positioning device provided by the present invention. Figure 2 As shown, the group collaborative positioning device provided in the embodiment of the present invention includes a request response module 210, a first positioning module 220 and a second positioning module 230, wherein: The request response module 210 is used to receive and respond to a collaborative positioning request carrying a current observation image sent by a first member device in the group platform system; wherein the first member device is any smart wearable device in the group platform system; the current observation image includes identification elements on other member devices except the first member device; the identification elements are used to distinguish smart wearable devices in the same group platform system.

[0073] The first positioning module 220 is used to obtain the absolute position information of each second member device in the world coordinate system by identifying the identification element in the current observation image; wherein the second member device is the member device corresponding to the identification element contained in the current observation image.

[0074] The second positioning module 230 is used to calculate the posture information of the first member device based on the absolute position information of each second member device in the world coordinate system, and transfer the posture information of the first member device to the first member device, so that the first member device can use the posture information of the first member device to initialize the inertial navigation device set in the first member device or correct the accumulated error.

[0075] Specifically, the request response module 210, the first positioning module 220 and the second positioning module 230 are electrically connected in sequence.

[0076] In the process where each member device in the group platform system periodically uploads perception data to the group collaborative positioning device, the request response module 210 can summarize and classify the received perception data through the identification elements of each member device, and can also receive a collaborative positioning request from the first member device and respond to the request, thereby parsing the current observation image obtained by the first member device when observing other member devices except itself.

[0077] The first positioning module 220 identifies the identification elements in the current observation image parsed by the request response module 210, marks the smart wearable device corresponding to the identifiable identification elements as a second member device, and obtains the absolute position information of the second member device in the world coordinate system perceived by the second member device in the real world through data interaction with the second member device.

[0078] The second positioning module 230 applies the absolute position information of all second member devices in the world coordinate system observed by the first member device to the resection principle, calculates the pose information of the first member device as the observer in the world coordinate system, and transfers the pose information of the first member device in the world coordinate system to the first member device.

[0079] Optionally, the second positioning module 230 includes a projection positioning unit and a first positioning unit, wherein: A projection positioning unit is used to determine the projection position information of each second member device in the camera coordinate system based on the current observation image when it is determined that the number of the second member devices is greater than 2.

[0080] The first positioning unit is used to determine the direction angle, pitch angle and roll angle in the posture information of the first member device based on the projection position information of each second member device in the camera coordinate system and the absolute position information of each second member device in the world coordinate system.

[0081] Optionally, the second positioning module 230 includes a second positioning unit, wherein: The second positioning unit is used to determine the direction angle in the posture information of the first member device according to the absolute position information of the second member device in the world coordinate system when it is determined that the number of the second member devices is equal to 2.

[0082] Optionally, the device further comprises an angle element conversion module, wherein: An angle element conversion module is used to convert the position and posture information of the first member device into Euler angle information or axis angle information.

[0083] Optionally, the first positioning module 220 includes an identification unit and a third positioning unit, wherein: The identification recognition unit is used to determine the timestamp of the current observation image and the identification information of each second member device based on the current observation image.

[0084] The third positioning unit is used to filter out the satellite positioning information whose timestamp coincides with the timestamp of the current observed image and matches the identification information of the second member device from the satellite positioning information uploaded by each member device in real time, as the absolute position information of the second member device in the world coordinate system.

[0085] The identification information of any second member device is determined according to the identification element belonging to the corresponding second member device in the current observed image.

[0086] Optionally, the identification element includes characters, barcodes, QR codes and color rings set on the smart wearable device.

[0087] The group collaborative positioning device provided in an embodiment of the present invention is used to execute the above-mentioned group collaborative positioning method of the present invention. Its implementation method is consistent with the implementation method of the group collaborative positioning method provided by the present invention and can achieve the same beneficial effects, which will not be repeated here.

[0088] When responding to the collaborative positioning request issued by the first member device, the embodiment of the present invention identifies all the second member devices observed by the first member device through the identification elements of the current observation image encapsulated in the collaborative positioning request, and then uses the absolute position information of the second member device in the world coordinate system to convert the position and orientation information of the first member device. It can provide a theoretical basis for the positioning and orientation of any member device in the group, especially for the group platform composed of smaller devices, which only needs to sense and deploy individual member devices, that is, it can solve the position and orientation of any member device through the group collaboration method adopted within the group platform system, and maximize the positioning accuracy on the basis of lightweight platform.

[0089] Figure 3 Schematic diagram of the structure of the group platform system provided by the present invention. Figure 3 As shown, the group platform system provided by the embodiment of the present invention includes a host device 310 at a group center node, and a plurality of smart wearable devices 320 at a group edge node.

[0090] The smart wearable device 320 includes a smart wearable device body 321, and a processor 322, a positioning device 323 and an inertial navigation device 324 arranged in the smart wearable device body 321. Each of the processors 322 is used to forward the absolute position information in the world coordinate system periodically collected by the positioning device 323 to the host device 310.

[0091] If the host device 310 is a remote server terminal, the processor 322 is also used to send a collaborative positioning request carrying the current observation image to the host device 310, and use the posture information of the smart wearable device fed back by the group collaborative positioning method as described in any of the above items when the host device 310 responds to the collaborative positioning request to initialize the inertial navigation device 324 in the smart wearable device to which it belongs or correct the accumulated error.

[0092] Among them, the smart wearable device body 321 includes augmented reality glasses, mixed reality glasses and extended reality glasses.

[0093] Specifically, the group platform system is composed of a smart wearable device 320 at the edge node of the group that continuously senses the user's movement, and a host device 310 at the center node of the group that performs backend processing.

[0094] The smart wearable device 320 is composed of a processor 322, a positioning device 323 and an inertial navigation device 324, which are fixedly arranged on the smart wearable device body 321 and can be communicatively connected. During the movement of the smart wearable device body 321, the positioning device 323 is used to sense the unknown environment in which it is currently located, and the absolute position information in the collected world coordinate system is transmitted to the host device 310.

[0095] Among them, the host device 310 can be deployed in the command center, and by establishing a communication connection with each smart wearable device, the function of periodically collecting the absolute position information of each smart wearable device in the group is realized. When there is a smart wearable device in the group that has a positioning and orientation requirement, the camera deployed inside the smart wearable device body 321 observes other member devices, and the processor 322 deployed therein encapsulates the observed current observation image into a collaborative positioning request and transmits it to the host device 310. The host device 310 executes the group collaborative positioning method as described above to obtain the position information of the smart wearable device that issued the request, so that the smart wearable device that issued the request can provide reference data for the perception data of the positioning device 323 according to its own position contained in the posture information issued by the host device 310, and can also initialize the perception data of the inertial navigation device 324 according to its own posture contained in the posture information issued by the host device 310 or correct the accumulated error.

[0096] It is understandable that the smart wearable device body 321 includes augmented reality glasses, mixed reality glasses and extended reality glasses, and the embodiment of the present invention does not specifically limit the application scenarios of the smart wearable device 310 within the group platform system.

[0097] Exemplarily, each smart wearable device 310 is fixedly provided with an identification element on the surface of its body, and the corresponding smart wearable device body is configured with a positioning device and an inertial navigation device that do not require complicated wiring and assembly. Due to the lightweight of the sensing device, there is a certain error in the sensing result, so the error can be corrected through collaborative positioning between member devices in the group.

[0098] For example, only a few smart wearable devices 310 have fixed identification elements on their surfaces as calibration nodes in the group, and the smart wearable device body serving as the calibration node is equipped with a complete positioning device and an inertial navigation device, while the sensing devices of the remaining smart wearable device bodies that are not equipped with identification elements are all deployed in a lightweight manner. Therefore, the smart wearable device that is not equipped with identification elements can observe the smart wearable device equipped with identification elements through video, and calculate its own more accurate posture through the precise position information of the calibration node, so as to correct the error caused by the lightweight of its own sensing device.

[0099] When responding to the collaborative positioning request issued by the first member device, the embodiment of the present invention identifies all the second member devices observed by the first member device through the identification elements of the current observation image encapsulated in the collaborative positioning request, and then uses the absolute position information of the second member device in the world coordinate system to convert the position and orientation information of the first member device. It can provide a theoretical basis for the positioning and orientation of any member device in the group, especially for the group platform composed of smaller devices, which only needs to sense and deploy individual member devices, that is, it can solve the position and orientation of any member device through the group collaboration method adopted within the group platform system, and maximize the positioning accuracy on the basis of lightweight platform.

[0100] Based on any of the above embodiments, if the host device is any smart wearable device, the processor is also used to execute the group collaborative positioning method as described in any of the above items.

[0101] Specifically, the host device in the group platform system can be any smart wearable device in the group that is in mobile perception. At this time, the nodes in the group are not static, but change dynamically in real time. Each node in the group is separated for a certain period of time and re-networked through wireless signal exchange. Each time, a resolution can be voted to elect the smart wearable device for overall planning as the host device, so that the smart wearable device itself can also complete collaborative positioning.

[0102] The embodiments of the present invention can quickly network multiple smart wearable devices without complex communication pre-planning, have strong scalability, can supplement nodes according to actual needs, can be divided or combined, can deal with certain hostile interference or unintentional interference, and improve the adaptability of the communication networking system to work in a complex electromagnetic environment.

[0103] In addition, the above-mentioned logical instructions can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0104] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the group collaborative positioning method provided by the above methods, the method comprising: receiving and responding to a collaborative positioning request carrying a current observation image sent by a first member device in a group platform system; wherein the first member device is any smart wearable device in the group platform system; the current observation image includes identification elements on other member devices except the first member device; the identification elements are used to distinguish smart wearable devices in the same group platform system; by identifying the identification elements in the current observation image, the absolute position information of each second member device in the world coordinate system is obtained; wherein the second member device is the member device corresponding to the identification element contained in the current observation image; based on the absolute position information of each second member device in the world coordinate system, the posture information of the first member device is calculated, and the posture information of the first member device is transferred to the first member device, so that the first member device can use the posture information of the first member device to initialize the inertial navigation device set in the first member device or correct the accumulated error.

[0105] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the group collaborative positioning method provided by the above-mentioned methods, the method comprising: receiving and responding to a collaborative positioning request carrying a current observation image sent by a first member device in a group platform system; wherein the first member device is any smart wearable device in the group platform system; the current observation image includes identification elements on other member devices except the first member device; the identification elements are used to distinguish smart wearable devices in the same group platform system; by identifying the identification elements in the current observation image, the absolute position information of each second member device in the world coordinate system is obtained; wherein the second member device is the member device corresponding to the identification element contained in the current observation image; based on the absolute position information of each second member device in the world coordinate system, the posture information of the first member device is calculated, and the posture information of the first member device is transferred to the first member device, so that the first member device can use the posture information of the first member device to initialize the inertial navigation device set in the first member device or correct the accumulated error.

[0106] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0107] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A group collaborative positioning method, characterized in that: include: Receiving and responding to a collaborative positioning request carrying a current observation image sent by a first member device in a group platform system; wherein the first member device is any smart wearable device in the group platform system; the current observation image includes identification elements on other member devices except the first member device; the identification elements are used to distinguish smart wearable devices in the same group platform system; Obtaining the absolute position information of each second member device in the world coordinate system by identifying the identification element in the current observation image; wherein the second member device is the member device corresponding to the identification element contained in the current observation image; Based on the absolute position information of each second member device in the world coordinate system, the posture information of the first member device is calculated, and the posture information of the first member device is transferred to the first member device, so that the first member device can use the posture information of the first member device to initialize the inertial navigation device set in the first member device or correct the accumulated error.

2. The group collaborative positioning method according to claim 1, characterized in that: The calculating the position information of the first member device based on the absolute position information of each second member device in the world coordinate system includes: When it is determined that the number of the second member devices is greater than 2, determining, based on the current observed image, projection position information of each second member device in the camera coordinate system; Based on the projection position information of each second member device in the camera coordinate system and the absolute position information of each second member device in the world coordinate system, the direction angle, pitch angle and roll angle in the posture information of the first member device are determined.

3. The group collaborative positioning method according to claim 1, characterized in that: The calculating the position information of the first member device based on the absolute position information of each second member device in the world coordinate system includes: When it is determined that the number of the second member devices is equal to 2, the direction angle in the position and posture information of the first member device is determined according to the absolute position information of the second member device in the world coordinate system.

4. The group collaborative positioning method according to claim 2 or 3, characterized in that: After calculating the position and posture information of the first member device based on the absolute position information of each second member device in the world coordinate system, the method includes: The position and posture information of the first member device is converted into Euler angle information or axis angle information.

5. The group collaborative positioning method according to claim 4, characterized in that: The step of obtaining the absolute position information of each second member device in the world coordinate system by identifying the identification element in the current observation image includes: Based on the current observed image, determine the timestamp of the current observed image and identification information of each second member device; Filter out the satellite positioning information whose timestamp coincides with the timestamp of the current observed image and matches the identification information of the second member device from the satellite positioning information uploaded by each member device in real time, as the absolute position information of the second member device in the world coordinate system; The identification information of any second member device is determined according to the identification element belonging to the corresponding second member device in the current observed image.

6. The group collaborative positioning method according to claim 5, characterized in that: The identification elements include characters, barcodes, QR codes, and color rings set on the smart wearable device.

7. A group collaborative positioning device, characterized in that: include: A request response module, configured to receive and respond to a collaborative positioning request carrying a current observation image sent by a first member device in a group platform system; wherein the first member device is any smart wearable device in the group platform system; the current observation image includes identification elements on other member devices except the first member device; the identification elements are used to distinguish smart wearable devices in the same group platform system; A first positioning module, configured to obtain the absolute position information of each second member device in a world coordinate system by identifying the identification element in the current observation image; wherein the second member device is the member device corresponding to the identification element contained in the current observation image; The second positioning module is used to calculate the posture information of the first member device based on the absolute position information of each second member device in the world coordinate system, and transfer the posture information of the first member device to the first member device, so that the first member device can use the posture information of the first member device to initialize the inertial navigation device set in the first member device or correct the accumulated error.

8. A group platform system, characterized in that: It includes a host device at the center node of the group, and multiple smart wearable devices at the edge nodes of the group; The smart wearable device includes a smart wearable device body, and a processor, a positioning device and an inertial navigation device arranged in the smart wearable device body, each of the processors is used to forward the absolute position information of the positioning device in the world coordinate system periodically collected by the positioning device to the host device; If the host device is a remote server terminal, the processor is further used to send a collaborative positioning request carrying a current observation image to the host device, and use the posture information of the smart wearable device fed back by the group collaborative positioning method according to any one of claims 1 to 6 executed by the host device in response to the collaborative positioning request to initialize the inertial navigation device in the smart wearable device to which it belongs or correct the accumulated error; Among them, the smart wearable device body includes augmented reality glasses, mixed reality glasses and extended reality glasses.

9. The group platform system according to claim 8, characterized in that: If the host device is any smart wearable device, the processor is also used to execute the group collaborative positioning method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the group collaborative positioning method as described in any one of claims 1 to 6 is implemented.

Images