Relative positioning method, relative positioning device, relative positioning system and storage medium

Abstract

The relative positioning method is used for cluster control of movable objects and comprises the steps that at least one frame of detection image of a detected first movable object carrying an optical identifier is obtained through a detection device, and the detection device is arranged on a second movable object; obtaining light identification information of the detected first movable object according to the at least one frame of detection image, and obtaining identity information of the detected first movable object according to the light identification information; and obtaining position and / or attitude information of the second movable object relative to the detected first movable object according to the at least one frame of detection image. According to the invention, under the condition of poor communication signals and the like, a plurality of movable objects can mutually detect the other surrounding movable objects and detect the position and / or attitude information of the movable objects, so that formation performance can be continued to be completed or mutual collision can be avoided in emergency operation execution.

Description

Technical Field

[0001] The present application relates to the field of control, and in particular to a relative positioning method, a relative positioning device, a system and a storage medium. Background Art

[0002] Movable objects (such as aircraft, unmanned aerial vehicles, underwater / aquatic robots, ground smart cars, cleaning robots or self-moving terminals, etc.) can carry different loads to show different purposes. As people's living standards improve, different application scenarios of movable objects are also being explored.

[0003] When large-scale movable objects are used for cluster control, different display effects can be achieved, thereby improving people's user experience. However, existing movable object cluster control requires relying on the Global Navigation Satellite System (GNSS) for positioning. GNSS is susceptible to interference from the atmospheric ionosphere and malicious radio jammers, and is prone to formation accidents due to poor communication quality when forming formations in narrow urban high-rise environments or indoors, which seriously affects the safety of life and property when movable object clusters are used. Therefore, how to complete normal cluster control of movable objects in a GNSS-free environment is a key issue that needs to be solved urgently. Summary of the invention

[0004] Based on this, the present application provides a relative positioning method, system and storage medium, aiming to solve the problem of cluster control of movable objects in a non-GNSS environment.

[0005] In a first aspect, an embodiment of the present application provides a relative positioning method for cluster control of movable objects, comprising: acquiring at least one frame of detection image of a first movable object to be detected and equipped with a light marker through a detection device, wherein the detection device is arranged on a second movable object; obtaining light marker information of the first movable object to be detected based on the at least one frame of detection image, and obtaining identity information of the first movable object to be detected based on the light marker information; and obtaining position and / or posture information of the second movable object relative to the first movable object to be detected based on the at least one frame of detection image.

[0006] In a second aspect, an embodiment of the present application provides a relative positioning device, comprising a processor, wherein the processor is used to perform the following steps: acquiring at least one frame of detection image of a first movable object to be detected and equipped with an optical marker through a detection device, wherein the detection device is arranged on a second movable object; obtaining optical marker information of the first movable object to be detected based on the at least one frame of detection image, and obtaining identity information of the first movable object to be detected based on the optical marker information; and obtaining position and / or posture information of the second movable object relative to the first movable object to be detected based on the at least one frame of detection image.

[0007] In a third aspect, an embodiment of the present application provides a relative positioning system, comprising: the movable object comprises a fuselage and a detection device, the detection device is fixedly or movably mounted on the fuselage, and is used to detect images; a relative positioning device, which is arranged on the fuselage, and is used to implement the aforementioned relative positioning method, and a remote control terminal, which is used to establish a communication connection with the relative positioning device of the movable object.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the above-mentioned relative positioning method.

[0009] The embodiment of the present application provides a relative positioning method, system and storage medium, the method includes: for cluster control of movable objects, including: obtaining at least one frame of detection image of a detected first movable object equipped with a light mark through a detection device, wherein the detection device is set on the second movable object; obtaining the light mark information of the detected first movable object according to the at least one frame of detection image, obtaining the identity information of the detected first movable object according to the light mark information; and obtaining the position and / or posture information of the second movable object relative to the detected first movable object according to the at least one frame of detection image. Through this implementation, when the communication signal is poor or there is no communication signal and the movable object cannot be supported to complete the formation performance, the second movable object can obtain the identity information of the detected first movable object, and obtain the position and / or posture information of the second movable object relative to the detected first movable object, so that multiple movable objects can detect which other movable objects are around each other, and detect the position and / or posture information of each other, so that the formation performance can be continued or avoid collision with each other in the execution of emergency operations.

[0010] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and cannot limit the disclosure of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 is a schematic diagram of a three-dimensional structure of a movable object provided in an embodiment of the present application;

[0013] Figure 2 The embodiment of this application provides Figure 1 One of the schematic diagrams of the side structure of the movable object shown;

[0014] Figure 3 The embodiment of this application provides Figure 1 The second schematic diagram of the side structure of the movable object shown;

[0015] Figure 4 The embodiment of this application provides Figure 1 A schematic diagram of the structure of the movable object as seen from above;

[0016] Figure 5 It is a structural schematic diagram of a relative positioning system for cluster control of multiple movable objects provided by an embodiment of the present application;

[0017] Figure 6 This is one of the flow charts of the relative positioning method provided in the embodiment of the present application;

[0018] Figure 7 It is a structural schematic diagram of a light marker provided in an embodiment of the present application;

[0019] Figure 8 This is the second flow chart of the relative positioning method provided in the embodiment of the present application;

[0020] Fig. 9 is a schematic diagram of a detection image of different light identification information detected by a detection device provided in an embodiment of the present application;

[0021] Fig.10 is a schematic diagram of the relative positions of a detection device and a light marker provided in an embodiment of the present application;

[0022] FIG11 is a schematic diagram of the relative position relationship of multiple movable objects at different times provided in an embodiment of the present application;

[0023] Fig.12 This is the third flow chart of the relative positioning method provided in the embodiment of the present application;

[0024] Fig.13 This is the fourth flow chart of the relative positioning method provided in the embodiment of the present application;

[0025] Fig.14 This is the fifth flow chart of the relative positioning method provided in the embodiment of the present application;

[0026] Fig.15 This is the sixth flow chart of the relative positioning method provided in the embodiment of the present application;

[0027] Fig.16 It is a schematic diagram of the hardware structure of the movable object positioning device provided in an embodiment of the present application;

[0028] Fig.17 It is a schematic diagram of the hardware structure of the relative positioning system provided in the embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0031] Movable objects (such as aircraft, unmanned aerial vehicles, underwater / aquatic robots, ground smart cars, cleaning robots or self-moving terminals, etc.) can carry different loads to show different purposes. As people's living standards improve, different application scenarios of movable objects are also being explored.

[0032] Taking drones (also applicable to aircraft, underwater / aquatic robots, ground smart cars, cleaning robots or self-movable terminals, etc.) as an example, they can perform many different tasks, such as: multiple drones perform formation performances, use multiple drones for logistics transportation or manned transportation, etc., but these scenarios require drones to be able to know whether there are other drones nearby, and how their positions and postures are compared to their own, so that they can successfully complete the tasks they need to perform and avoid collisions or unnecessary losses. It is understandable that drones can include but are not limited to any one of manned drones, logistics drones, aerial photography drones, performance drones, combat drones, and agricultural plant protection drones. The embodiments of this application do not specifically limit the types of the drones.

[0033] At present, the relative positioning of drones can be achieved by GNSS positioning, UWB positioning technology and motion capture systems. However, GNSS is susceptible to interference from the atmospheric ionosphere and malicious radio jammers, and in narrow urban high-rise environments and indoors, it seriously affects the safety of life and property when drone clusters are used. UWB and motion capture systems require frequent communication between drones to measure distance, which places great demands on communication bandwidth. Currently, the number of drones using this method is limited to less than 10 to 20. Moreover, UWB can only provide ranging information, and relative motion must exist between drones to produce sufficient observability, which reduces the safety of drone clusters using this method. It is only suitable for outdoor scenarios and cannot reasonably expand the application scenarios of drone clusters.

[0034] At the same time, the UAV can also be positioned by map positioning (for example: Visual Positioning System, i.e. VPS method). The VPS positioning method includes two steps: offline mapping and online positioning. That is, it is necessary to first use the UAV to detect and record the external environment to establish a map before relying on the map positioning method for positioning in the subsequent actual execution of the task. At the same time, since the UAV may be far away from the ground, the accuracy of the VPS positioning is low, generally around 1 meter. This accuracy can support multiple movable objects to fly from the tower, airport or base station to the predefined performance area, return from the predefined performance area or land in a safe area. However, this accuracy cannot support multiple movable objects to continue to complete the performance or complete the task under interference.

[0035] In summary, in the process of using multiple drones for formation performances, due to the limitations of various factors such as performance scenes, weather environment, detection accuracy and cost, the communication quality in the environment is poor, and it is impossible to support multiple drones to complete formation performances, which greatly affects the user experience. The relevant technologies have more or less shortcomings, so the precise relative positioning technology between drones or robots is the key to safe and intelligent drone clusters.

[0036] Based on this, the present application provides a relative positioning method for cluster control of movable objects.

[0037] First, a feasible structure of the movable object of the present application is introduced, so as to facilitate the subsequent detailed introduction of the relative positioning method of the present application. For details, please refer to Figure 1 The three-dimensional structure diagram of the movable object shown. The movable object may specifically include: a central body 10; a plurality of arms 11, the arms 11 are connected to the central body 10; a plurality of power units 12, the power units 12 are connected to the arms 11; a plurality of protective covers 13, the protective covers 13 are directly or indirectly connected to the central body 10, and the power units 12 are at least partially shielded in the direction away from the central body 10. It can be understood that in some embodiments, the protective cover 13 may not be provided, and only the central body 10, the arms 11 and the plurality of power units 12 are provided. Those skilled in the art may set different reasonable structures of the movable object as needed.

[0038] In some embodiments, multiple arms 11 can be used to connect and support a power unit 12. Specifically, the power unit 12 includes a driving device and a propeller. The driving device can be a motor, which is disposed at one end of the rotating shaft of the propeller. The motor drives the propeller to rotate to generate power, thereby enabling the movement or flight of the movable object. In other embodiments, an engine can also be disposed in the central body, and the force can be transmitted to the propeller of the power unit through the engine and a transmission device to rotate it and generate lift. The present application does not limit this. As long as the device can drive the movable object to move, it is within the protection scope of the present application.

[0039] In some embodiments, the protective cover 14 may be a propeller protective cover, which can be used to protect the propeller to prevent the propeller from being hit by external objects and to prevent the rotating propeller from injuring people.

[0040] In some embodiments, the multiple arms 11 and the multiple protective covers 14 can be an integrally formed structure so that the movable object has good overall strength. Specifically, the multiple arms 11 and the multiple protective covers 14 can be processed into an integrally formed structure by stamping or other processing methods. Optionally, the arms 11 and the protective covers 14 can be split structures, and the arms 11 and the protective covers 14 are connected, so that the structure of a single arm 11 or a propeller protective cover 14 is relatively simple, and the processing technology is correspondingly relatively simple.

[0041] For further information, see Figure 2 , showing Figure 1 One of the side structural diagrams of the movable object shown, referring to Figure 3 , showing Figure 1 The second schematic diagram of the side structure of the movable object shown in FIG. Figure 4 , showing Figure 1 A schematic diagram of the structure of the movable object shown is viewed from above.

[0042] The movable object also includes a detection device 14, and the optional detection device is arranged on the central body 10 of the movable object. Specifically, the detection device 14 may include a plurality of detection elements, and is arranged on the lower side, front side or rear side of the central body 10. Preferably, the detection element can be used in pairs so that the movable object can measure distance or locate based on binocular vision. More preferably, the present application is provided with two detection elements on the lower side, front side (specifically one on the left and right sides of the front side) or rear side (specifically one on the left and right sides of the rear side) of the central body, so that vision can be obtained in multiple directions such as the front, rear, left, right and lower side, and five-way surround view can be achieved. It should be noted that although the movable object of the present application can be provided with a plurality of detection elements, in some embodiments, only one detection element is provided and the technical purpose of the present application can still be achieved.

[0043] In some embodiments, the movable object further includes a light emitting device 15, which is disposed on the lower side of the central body 10 and can be used to emit light of a specified color to enhance the performance effect of the movable object, or to warn the position or flight status of the movable object. In practical applications, the light emitting device 15 can be used to emit a single color or various colors of light for viewing when the movable object performs. In the embodiments of the present application, the light emitting device 15 can also be used as a light mark to assist in the implementation of the relative positioning method described in the present application, so that when the positioning methods such as GNSS are interfered with, relative positioning can still be achieved with the help of the light mark. It can be understood that the light emitting device 15 needs to be used as a light mark in the intervals between the performances of the movable object for detection and identification by other detection devices, so the direct use of the light emitting device 15 has limited improvement on the user experience. Furthermore, an additional light mark can be set on the central body 10 or the arm 11. The detailed scheme will be described later and will not be repeated again.

[0044] See also Figure 5, shows a schematic diagram of the structure of a relative positioning system in which multiple movable objects are cluster-controlled, provided by an embodiment of the present application, wherein multiple movable objects in the relative positioning system, such as the first movable object 101, the second movable object 102, the third movable object 103 and the fourth movable object 104 in the figure, can be used to execute and obtain route tasks from the remote control terminal 20. The remote control terminal 20 includes two route planning methods, namely, offline planning and online planning. Considering that general formation performances require large-scale movable objects to execute different routes to produce the expected performance effect, or multiple mobile objects respectively perform different logistics tasks, manned tasks, agricultural sowing tasks or industry detection tasks, etc., it is preferred to adopt an offline route planning method. Specifically, in the present application, the route tasks of different movable objects are pre-planned by the remote control terminal 20 according to the patterns or videos to be performed by the multiple movable objects. Specifically, in the process of planning the route missions of different movable objects, the number of movable objects, possible route collisions, conflicts or the impact of the surrounding environment on route planning will be considered in advance. Therefore, in theory, when the movable objects perform route missions according to the routes sent by the remote control terminal 20, there will be no collision between each other. At the same time, due to the limitation of the communication bandwidth, only communication between the remote control terminal and the movable objects is required, and there is no need for multiple movable objects to be able to communicate. Therefore, each movable object does not know whether there are other movable objects around it and the specific identity information of the surrounding movable objects. Therefore, if the network condition is interfered by the atmospheric ionosphere and malicious radio jammers, multiple movable objects may drift and deviate from the pre-planned route mission, thereby causing unnecessary damage. Therefore, it is necessary to enable multiple movable objects to be relatively identified and positioned in the absence of GNSS.

[0045] Next, a relative positioning method provided by an embodiment of the present application for cluster control of movable objects will be described in detail. Specifically, the method includes: Figure 6 Steps shown:

[0046] S110: Acquire at least one frame of detection image of a first movable object to be detected and equipped with an optical marker by a detection device, wherein the detection device is disposed on a second movable object.

[0047] Specifically, this relative positioning method is mainly used in the aforementioned relative positioning system, which includes multiple movable objects, such as the aforementioned first movable object 101, the second movable object 102, the third movable object 103 and the fourth movable object 104, etc. Each movable object is equipped with an optical marker, and it can be understood that the optical marker can record optical marker information indicating the identity information of the movable object. For details, continue to refer to Figure 5Taking the first movable object 101 and the second movable object 102 as examples, the first movable object 101 can be detected by using the detection device on the second movable object 102, that is, the optical mark on the detected first movable object can be detected by the second movable object 102. Furthermore, the identity information of the second movable object can be identified by only using one detection element in the second movable object 102 to detect at least one frame of image, thereby reducing the performance requirements for the detection element and effectively reducing the cost of each movable object.

[0048] For the sake of convenience, the subsequent detection devices are all taken as the detection device on the second movable object, so the detection devices referred to below are all detection devices on the second movable object. It can be understood that the second movable object can refer to any one of the multiple movable objects. It can be understood that the first movable object is one or more movable objects other than the second movable object, and this application does not limit this.

[0049] In some embodiments, the optical marker is set on the body (i.e., the central body 10) and / or the arm of the first movable object to be detected, that is, an additional optical marker is set on the aforementioned central body 10 or arm 11. Specifically, the optical marker can be set on the central body 10 and adapted to the upper surface of the central body, so as to cover the upper surface of the central body 10 as much as possible, so that the optical marker can transmit enough optical marker information, which is conducive to being detected by the detection device on the second movable object; in another embodiment, the optical marker can be set on the arm 11, so that the optical marker can be set to a long strip, which is conducive to cooperating with a low-cost line-by-line scanning camera to obtain different optical marker information.

[0050] In some embodiments, the light mark is a lamp body, such as Figure 7 As shown in (a), the light marker is a lamp body of a circular, square, rectangular or any shape, and is encoded in its time domain.

[0051] In some embodiments, Figure 7 As shown, the light mark can be a lamp composed of multiple light-emitting bodies, and multiple light-emitting bodies can form a long strip light ( Figure 7 (b)) or QR code light ( Figure 7 (c)) By controlling the on / off state or frequency state of different light sources, different digital or coded signals can be transmitted, thereby helping to identify and distinguish the identity information of different movable objects.

[0052] Specifically, when the light mark is a long light bar composed of multiple light-emitting bodies, it is necessary to encode the multiple light-emitting bodies in the same time domain at the same time, and more specifically, encode the frequencies of the light-emitting bodies in the same time domain, so that the detection device can more easily detect the light mark and obtain the light mark information. In this embodiment, the frequency can be encoded in the time domain, and the encoding method is simpler and easier to decode. At the same time, it is helpful to cooperate with the rolling shutter to achieve a single frame to obtain the light mark information of the detected movable object.

[0053] Specifically, when multiple light-emitting bodies form a two-dimensional code light (array light), they need to be encoded in geometric space, that is, the light and dark state of each light-emitting body is encoded in geometric space, so that when a global camera is used to detect the light identification information, only one frame of image is needed to obtain the light identification information. In this embodiment, the space (especially the upper surface) of the central body 10 of the movable object can be fully utilized, and as many light identifications of light-emitting bodies as possible can be set according to the shape of the movable object, so as to express richer identity information, help improve the relative positioning accuracy, and enhance the adaptability of the relative positioning method to larger-scale movable objects.

[0054] In some embodiments, the optical identification may include a lamp. Specifically, the optical identification may be a visible light lamp, that is, a white light composed of any light beam in the wavelength range of 340nm-760nm, such as a visible light lamp emitted by an LED light board. Correspondingly, the detection device is a visible light detector. In some embodiments, the lamp is an invisible light lamp, which may be an infrared lamp or an ultraviolet lamp. Such invisible light lamps cannot be seen by the human eye, but can be recognized by the corresponding detection device, thereby assisting in identifying the identity information of the detected movable object. Correspondingly, the detection device is a non-visible light detector or a visible light detector without a non-visible light filter (infrared filter or ultraviolet filter). Further, the invisible light lamp may also be a narrow-band light source (such as a laser light source) with good monochromaticity, which is used to emit light within a certain wavelength range, such as a narrow-band infrared light source or a narrow-band ultraviolet light source. Correspondingly, a bandpass filter is also provided on the detection device on the movable object, which can output light of a wavelength band including the narrow-band light source, and the bandpass filter may be a narrow-band filter. If the invisible light lamp can emit light with a central wavelength of 1080nm, the bandpass filter device can be set to pass light in the wavelength range of 1070-1090nm, or a narrow-band filter with a specification of 1080nm can be selected, so that the detection device can capture the light emitted by the invisible light lamp and filter out the light emitted by other lamps (such as performance light groups), thereby reducing the interference of other lamps and improving the accuracy of identifying landmarks. After capturing and filtering, the center of the light spot can be extracted by the visual recognition algorithm of the movable object. In this embodiment, since the invisible light lamp is used as the optical marker of the movable object, it can achieve relative positioning between multiple movable objects without affecting the overall formation effect.

[0055] S120: Obtaining light identification information of the detected first movable object according to the at least one frame of detection image, and obtaining identity information of the detected first movable object according to the light identification information.

[0056] Specifically, the implementation of the present application can obtain the optical identification information of the detected first movable object with only one frame of detection image, and obtain the identity information of the detected first movable object according to the one frame of detection image. Figure 8 The steps shown include:

[0057] S1201: The light marker is encoded according to a first preset rule, and the at least one frame of detection image includes the light marker information of the light marker encoded according to the first preset rule.

[0058] Among them, the first preset rule can be a mixture of at least two of tone coding, frequency coding, Manchester coding, QR code or sound. Furthermore, the first preset rule is used to encode the light identification in the time domain. Furthermore, the first preset rule is tone coding, and the light identification information is encoded into the light identification through tone coding, that is, the flashing frequency of each light is different (usually in several thousand Hz or higher), and such a flashing frequency exceeds the observation limit of the human eye. Therefore, through such a coding method, the identity information of the detected first movable object can be transmitted without affecting the user experience to assist multiple movable objects in relative identification and positioning. More preferably, the first preset rule can be used to encode the aforementioned long strip light in the time domain. That is, the identity information corresponding to the first movable object is pre-encoded into the light identification information in the time domain according to the tone coding rule. Specifically, the light identification information is expressed as different frequency information and presented by the light identification, and is displayed as different strip patterns after being detected by the detection device. Specifically, as Fig. 9 (a) and (b) show the optical marks taken by the detection device at different coding frequencies ( Figure 7 (a) or Figure 7 (b) The corresponding detection fringe image of the light marker after time domain encoding.

[0059] In some embodiments, for example, a plurality of light-emitting bodies are m×n array lights, and m and n are positive integers, wherein m is the number of rows and n is the number of columns. Further, when m≤n and m=1, that is, one light-emitting body or a plurality of light-emitting bodies form a long strip light, the detection device is a rolling camera, and the length direction of the pattern formed by the one or more light-emitting bodies on the detection device at least does not overlap with the scanning direction of the rolling camera. In some embodiments, there is an angle between the length direction (specifically, the length direction is the direction where x is located in the figure) of the pattern formed by the one or more light-emitting bodies on the detection device (it can be understood that this is also the imaging plane of one or more light-emitting bodies) and the scanning direction of the rolling camera. More specifically, the inventor discovered through creative labor that Fig.10As shown, when the length direction of the pattern formed by one or more light sources on the detection device is at an angle to the perpendicular direction of the scanning direction of the rolling camera, the shortcomings of the rolling camera itself (since only one row of pixels is captured each time, moving up or down on the grid to construct a complete image, there may be distortion or blurring for fast-moving objects, but a row of pixels can be captured quickly. That is, the distortion of the rolling shutter is generally considered to have a negative effect on the perception technology of movable objects. Unable to "deal with" an inaccurate description or a shot, the rolling shutter will produce slight or severe deformation regardless of the high-speed movement of the photographer himself (mounted on a moving object such as an airplane, a drone or a car) or the high-speed movement of the photographed target. However, in fact, the above-mentioned slight or severe deformation depends on the exposure speed and the relative movement speed. When the exposure time is short and the relative movement speed is small, this error can actually be ignored.) can be converted into advantages. By controlling the exposure time and the relative movement speed, the detection image of the movable object carrying different light markers can be quickly detected, which greatly overcomes the above-mentioned technical prejudice of technicians in this field.

[0060] More specifically, in this embodiment, the rolling shutter effect of the rolling camera can be utilized, in conjunction with a light source or multiple light sources that are progressively encoded in the time domain to form a long strip light, and different stripe patterns encoded according to a first preset rule can be captured during different rolling shutter periods, thereby obtaining different light identification information. More specifically, when the detection device of the second movable object captures the light identification (light) of other movable objects (such as the first movable object being detected), since the shutter of the rolling camera scans line by line, when the scanning speed is appropriate and the light identification flashes at a high speed with a certain pattern (frequency encoding at a fixed frequency), it can be recorded that the light identification presents different brightness in different areas, so that the different light identifications captured by the rolling camera can be used to record multiple movable objects (such as Fig. 9(a), (b)). In this embodiment, due to the existence of the rolling shutter effect, the rolling shutter camera only needs to detect one frame of image to record the light identification information of the currently detected movable object, thereby avoiding the need to detect multiple frames of image (in the related art, the global camera generally needs to detect multiple frames of image to calculate the flicker of the light. For example, in the first light detected by the global camera, 4 frames out of 10 frames of image are bright, then its flicker is 40%, and in the second light detected by the global camera, 6 frames out of 10 frames of image are bright, then its flicker is 60%. In this way, the flicker calculation is realized. However, this method requires continuous tracking of the detected movable object, and its stability is poor and the algorithm is difficult. Once the movable objects in the field of view of the detection device block each other or are blocked by obstacles, it may cause statistical errors in relative positioning) to record the light identification information of the detected movable object, which is susceptible to interference between multiple movable objects when they move relative to each other. That is to say, since the encoding of the light mark according to the first preset rule and the rolling shutter effect can transmit more information, more complex digital codes can be distinguished in one picture. Therefore, the movable object cluster applicable to this embodiment can be expanded to hundreds or thousands. More specifically, in combination with Fig. 9 , Fig.10 , the fringe pattern of one or more luminous bodies on the imaging plane where the detection device is located conforms to the following relationship:

[0061] (h / d)×f c ≈n×a and n×a≥n min ×a min ; (1)

[0062] Where, h represents the actual height of the luminous body (in meters), d represents the distance of the detection device relative to the light mark, n represents the number of stripes of the luminous body on the imaging plane of the detection device, a represents the number of pixels occupied by each stripe, and f represents the distance between the luminous body and the light mark. c Indicates the focal length of the detection device. At the same time, it also meets the following requirements:

[0063] f≤h c / (2×t au ×a min ); (2)

[0064] Where, f represents the flashing frequency of the light mark, h c Indicates the number of vertical pixels of the pattern formed by the luminous body on the imaging plane, t au represents the exposure time, a min Indicates the minimum number of pixels each stripe occupies.

[0065] In summary, the number of pixels occupied by each stripe a is related to the distance d from the light marker to the detection device and the focal length f of the detection device itself.c It depends only on the scanning speed of the detection device (h c / t au ) and the speed of the light flashing of the light mark. At the same time, the number of stripes (the number of stripes n) that the light mark can image on the imaging plane depends on the distance d from the light mark to the detection device and the focal length f of the detection device. c and the minimum stripe size a min . In some embodiments, amin≥5 pixels, thereby ensuring effective decoding. That is to say, the frequency relationship can be adjusted by adjusting the frequency range of the light coding or the camera exposure time. If the light mark has a sufficient size or the detection device has enough pixels, the distance relationship can be adjusted. In this embodiment, by using a rolling camera and reasonably controlling the frequency range of the light coding or the camera exposure time, that is, controlling the length direction of the pattern formed by the light mark on the detection device to at least not coincide with the scanning direction of the rolling camera, the coded pattern of the light mark can be obtained through a frame of image, which helps to quickly identify the identification information of the detected object. At the same time, with the help of this frame of image, the direction and distance of the detected light mark from the current detection device can also be obtained, which greatly improves the accuracy and adaptability of the relative positioning method.

[0066] When m>1 and m≤n, the detection device is a global camera (or area array camera). Compared with the aforementioned related art that uses a global camera to capture the dynamic flickering frequency of light to identify it, the global shutter of the global camera actually captures a dynamic video frame. In this embodiment, since the number of rows m and the number of columns n of the light-emitting body satisfy m>1 and m≤n, the global camera only needs to capture one frame of image to obtain the light identification information of the detected movable object, that is, the one frame of image records the two-dimensional code information composed of multiple light-emitting bodies, and the light identification information of the detected movable object can be directly obtained by using the two-dimensional code information composed of the multiple light-emitting bodies, thereby avoiding the problem that the dynamic video frame method is easily affected by the interference between multiple movable objects when they move with each other.

[0067] In another embodiment, when m>1 and m≤n, a rolling camera can also be used as a detection device. At this time, only one or a row of light-emitting bodies in the m rows (the implementation method of a row needs to be encoded at the same time and the encoding content is the same) is used as the encoding basis and time domain encoding is performed on it rather than geometric space encoding. The rolling shutter effect of the rolling camera can still be used to realize the encoding and identity information recognition of different movable objects. The specific implementation method can refer to the introduction of the aforementioned rolling shutter related embodiments, which will not be repeated here.

[0068] S1202: Decode the light identification information included in the at least one frame of detection image according to a second preset rule to obtain the identity information of the detected first movable object.

[0069] Specifically, the second preset rule may be a decoding rule corresponding to the first preset rule. For example, when the first preset rule is tone coding, the second preset rule may be Fourier transform or neural network. In this way, the light identification information of the tone-coded light identification can be solved, that is, the flashing frequency information of the light identification, and then the identity information of the detected movable object can be obtained according to the frequency.

[0070] In some embodiments, the optical identification information includes at least identity information, and the identity information of the detected first movable object is obtained based on the optical identification information of the detected first movable object and a first identity database. In this embodiment, identity recognition can be performed with the help of the first identity database composed of the optical identification information compiled when the optical identification is encoded.

[0071] In some embodiments, in theory, a large number of optical markers can be mounted on the movable object to assist in achieving relative positioning control of more movable objects. For example, if the number of luminous bodies of the optical markers mounted on the movable object is n, then 2 n movable objects are encoded. However, due to the limited number of light identifications that can be presented on movable objects, the above n cannot be infinite as in theory. For example, in a cluster of movable objects, the light identification code can only encode 100 movable objects, while there are 1,000 movable objects actually used for performances, formations or logistics transportation. Therefore, there must be reuse of coding numbers (that is, light identification information). At this time, the first identity database is needed to determine which movable object has the same coding number, that is, its exact identity information.

[0072] Specifically, assuming that the remote control terminal pre-plans routes for multiple movable objects according to the pattern to be performed or the control command, the route of each movable object includes multiple waypoints. As shown in FIG11(a), at the first moment, the remote control terminal configures different light identification information for the movable object according to its identity information, wherein the movable object 101 in the A area is configured with a code of 0001, and the movable object 111 in the H area is also configured with a number of 0001. However, since the distance between the A and F areas is relatively far at the current moment, it will not interfere with the recognition and relative positioning of the movable object coded as 0001 by other movable objects in the A and F areas. However, at the second moment, as shown in FIG11(b), multiple movable objects are located at their respective other waypoints. At this time, the waypoints of the movable object coded as 0001 located in area A at the first moment and the movable object coded as 0001 located in area H at the first moment may both be located in area F, which may result in the inability to identify the specific identity information of movable object 0001 in area H, that is, the remaining nearby movable objects cannot distinguish whether the identity information of the currently detected movable object is movable object 101 located in area A at the first moment or movable object 111 in area H, thereby causing ambiguity, resulting in misidentification and affecting the control effect of the movable object.

[0073] Therefore, at this time, it is necessary to use the first identity database for specific identity recognition. Specifically, in some embodiments, the remote control terminal avoids movable objects with the same number appearing in adjacent areas when planning the route in advance. For example, the first identity database can be shown in Table 1 below:

[0074] Table 1 First identity database 1

[0075]

[0076] Among them, in the first identity database 1, when planning the route for the movable object with the code number 0001 and the serial number 1, its waypoint position is restricted to appear in the A area and the vicinity of the A area (the B area or the E area) as much as possible, and avoid appearing in the H area. When planning the route for the movable object with the code number 0001 and the serial number 81, its waypoint position is restricted to appear in the H area and the vicinity of the H area (the G area or the D area) as much as possible, so as to avoid the distance between different movable objects with the same number being too close during the route planning process, thereby affecting the mutual recognition and positioning of multiple movable objects. As shown in Figure 11 (c), the movable object with the code number 0001 located in the B area is far away from the movable object with the code number 0001 located in the G area, and will not cause misidentification. Therefore, if movable objects with the same code are identified simultaneously during the relative identification process, the first identity database can be used to determine which of the movable objects with the same code is a movable object near the current movable object (e.g., the second movable object 102) (e.g., the first movable object detected: movable object No. 101), and which one is a movable object outside the surroundings of the current movable object (e.g., the second movable object 102) (e.g., movable object No. 81 111).

[0077] In another embodiment, the following first identity database 2 may also be used for identity recognition. Specifically, the code number of each movable object changes dynamically, as shown in the following table of the first identity database 2.

[0078] Table 2 First identity database 2

[0079]

[0080] Among them, the movable object No. 1 and the movable object No. 81 have the same number at time t0, but since their respective coding numbers are dynamically changing, even if the movable object No. 1 and the movable object No. 81 are close to each other at a certain waypoint, since their coding numbers at the waypoint time (for example, time t2) are different, it will still not cause misidentification.

[0081] It can be understood that those skilled in the art can set the specific form of the first identity database as needed, as long as the database can assist in identifying the specific identity information of the movable object, and the first identity database can be entered or sent to multiple movable objects during the process of route planning for each movable object at the remote control end.

[0082] To sum up, with the help of the first identity database, it is possible to avoid the close distance of movable objects with the same number when planning routes or to avoid misidentification of multiple movable objects by using dynamic coding, thereby effectively improving the judgment errors of multiple movable objects when forming a formation or performing tasks, greatly improving the user experience.

[0083] In some embodiments, the optical identification information includes at least the identity information, position information and / or posture information of the first movable object being detected, and the first movable object being detected broadcasts the optical identification information through optical communication to obtain the identity information of the detected movable object. In this embodiment, in the process of encoding the optical identification, the identity information, position information and posture information of the movable object can all be encoded into the optical identification, and a rolling shutter camera with a higher frame rate is used as a detection device to obtain the optical identification information, thereby transmitting more information about the movable object with the help of the optical communication mode, which helps to relatively locate multiple movable objects.

[0084] In some embodiments, Fig.12 As shown, before step S110, the following steps are also included:

[0085] S1401: Obtaining the frame rate of the detection device;

[0086] S1402: Determine whether the frame rate of the detection device is greater than a first threshold;

[0087] S1403: When the frame rate of the detection device is greater than a first threshold, the light mark and other information other than the light mark are encoded at different times according to different preset rules.

[0088] That is to say, the encoding rule of the light identification can be determined according to the frame rate of the detection device. For example, the first threshold is 20FPS. If the frame rate of the detection device is less than 20FPS, the light identification is encoded according to the first preset rule, and the light identification information can only include identity information. The specific implementation method can be referred to the above introduction, which will not be repeated here.

[0089] When the frame rate of the detection device is greater than 20FPS, the light identification can be encoded according to different preset rules in different shutter cycles. Specifically, in the first shutter cycle, the light identification is encoded according to the first preset rule, and the light identification information includes at least identity information; in the second shutter cycle, the light identification is encoded according to the third preset rule, and the light identification includes the current estimated position information (approximate position information) and movement status information (such as flight speed) of the first movable object being detected. That is, different information can be acquired in time-sharing using a high frame rate detection device. Specifically, the third preset rule is different from the first preset rule, so that different information can be encoded and transmitted. Optionally, the third preset rule can also be phase encoding or tone encoding.

[0090] In this embodiment, the detection device is a rolling shutter camera with a high frame rate, so that the identity information of the movable object is transmitted through optical communication in the first shutter cycle, and the position information of the movable object and the posture information of the movable object can also be transmitted through optical communication in the second shutter cycle. Therefore, relying only on the rolling shutter with a high frame rate, the identity information, position information and posture information of the movable object transmitted by optical communication can be recorded and detected. Even when facing a movable object that is not pre-controlled by the current remote control terminal, this optical communication method can be used to quickly exchange information and perform tasks such as coordination and mutual obstacle avoidance.

[0091] It should be noted that, in the present embodiment, the position information transmitted in the second shutter period is specifically the approximate position information of the movable object itself relative to the ground identified by binoculars and maps, and the attitude information is the attitude information detected by the movable object itself based on its own different sensors. The transmission of this part of information in the first shutter period and the second shutter period can directly send accurate optical identification information or other information without relying on the aforementioned first identity recognition library, so as to achieve quasi-real-time mutual communication or duplex communication. For example, it is assumed that the first movable object and the second movable object can know the first relative position relationship between the first movable object and the second movable object by means of the aforementioned relative positioning method, especially the optical identification information transmitted during the first shutter period (including the following S130 step), but the first relative position relationship may not be accurate enough and can only reach the quasi-centimeter level. During the second shutter period, the first movable object sends (optical communication mode) the approximate position (absolute position, meter level) information of the first movable object relative to the ground to the second movable object, and the second movable object sends (optical communication mode) the approximate position information (absolute position, meter level) of the second movable object relative to the ground to the first movable object. Then, by fusing the approximate position (absolute position, meter level) of the first movable object relative to the ground, the approximate position information (absolute position, meter level) of the second movable object relative to the ground and the aforementioned first relative position relationship, the second relative position relationship can be obtained, and the accuracy of the second position relationship can reach the centimeter level. Thus, with the help of a rolling shutter camera with a high frame rate, the identity information of the movable object is transmitted through optical communication in the first shutter period, and the position information of the movable object and the posture information of the movable object can also be transmitted through optical communication in the second shutter period, which greatly improves the accuracy of relative positioning between multiple movable objects and realizes a task similar to multiple movable objects performing autonomous movement. It can be understood that other information can also be sent in the second shutter period, and those skilled in the art can set it according to their own needs, and this application does not limit this.

[0092] S130: Obtaining position and / or posture information of the second movable object relative to the detected first movable object according to the at least one frame of detection image.

[0093] Specifically, in some implementations, S130 includes:

[0094] The position and / or posture information of the second movable object relative to the detected first movable object is obtained according to the at least one frame of detection image and the identity information of the detected first movable object.

[0095] In this embodiment, for example, at least one frame of image includes multiple light identification information, that is, there may be multiple movable objects in one image. It is necessary to identify the specific identity of the current movable object before determining whether the current position and posture of the second movable object relative to the detected movable object are the position and posture of the second movable object relative to the first movable object. Specifically, for example, the second movable object is movable object No. 2, and the detected image includes 2 movable objects (for example, No. 1 and No. 3). The distance and posture information of movable object No. 2 to the two movable objects in the image are calculated by the light identification information in at least one frame of image, and are L1, p1 and L2, p2 respectively, but it is not known which of No. 1 and No. 3 corresponds to L1, p1, and which corresponds to L2, p2. Therefore, it is necessary to know, for example, that No. 1 is the first movable object detected, and the distance of movable object No. 1 from movable object No. 2 is L1, and the relative posture is p1 based on at least one frame of detection image and the identity information of the first movable object detected. In this embodiment, based on the at least one frame of detection image and the identity information of the detected first movable object, when multiple detected objects are included in at least one frame of image, their respective distances and postures can be identified, which helps to improve the precision and accuracy of relative positioning.

[0096] In some other implementations, S130 includes:

[0097] Based on the identity information of the detected first movable object and the position and size of the optical identification information of the detected first movable object in at least one detection image, the position and / or posture information of the second movable object relative to the detected first movable object is obtained. Specifically, based on the position and size of the optical identification information of the detected first movable object in at least one detection image and the camera parameters of the detection device, the position and / or posture information of the second movable object relative to the detected first movable object is obtained.

[0098] Wherein, assuming that the dimension of the image is X×Y, the position of the light identification information in at least one detection image is expressed as: the center coordinate is (x1, y1), and the area is f(x1, y1; r), where r is the radius or side length (in pixels) of the light identification information in at least one detection image, and the size of the identification information in at least one detection image is expressed as the number of pixels occupied by f(x1, y1; r). Wherein, the camera parameters include internal parameters or external parameters. Specifically, the internal parameters are parameters related to the characteristics of the camera itself, such as the focal length and pixel size of the camera; the external parameters are parameters in the world coordinate system, such as the position and rotation direction of the camera. In this embodiment, the distance (i.e., position information) and azimuth angle (i.e., posture information) of the detected movable object relative to the current detection device can be obtained based on the camera external parameters combined with the position and size of the aforementioned light identification information in at least one detection image.

[0099] In some other implementations, S130 may include:

[0100] The detection device includes at least two detection elements, and the position and / or posture information of the second movable object relative to the first movable object being detected is obtained based on the identity information of the detected movable object and the optical identification information of the first movable object being detected photographed by the at least two detection elements. The specific structure of the movable object can be found in the above introduction. That is, the binocular vision principle or the panoramic vision principle is used to simultaneously observe the same detected movable object, and the triangulation principle is combined to restore the position information and posture information of the detected movable object relative to the second movable object where the detection device is located.

[0101] In some embodiments, after obtaining the identity information of one or more detected movable objects near the second movable object by using the above relative positioning method, and the position and posture information relative to the second movable object, the aforementioned mutual communication between the movable objects (in the aforementioned relative positioning method, the light identification, etc. is real-time communication and mutual exchange) and the routes or flight trajectories of other movable objects pre-entered in the movable object by the remote control terminal (such as a ground control station, which can use a communicative control device such as a mobile phone, a computer, a mobile tablet, or a cloud server) can be combined with the Kalman filtering method, distributed state estimation and other methods to further estimate the state and relative position of multiple movable object clusters, so as to avoid collision or drift between the movable object clusters in the subsequent execution of tasks. That is to say, in a large-scale movable object cluster, each movable object solves the position of other movable objects within its field of view, and the relative position of any two movable objects can be obtained. This algorithm enables the movable object cluster to perform relative positioning without communication or only a small amount of communication, without relying on other positioning information (although other information can be used for fusion and enhancement). This allows clusters of movable objects to complete dense formations or other tasks when GNSS is interfered with, such as performing performances without GNSS.

[0102] It should be noted that, in the present embodiment, the reason for pre-recording the routes or flight trajectories of other movable objects inside the movable object is that due to the limitations of state estimation algorithms such as the Kalman filtering method and distributed state estimation, in a non-GNSS environment, the movable object is completely unable to accurately execute this preset route, and over time, there may be errors at the meter level or even tens of meters level. This makes it impossible for the cluster of movable objects to perform tasks. Therefore, after adopting this embodiment, the current movable object can correct the flight or movement of the current movable object through the pre-recorded trajectories of "where it should be" of other movable objects and the detection of relative positioning, so that the current movable object is closer to the preset trajectory.

[0103] In some embodiments, Fig.13 As shown, before step S110, it also includes:

[0104] S1501: Obtain current communication status;

[0105] S1502: Determine whether the communication status satisfies a first condition, where the first condition at least includes that the number of frequency bands interfered with by network signals is less than a first interference threshold;

[0106] S1503: If the communication status does not satisfy the first condition, controlling the movable object to perform an emergency operation.

[0107] Among them, since the communication conditions are easily blocked by high-rise buildings or interfered by jammers, the relative positioning method of the present application can enable multiple mobile objects to continue to perform or complete tasks when they are interfered. However, when the interference is too serious, in order to ensure the safety of the movable objects and the ground, it is necessary to control the movable objects to perform emergency operations, such as return operations or landing in a safe area. Among them, it can be determined whether it is necessary to control the movable object to perform emergency operations by detecting the number of frequency bands interfered with by the network signal. Specifically, the first interference threshold is: 2.4+5.8 The full frequency band of the jammer gun is interfered. If it is less than this first interference threshold (for example, certain frequency points are interfered, and the image transmission between the remote control terminal and the movable object can be frequency-hopped to the available frequency point, which is a poor network signal at this time), the performance can be continued or the task can be completed. If it is greater than the first interference threshold, the movable object is controlled to perform the aforementioned emergency operation, so that the present embodiment can flexibly adjust the positioning method and the priority of task execution according to the network conditions to avoid causing losses to the movable object or affecting the user experience.

[0108] In some embodiments, Fig.14 As shown, before step S110, it also includes:

[0109] S1601: Determine whether a second condition is met, where the second condition includes: the number of the movable objects is less than a first number threshold, and the distance between the movable objects and the reference object is less than a first distance threshold;

[0110] S1602: When the second condition is met, relative positioning is achieved according to the map and / or the relative positioning method.

[0111] In this embodiment, it can be determined whether to use only the aforementioned relative positioning method according to the number of movable objects. When the number of movable objects is small and the distance from the reference object (such as the ground) is close, for example, the first number threshold is 5, and only 5 movable objects are controlled to form a formation or perform agricultural or logistics, industry detection tasks, relative positioning can be achieved by integrating the map. For example, a map is established by relying on the vision (binocular vision method) or laser sensor carried by the movable object, and the 5 movable objects can be relatively positioned according to the map positioning information. That is to say, when the second condition is met, the map positioning information can be integrated into the relative positioning method of the present application to further improve the accuracy of relative positioning. More specifically, the anchor point established by this map positioning method can be relied on to improve the accuracy of relative positioning in this embodiment. For example, for a movable object at the edge of a plurality of movable objects, the position calculated according to the nearby movable objects is a relative position, which may have errors. At this time, with the help of the anchor point provided by the map positioning method (for example, the position relative to the ground, that is, the absolute position), it can ensure that the relative positioning of multiple movable objects and the absolute positioning relative to the ground are accurate.

[0112] In some embodiments, Fig.15 As shown, before step S110, it also includes:

[0113] S1701: Determine whether a third condition is satisfied, wherein the third condition includes: the number of the movable objects is less than a second number threshold, and there is relative motion between the movable objects;

[0114] S1702: When the third condition is met, relative positioning is implemented according to ultra-wideband positioning and / or the relative positioning method.

[0115] In this embodiment, it can also be decided whether to only use the aforementioned relative positioning method based on the number of movable objects. When the number of movable objects is less than the second number threshold (for example, 20) and there is relative movement between multiple movable objects, relative positioning can be achieved by integrating ultra-wideband positioning. For example, ultra-wideband positioning uses a visual observation odometer to detect and locate light markers, and fuses the information obtained by the visual observation odometer with the information obtained by the aforementioned relative positioning method to obtain the relative position of the movable objects with higher accuracy.

[0116] In the above implementation manner, other positioning methods, such as a map positioning method or an ultra-wideband positioning method, may be integrated into the relative positioning method of the present application to further increase the accuracy of relative positioning.

[0117] In an embodiment of the present application, at least one detection image of a detected first movable object equipped with a light mark is obtained by a detection device, wherein the detection device is arranged on the second movable object; the light mark information of the detected first movable object is obtained according to the at least one detection image, and the identity information of the detected first movable object is obtained according to the light mark information; and the position and / or posture information of the second movable object relative to the detected first movable object is obtained according to the at least one detection image. Through this embodiment, when the communication signal is poor or there is no communication signal and the movable object cannot be supported to complete the formation performance, the second movable object can obtain the identity information of the detected first movable object, and obtain the position and / or posture information of the second movable object relative to the detected first movable object, so that multiple movable objects can detect which other movable objects are around each other, and detect the position and / or posture information of each other, so that the formation performance can be continued or collision can be avoided in the execution of emergency operation.

[0118] Furthermore, since the shutter of the rolling camera is scanned line by line, by using multiple light-emitting bodies to form a long light strip, the detection device is a rolling camera, and the length direction of the pattern formed by the light-emitting bodies on the detection device is at least not overlapped with the scanning direction of the rolling camera. When the scanning speed is appropriate, it can be recorded that the light mark presents different brightness in different areas, so that multiple movable objects can be recorded using different light marks taken by the rolling camera. In this embodiment, due to the existence of the rolling shutter effect, the rolling camera only needs to detect one frame of image to record the light mark information of the movable object currently being detected.

[0119] Based on the relative positioning method described in any of the above embodiments, the present disclosure further provides a relative positioning device, including a processor, wherein the processor is configured to perform the following steps:

[0120] Acquiring at least one frame of detection image of the first movable object to be detected and equipped with the optical marker by means of a detection device, wherein the detection device is arranged on the second movable object;

[0121] Obtaining light identification information of the detected first movable object according to the at least one detection image frame, and obtaining identity information of the detected first movable object according to the light identification information; and

[0122] The position and / or posture information of the second movable object relative to the detected first movable object is obtained according to the at least one frame of detection image.

[0123] In some embodiments, the processor is further configured to execute: the light identification is encoded according to a first preset rule, and the at least one frame of detection image includes the light identification information of the light identification encoded according to the first preset rule.

[0124] In some embodiments, the processor is further configured to execute: decoding the light identification information included in the at least one frame of detection image according to a second preset rule to obtain the identity information of the detected first movable object.

[0125] In some embodiments, the processor is also used to execute: the optical identification information includes at least identity information, and the identity information of the detected first movable object is obtained based on the optical identification information of the detected first movable object and a first identity database, and / or the optical identification information includes at least the identity information, position information and / or posture information of the detected first movable object, and the detected first movable object broadcasts the optical identification information via optical communication to obtain the identity information of the detected movable object.

[0126] In some embodiments, the processor is also used to execute: obtaining the frame rate of the detection device; determining whether the frame rate of the detection device is greater than a first threshold; when the frame rate of the detection device is greater than the first threshold, encoding the light identification and other information other than the light identification according to different preset rules at different times.

[0127] In some embodiments, the processor is also used to execute: in a first shutter cycle, the light identification is encoded according to a first preset rule, and the light identification information includes at least identity information; in a second shutter cycle, the light identification is encoded according to a third preset rule, and the light identification includes the current estimated position information and movement status information of the detected first movable object.

[0128] In some embodiments, the processor is further used to execute: obtaining the position and / or posture information of the second movable object relative to the detected first movable object based on the at least one detection image frame and the identity information of the detected first movable object.

[0129] In some embodiments, the processor is also used to execute: obtaining the position and / or posture information of the second movable object relative to the detected first movable object based on the identity information of the detected first movable object and the position and size of the optical identification information of the detected first movable object in at least one detection image, and / or the detection device includes at least two detection elements, and obtaining the position and / or posture information of the second movable object relative to the detected first movable object based on the identity information of the detected movable object and the optical identification information of the detected first movable object captured by the at least two detection elements.

[0130] In some embodiments, the processor is also used to execute: obtaining the position and / or posture information of the second movable object relative to the detected first movable object based on the position, size and camera parameters of the detection device of the optical identification information of the detected first movable object in at least one detection image.

[0131] In some embodiments, the processor is also used to execute: obtaining the current communication status; determining whether the communication status satisfies a first condition, the first condition at least including that the number of frequency bands interfered with by the network signal is less than a first interference threshold; if the communication status does not satisfy the first condition, controlling the movable object to perform an emergency operation.

[0132] In some embodiments, the processor is also used to execute: determining whether a second condition is met, the second condition including: the number of the movable objects is less than a first number threshold, and the distance between the movable objects and the reference object is less than a first distance threshold; when the second condition is met, relative positioning is achieved according to the map and / or the relative positioning method; and / or determining whether a third condition is met, the third condition including: the number of the movable objects is less than a second number threshold, and there is relative motion between the movable objects; when the third condition is met, relative positioning is achieved according to ultra-wideband positioning and / or the relative positioning method.

[0133] In some embodiments, the optical marker is disposed on a body and / or an arm of the detected first movable object.

[0134] In some embodiments, the optical marker comprises a lamp, the lamp is a visible light lamp or an invisible light lamp, the invisible light lamp is an infrared or ultraviolet lamp, and the detection device is a visible light detector or a non-visible light detector.

[0135] In some embodiments, the light indicator is a lamp composed of one or more light-emitting bodies, and the multiple light-emitting bodies are m×n array lamps, and m and n are positive integers, where m is the number of rows and n is the number of columns.

[0136] In some embodiments, when m < n and m < 2, the detection device is a rolling shutter camera, and the length direction of the pattern formed by the light emitter on the detection device does not coincide with the scanning direction of the rolling shutter camera at least;

[0137] In some embodiments, when m > 1 and m ≤ n, the detection device is a global shutter camera or a rolling shutter camera.

[0138] Fig.16 FIG. shows a more specific schematic hardware structure diagram of a movable object positioning device provided by an embodiment of this specification. The device may include: a processor 310, a memory 320, an input / output interface 330, a communication interface 340, and a bus 350. Among them, the processor 310, the memory 320, the input / output interface 330, and the communication interface 340 achieve communication connections with each other inside the device through the bus 350.

[0139] The processor 310 may be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided by the embodiments of this specification.

[0140] The memory 320 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, etc. The memory 320 may store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 320 and are called and executed by the processor 310.

[0141] The input / output interface 330 is used to connect to an input / output module to implement information input and output. The input / output module may be configured as a component in the device (not shown in the figure) or may be externally connected to the device to provide corresponding functions. Among them, the input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0142] The communication interface 340 is used to connect to a communication module (not shown in the figure) to implement communication interaction between this device and other devices. Among them, the communication module may implement communication in a wired manner (such as USB, network cable, etc.) or may implement communication in a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0143] The bus 350 comprises a pathway that transmits information between the various components of the device (eg, the processor 310 , the memory 320 , the input / output interface 330 , and the communication interface 340 ).

[0144] It should be noted that, although the above device only shows the processor 310, the memory 320, the input / output interface 330, the communication interface 340 and the bus 350, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0145] Based on the positioning method described in any of the above embodiments, the present disclosure also provides: Fig.17 The hardware structure diagram of the relative positioning system shown in the figure, at the hardware level, the movable object includes a body, a detection device and a relative positioning device, and of course may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the positioning method described in any of the above embodiments. In some embodiments, the movable object is a drone.

[0146] The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the relative positioning method provided in the above embodiment. Among them, the computer-readable storage medium can be an internal storage unit of the movable object described in any of the aforementioned embodiments, such as a hard disk or memory of the movable object. The computer-readable storage medium can also be an external storage device of the movable object, such as a plug-in hard disk equipped on the movable object, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc.

[0147] It should be understood that the terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0148] It will also be understood that the term "and / or" as used in this application and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0149] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A relative positioning method for cluster control of movable objects, characterized in that: include: Acquiring at least one frame of detection image of the first movable object to be detected and equipped with the optical marker by means of a detection device, wherein the detection device is arranged on the second movable object; Obtaining light identification information of the detected first movable object according to the at least one detection image frame, and obtaining identity information of the detected first movable object according to the light identification information; and The position and / or posture information of the second movable object relative to the detected first movable object is obtained according to the at least one frame of detection image.

2. The relative positioning method according to claim 1, characterized in that: Obtaining the light identification information of the detected first movable object according to the at least one detection image frame includes: the light identification is encoded according to a first preset rule, and the at least one detection image frame includes the light identification information of the light identification encoded according to the first preset rule.

3. The relative positioning method according to claim 2, characterized in that: Obtaining the identity information of the detected first movable object according to the optical identification information also includes: decoding the optical identification information included in the at least one frame of detection image according to a second preset rule to obtain the identity information of the detected first movable object.

4. The relative positioning method according to claim 1, characterized in that: Obtaining the identity information of the detected first movable object according to the optical identification information includes: The optical identification information at least includes identity information, and the identity information of the detected first movable object is obtained according to the optical identification information of the detected first movable object and a first identity database, and / or The optical identification information at least includes the identity information, position information and / or posture information of the detected first movable object. The detected first movable object broadcasts the optical identification information via optical communication to obtain the identity information of the detected movable object.

5. The relative positioning method according to claim 1, characterized in that: Before the step of acquiring at least one frame of detection image of the first movable object to be detected and carrying the optical marker by the detection device, wherein the detection device is arranged on the second movable object, the method further includes: Acquiring a frame rate of the detection device; Determining whether the frame rate of the detection device is greater than a first threshold; When the frame rate of the detection device is greater than a first threshold, the light mark and other information other than the light mark are encoded at different times according to different preset rules.

6. The relative positioning method according to claim 5, characterized in that: When the frame rate of the detection device is greater than the first threshold, encoding the light mark and other information other than the light mark at different times according to different preset rules includes: In a first shutter cycle, the light identification is encoded according to a first preset rule, and the light identification information includes at least identity information; In the second shutter cycle, the light identification is encoded according to a third preset rule, and the light identification includes current estimated position information and movement state information of the detected first movable object.

7. The relative positioning method according to claim 1, characterized in that: The obtaining, according to the at least one frame of detection image, the position and / or posture information of the second movable object relative to the detected first movable object comprises: The position and / or posture information of the second movable object relative to the detected first movable object is obtained according to the at least one frame of detection image and the identity information of the detected first movable object.

8. The relative positioning method according to claim 7, characterized in that: The obtaining, according to the at least one frame of detection image, the position and / or posture information of the second movable object relative to the detected first movable object comprises: Obtaining the position and / or posture information of the second movable object relative to the detected first movable object according to the identity information of the detected first movable object and the position and size of the optical identification information of the detected first movable object in at least one detection image, and / or The detection device includes at least two detection elements, which obtain the position and / or posture information of the second movable object relative to the first movable object being detected based on the identity information of the detected movable object and the optical identification information of the first movable object being detected photographed by the at least two detection elements.

9. The relative positioning method according to claim 8, characterized in that: The obtaining, according to the at least one frame of detection image, the position and / or posture information of the second movable object relative to the detected first movable object also includes: The position and / or posture information of the second movable object relative to the detected first movable object is obtained based on the position and size of the optical identification information of the detected first movable object in at least one detection image and the camera parameters of the detection device.

10. The relative positioning method according to claim 1, characterized in that: In the step of acquiring at least one frame of detection image of the first movable object to be detected and equipped with the optical marker by the detection device, before the detection device is arranged on the second movable object, the method further comprises: Get the current communication status; Determining whether the communication status satisfies a first condition, wherein the first condition at least includes that the number of frequency bands interfered with by network signals is less than a first interference threshold; If the communication status does not satisfy the first condition, the movable object is controlled to perform an emergency operation.

11. The relative positioning method according to any one of claims 1 to 10, characterized in that: The method further comprises: Determining whether a second condition is met, the second condition comprising: the number of the movable objects is less than a first number threshold, and the distance between the movable objects and the reference object is less than a first distance threshold; When the second condition is met, relative positioning is achieved according to the map and / or the relative positioning method; and / or Determining whether a third condition is satisfied, the third condition comprising: the number of the movable objects is less than a second number threshold, and there is relative motion between the movable objects; When the third condition is met, relative positioning is achieved according to ultra-wideband positioning and / or the relative positioning method.

12. The relative positioning method according to any one of claims 1 to 10, characterized in that: The optical marker is arranged on the body and / or the arm of the first movable object to be detected.

13. The relative positioning method according to any one of claims 1 to 10, characterized in that: The optical marker comprises a lamp, which is a visible light lamp or an invisible light lamp, which is an infrared or ultraviolet lamp, and the detection device is a visible light detector or a non-visible light detector.

14. The relative positioning method according to any one of claims 1 to 10, characterized in that: The light identification is a lamp composed of one or more light-emitting bodies, and the multiple light-emitting bodies are m×n array lamps, and m and n are positive integers, wherein m is the number of rows and n is the number of columns.

15. The relative positioning method according to claim 14, characterized in that: When m≤n and m=1, the detection device is a rolling camera, and the length direction of the pattern formed by the light mark on the detection device at least does not overlap with the scanning direction of the rolling camera.

16. The relative positioning method according to claim 14, characterized in that: When m>1 and m≤n, the detection device is a global camera or a rolling camera.

17. A relative positioning device, comprising a processor, characterized in that: The processor is configured to perform the following steps: Acquiring at least one frame of detection image of the first movable object to be detected and equipped with the optical marker by means of a detection device, wherein the detection device is arranged on the second movable object; Obtaining light identification information of the detected first movable object according to the at least one detection image frame, and obtaining identity information of the detected first movable object according to the light identification information; and The position and / or posture information of the second movable object relative to the detected first movable object is obtained according to the at least one frame of detection image.

18. A relative positioning system, characterized in that: include: The movable object includes a body and a detection device, and the detection device is fixedly or movably mounted on the body and is used to detect images; A relative positioning device, arranged on the fuselage, the relative positioning device is used to implement the relative positioning method according to any one of claims 1 to 16, and The remote control terminal is used to achieve communication connection with the relative positioning device of the movable object.

19. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor is enabled to implement: The steps of the relative positioning method as described in any one of claims 1 to 16.

Images